Dictionary of Applied Machine Learning
Typeset PDF version — the authoritative form of this entry
Python demo — a script that recomputes what this entry states and prints one line per check
A machine learning (ML) method is explainable if there is an effective way to explain its predictions. The method delivers an explanation along with every prediction, and that explanation is effective if it lets a human user comprehend how the features of a data point drive the prediction made for it. Explainability is always relative to a specific user (group): an explanation can be very effective for a specific user (group) but useless for another user (group). Explainability can be measured by comparing the predictions of a learned hypothesis to the prediction of a user before and after they are provided with an explanation.
B-linThe explainability of a
machine learning (ML) method is the level to which a human user can anticipate
the predictions it delivers, based
on the explanations it provides
(Colin et al., 2022; Jung and Nardelli, 2020). Explainability
thus includes the notion of an explanation: each
prediction is delivered with an explanation for
this specific prediction, such as the feature values
that drove it or, for image data, the relevant pixels
(see Fig. 1).
The definition asks for the existence of an explanation, not
for a particular one: an ML method is explainable as soon as
some explanation lets the user anticipate its
predictions. Explainability is therefore certified by
exhibiting one explanation that works, and a single
explanation the user cannot follow refutes nothing.
Explainability is user-relative: the same learned hypothesis
can be explainable for one user and inscrutable for another.
Using a probabilistic model for data generation allows measuring explainability by the conditional differential entropy of the predictions (Chen et al., 2018; Jung and Nardelli, 2020). The conditional differential entropy quantifies the uncertainty about the predictions that remains once the anticipations are known, so a smaller value means that the anticipations determine the predictions more tightly. In practice it is unknown and must be replaced by an estimator, e.g., a plug-in estimate computed from discretized predictions and anticipations on a test set.
B-explAn explanation raises explainability when the user can
reason with it.
Fig. 2 illustrates this for a
user who reasons in terms of linear maps and anticipates the
predictions of an opaque hypothesis (a
kernel method): without explanations, the anticipations
deviate strongly from the predictions. Given a local linear
approximation of the hypothesis around each data point
(cf.\ local interpretable model-agnostic explanations (LIME)), the anticipations match the predictions
almost exactly. The example presupposes that the user knows how to
apply a linear approximation: the anticipations improve only
because the user can evaluate the explanation for a
data point. A user who cannot is left where they started.
pythondemos/explainability.py
Regulation treats explainability as an ingredient of transparency. The EU AI Act, in its Article 13, requires that high-risk artificial intelligence systems (high-risk AI systems) are sufficiently transparent to enable deployers to interpret their outputs (Parliament and Union, 2024). For individual automated decisions, the right to explanation entitles an affected person to a clear and meaningful account of the role that an AI system played in the decision (Parliament and Union, 2024, Art. 86).
What counts as an automated decision is drawn widely: the Court of Justice of the European Union held that a credit information agency computing a person's ability to meet future payments already makes one, when a third party draws strongly on that value (Union, 2023). The Colorado Automated Decision-Making Technology Act of 2026 requires that the deployer give the affected consumer a plain-language description of the decision and of the role the technology played in it, together with the means to request further information (Assembly, 2026).
See also: explanation, interpretability, explainable artificial intelligence, explainable empirical risk minimization, local interpretable model-agnostic explanations, transparency, right to explanation, trustworthy artificial intelligence, regularization.
@misc{dictml_explainability,
author = {Jung, Alexander},
title = {explainability},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-21},
url = {https://dictionaryofml.org/terms/explainability.html}
}