Explainable AI #
Explainable AI (XAI) refers to artificial intelligence and machine learning techniques that can provide human-understandable justification for their output behavior. Explainability, along with [[ai-alignment]] are important considerations in safe deployment of systems which depend on machine learning, and [[ai-legislation]] refers to both.
Explainability can be achieved through use of interpretable AI techniques, which allow people to directly see how the result was arrived at, or through post-hoc explanations which can provide a human interpretable explanation of a particular result. Decision Trees are an example of a technique which is traditionally thought of as interpretable, while Artificial Neural Networks are generally considered to be uninterpretable, they are sometimes referred to as opaque or black-box algorithms. In some cases generated explanations may not accurately reflect the actual calculations that lead to a result.
Upol Ehsan introduces Human Centered Explainable AI or HCXAI as a form of Explainable AI which takes account of the different contexts in which an explanation may be used and the different people who may need explanations. Simply describing the steps leading to a result may not be enough if it is so complicated that the person recieiving the explanation cannot understand.
Relationship with [[Interpretability]] #
Mechanistic Interpretability Research attempts to analyse the behaviour of previously uninterpretable algorithms, in particular neural networks so that their results can be interpreted. An article by Patrick Grady at the Center for Data Innovation highlights the difference between explainability and interpretability, and highlights that an algorithm can be explainable but uninterpretable where it is not feasible to give details of how a specific result was arrived at given a particular input.