Reconciling Theory and Practice in Explainable Artificial Intelligence

Recent breakthroughs in Artificial Intelligence (AI) have revived a foundational challenge: how do we understand systems that exhibit intelligent behavior yet defy intelligible explanation? As deep learning models scale to unprecedented complexity, their opacity creates a widening chasm between performance and comprehension—a tension that is both epistemologically and practically urgent. This thesis confronts this challenge directly by asking: what constitutes a meaningful explanation in the landscape of contemporary AI? To address this, the thesis first establishes the necessary theoretical foundation. It dissects the question “what is an explanation?” by tracing its evolution across three dominant paradigms in Explainable AI (XAI): the causal, the mechanistic, and the generative. While each offers a unique lens — focusing, respectively, on inference, transparency, and communication — the thesis argues that none alone can reconcile epistemic rigor with practical utility. From their synthesis emerges a novel theoretical framework of explainability, which reframes explanation not as a model property, but as a dynamic epistemic relationship between a model, its representations, and human understanding. Theory, however, must be accountable to practice. The second part of the thesis, therefore, operationalizes this framework at what it terms the triple frontier of XAI pursuit: intelligibility, alignment, and faithfulness. Each frontier is explored through a high-stakes case study that grounds abstract concepts in tangible applications: intelligibility as an act of communication in medical imaging; alignment as knowledge-building in smart contract analysis with large language models; and faithfulness as a principle of design in industrial decision-making governance. This empirical work is underpinned by a key methodological contribution: a custom test suite for temporal attention mechanisms, which demonstrates that faithfulness—the truthfulness of an explanation to a model’s actual reasoning—can be rigorously measured rather than merely assumed. Synthesizing the insights from this journey from theory to practice, the thesis culminates in the Principle of Appropriate Complexity. This principle moves beyond the simplistic trade-off between accuracy and explainability, proposing that a model’s complexity should be actively governed rather than merely minimized to ensure epistemic responsibility and foster trust. By wedding theoretical reflection to empirical validation, this work ultimately reframes explainability as a constitutive dimension of intelligence itself—a guiding principle for the co-evolution of human and machine understanding.