Doktorandské kolokvium KAI - Aisha Suleiman Umar (18.5.2026)
v pondelok 18.5.2026 o 13:10 hod. v miestnosti I 9
Prednášajúci: Aisha Suleiman Umar
Názov: Formal Verification of the Consistency of Explainable AI in Cyber-Physical Systems
Termín: 18.5.2026, 13:10 hod., I/9
Abstrakt:
As Deep Learning (DL) models are increasingly integrated into the control loops of safety-critical Cyber-Physical Systems (CPS), such as smart grids, autonomous vehicles and industrial robotics, the black-box nature of these models poses a significant barrier to trust and regulatory compliance. While Explainable AI (XAI) aims to provide transparency, current post-hoc explanation methods are often heuristic, computationally unstable, and lack formal guarantees. Specifically, the consistency problem where small, non-semantic perturbations in input lead to radically different explanations, remains a critical vulnerability that can mislead human operators or mask adversarial attacks.
A crucial but underexplored aspect is the formal verification of the consistency of these explanations: ensuring that explanations are robust, reliable, and faithfully reflect model behavior across different scenarios. This research review synthesizes recent literature on the intersection of formal verification, explanation consistency, and XAI in CPS, highlighting current approaches, evaluation strategies, limitations, and open research directions.