Andrii Bidochko

• Updated: July 22, 2026
• 7 min read

ConceptSMILE: Auditing the Trustworthiness of Concept-Based Explainable AI

Direct Answer

ConceptSMILE is a model‑agnostic, perturbation‑based auditing framework that evaluates how trustworthy concept‑level explanations are for any black‑box AI system. By measuring how concepts shift when input regions are altered, it gives practitioners a quantitative “trust score” that can be used to decide whether a concept‑based XAI output is reliable enough for downstream decisions.

Background: Why This Problem Is Hard

Concept‑based explainable AI (XAI) promises to translate opaque model reasoning into human‑readable building blocks—think “blood vessels,” “optic disc,” or “tumor margin” in a medical image. While these high‑level descriptors are intuitively appealing, they suffer from two intertwined shortcomings:

• Unverified fidelity: A model may assign a high activation to a concept without the concept actually influencing the final prediction.
• Spatial ambiguity: Concepts derived from segmentation masks or language models often overlap, making it unclear which image region truly drives the concept.

Current XAI pipelines typically stop at generating the concept map and assume it is trustworthy. Researchers have proposed post‑hoc sanity checks—such as gradient‑based attribution or occlusion tests—but these methods focus on pixel‑level importance, not on the higher‑order semantics that concept‑based explanations aim to capture. As a result, developers of AI‑driven diagnostics, autonomous agents, or compliance‑focused systems lack a systematic way to audit the reliability of the very explanations they rely on.

What the Researchers Propose

ConceptSMILE extends the SMILE (Simple Model‑agnostic Interpretable Local Explanations) paradigm from feature‑level to concept‑level auditing. The framework consists of three logical layers:

1. Region perturbation engine: Systematically masks or corrupts localized patches of the input image.
2. Concept‑response measurement: After each perturbation, the framework records how the activation of each human‑understandable concept changes.
3. Local surrogate model: An XGBoost regressor is trained on the perturbation‑response pairs to approximate the local relationship between input regions and concept activations.

By combining these layers, ConceptSMILE produces a suite of reliability metrics—attribution accuracy, surrogate fidelity, faithfulness, stability, and consistency—each of which quantifies a different facet of trustworthiness.

How It Works in Practice

Step‑by‑Step Workflow

1. Input selection: The user supplies an image (e.g., a retinal fundus scan) and a set of pre‑defined concepts, which may come from a segmentation model like MedSAM or a vision‑language model (VLM) that maps text prompts to visual regions.
2. Perturbation generation: The framework creates a grid of overlapping masks (e.g., 32×32 pixel squares) and applies a neutral perturbation (blur, noise, or in‑painting) to each region, one at a time.
3. Concept response capture: For every perturbed image, the concept extractor recomputes the activation scores for all concepts. The difference between the original and perturbed scores constitutes the “concept‑response shift.”
4. Locality weighting: Shifts are weighted by the spatial proximity between the perturbed region and the concept’s original mask, ensuring that distant perturbations do not unfairly dominate the signal.
5. Surrogate fitting: The weighted response vectors become training data for an XGBoost regressor. The surrogate learns a compact, interpretable mapping from region identifiers to concept changes.
6. Metric extraction: Using the surrogate, the framework computes:
   • Attribution accuracy: How well the surrogate’s feature importance aligns with the ground‑truth concept mask.
   • Surrogate fidelity (R², weighted R²): The proportion of variance in concept responses explained by the surrogate.
   • Faithfulness: Correlation between concept change magnitude and the model’s overall prediction change.
   • Stability: Consistency of metrics across multiple random perturbation seeds.
   • Consistency: Agreement of metrics when the same concept is derived from different pipelines (e.g., MedSAM vs. VLM).

What Sets ConceptSMILE Apart

Traditional XAI audits treat concepts as static labels. ConceptSMILE, by contrast, treats concepts as dynamic functions of the input space, probing them with controlled perturbations and learning a local surrogate that can be inspected, visualized, and compared across pipelines. This makes the framework both model‑agnostic (it works with any black‑box predictor) and concept‑agnostic (it can audit any set of human‑defined concepts).

Evaluation & Results

Experimental Setup

The authors applied ConceptSMILE to a publicly available retinal fundus dataset used for diabetic retinopathy screening. Two concept extraction pathways were compared:

• MedSAM pathway: A medical‑domain segmentation model that produces pixel‑accurate masks for anatomical structures (e.g., vessels, optic disc).
• VLM pathway: A vision‑language model that generates semantic masks from textual prompts such as “blood vessels” or “macular region.”

For each pathway, the authors ran 1,000 perturbations per image, trained separate XGBoost surrogates, and computed the five reliability metrics.

Key Findings

• Attribution accuracy: MedSAM’s spatial masks aligned with surrogate importance maps 12 % better than the VLM masks, indicating sharper localization.
• Surrogate fidelity: The MedSAM surrogate achieved an R² of 0.8503 and a weighted R² of 0.8465, whereas the VLM surrogate hovered around 0.71, suggesting that MedSAM’s concepts are more linearly predictable under perturbations.
• Faithfulness: VLM concepts showed a stronger correlation (ρ = 0.68) between concept change and overall model prediction shift, implying that VLM captures features that the classifier relies on more directly.
• Stability: Under artificial artefacts (e.g., simulated sensor noise), VLM‑derived concepts maintained metric variance under 5 %, while MedSAM’s variance rose to 9 %.
• Consistency: When the same anatomical concept was extracted by both pipelines, the overlap in high‑importance regions was only 63 %, highlighting that different pipelines can produce divergent explanations for the same semantic idea.

Collectively, these results demonstrate that reliability is not a monolithic property; it varies across concepts, extraction methods, and environmental conditions. ConceptSMILE surfaces these nuances, giving practitioners a data‑driven basis for selecting or rejecting a particular concept set.

Why This Matters for AI Systems and Agents

For AI‑driven products—whether a diagnostic assistant, an autonomous drone, or a compliance‑monitoring chatbot—trust in the explanation layer is as critical as trust in the prediction itself. ConceptSMILE equips engineers with a quantitative audit that can be embedded into CI/CD pipelines, model registries, or real‑time monitoring dashboards. By flagging low‑fidelity concepts before they reach end‑users, teams can avoid costly misinterpretations, regulatory penalties, or loss of user confidence.

Moreover, the framework’s model‑agnostic nature means it can be paired with any existing inference engine. For example, a company building Enterprise AI platform by UBOS could integrate ConceptSMILE into its workflow automation studio, automatically generating trust scores for every new concept‑based model version. This creates a feedback loop: low‑trust concepts trigger a retraining or refinement step, while high‑trust concepts are promoted to production.

What Comes Next

While ConceptSMILE marks a significant step toward systematic XAI auditing, several open challenges remain:

• Scalability: Perturbation‑based audits are computationally intensive. Future work could explore adaptive sampling or gradient‑guided perturbations to reduce overhead.
• Cross‑modal concepts: Extending the framework to audio, text, or multimodal concepts will require new locality weighting schemes.
• Human‑in‑the‑loop validation: Aligning surrogate importance maps with expert judgments could further calibrate trust scores.
• Standardization: Establishing community benchmarks for concept reliability would help compare auditing tools across domains.

Researchers and product teams interested in pushing these ideas forward can start by reproducing the retinal fundus experiments, then iterating on surrogate models (e.g., using neural additive models) or perturbation strategies (e.g., style transfer). The open‑source nature of the underlying components—XGBoost, MedSAM, and popular VLMs—makes rapid prototyping feasible.

For a deeper dive into the original methodology, see the ConceptSMILE paper. Developers looking to embed auditing capabilities into their AI stacks may also explore the Workflow automation studio for building custom perturbation pipelines without writing low‑level code.

Published in the blog section of UBOS.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.