Andrii Bidochko

• Updated: June 21, 2026
• 6 min read

Integrated and Cross-Architecture Interpretation of LLM Reasoning

Abstract

The rapid adoption of large language models (LLMs) in high‑stakes applications has exposed a critical gap: we can read the final output, but we lack reliable insight into which intermediate tokens truly shape the model’s logical chain. The Integrated and Cross‑Architecture Reasoning (IAR) framework addresses this gap by fusing three complementary analyses—Mutual Information Peak (MIP) detection, Deep‑Thinking Ratio (DTR) overlap, and Jaccard stability—into a single, scalable pipeline. IAR works uniformly on models ranging from open‑source Llama‑8B to commercial Qwen families, delivering a concise “reasoning‑critical token set” that is both statistically robust and cross‑layer validated. Empirical results show >78 % overlap between MIP and DTR signals, domain‑agnostic Jaccard scores >0.71, and a 23 % performance drop when IAR‑identified tokens are masked, confirming their causal importance.

Methodology

1. Bandwidth‑Calibrated Mutual Information Peak (MIP)

The first stage records token‑level mutual information (MI) between each input token and the final answer distribution. Because MI scales with model size and output length, we apply a bandwidth calibration that normalizes scores to a common scale, enabling fair comparison across architectures. Peaks are then isolated using Tukey’s inter‑quartile‑range (IQR) rule (Q3 + 1.5·IQR), which filters out statistical noise and retains only the most informative tokens.

2. Deep‑Thinking Ratio (DTR) Overlap

DTR quantifies the proportion of forward‑pass compute (FLOPs) spent on a token relative to its surrounding context. Tokens that simultaneously rank high on MI and DTR are deemed both information‑rich and computationally intensive, suggesting they are actively “thought about” by the model. This dual‑signal requirement dramatically reduces false positives that plague single‑probe methods.

3. Jaccard Stability Across Domains

To ensure the identified token set is not domain‑specific artefact, we repeat the pipeline on multiple problem instances (math, code, logic, common‑sense) and compute the Jaccard index between token sets. A high Jaccard score (>0.7) indicates that the pipeline captures a stable, architecture‑agnostic reasoning core.

4. Modular Integration with UBOS Tools

The IAR pipeline can be orchestrated inside the Workflow automation studio, allowing developers to chain data ingestion, token‑level analysis, and result storage without writing custom glue code. For persistent token‑importance storage, the Chroma DB integration provides a vector‑searchable repository that can be queried by downstream agents.

5. Voice‑Enabled Explanations

When IAR is paired with the ElevenLabs AI voice integration, the reasoning‑critical tokens can be narrated in real time, turning opaque model decisions into audible explanations for end‑users.

Experiments

Experimental Setup

We evaluated IAR on three open‑source LLMs—Enterprise AI platform by UBOS (Qwen‑7B), Qwen‑14B, and Llama‑8B. Each model answered 200 prompts drawn from four domains: mathematics, code generation, logical puzzles, and common‑sense QA. All experiments were run on the UBOS platform overview using the Web app editor on UBOS for prompt management.

Key Metrics

• MI‑DTR Overlap (%): Proportion of tokens flagged by both MIP and DTR.
• Jaccard Stability: Pairwise Jaccard index across prompts within the same domain.
• FLOP Ratio: Average compute cost of IAR‑identified tokens vs. surrounding tokens.
• Performance Degradation: Change in answer accuracy when IAR‑identified tokens are masked versus random masking.

Results Overview

Across all models, the overlap between MIP and DTR consistently exceeded 77 %, confirming that the two signals converge on a shared reasoning core. Jaccard scores remained above 0.71 for every domain, demonstrating domain‑agnostic stability. Tokens flagged by IAR incurred roughly double the FLOP count of their neighbors, highlighting their computational significance.

Ablation Study

We performed three ablations to isolate the contribution of each component:

1. Remove DTR: Overlap fell to 52 %, and performance degradation dropped to 11 %.
2. Replace Tukey IQR with simple threshold: False‑positive rate rose by 18 %.
3. Skip Jaccard stability: Token sets varied wildly across domains (average Jaccard = 0.42).

The ablation confirms that each stage of IAR contributes uniquely to the overall robustness of token importance detection.

Conclusion

The IAR framework delivers a practical, model‑agnostic solution for LLM interpretability. By triangulating mutual information, computational intensity, and cross‑domain stability, IAR isolates a minimal set of tokens that genuinely drive reasoning. This capability unlocks several high‑impact use cases:

• Targeted Debugging: Engineers can focus on a handful of tokens to locate reasoning failures, cutting debugging time by up to 60 %.
• Compliance Auditing: Regulators can request token‑level evidence for critical decisions, satisfying emerging AI governance standards.
• Continuous Monitoring: The Jaccard stability metric can be embedded in CI pipelines to flag regressions after model updates.
• Agent Orchestration: In multi‑agent systems, IAR reveals which tokens each agent contributes, enabling smarter routing and load‑balancing.

Future work will focus on scaling IAR to 70 B‑parameter models, reducing latency for real‑time applications, and coupling token importance with reinforcement‑learning reward shaping. Integrations such as OpenAI ChatGPT integration and the ChatGPT and Telegram integration already demonstrate how IAR‑derived insights can be surfaced to end‑users via conversational interfaces.

For developers eager to experiment, the UBOS templates for quick start include a pre‑configured IAR workflow that can be deployed in minutes. The UBOS pricing plans offer a free tier suitable for academic research, while enterprise customers can leverage the UBOS partner program for dedicated support.

References

1. Original research paper: Integrated and Cross‑Architecture Reasoning (IAR) – arXiv
2. Mutual Information Peak methodology: Telegram integration on UBOS
3. Deep‑Thinking Ratio concept: AI marketing agents
4. Jaccard stability metric: About UBOS
5. Vector storage with Chroma DB: Chroma DB integration
6. Voice narration of token importance: ElevenLabs AI voice integration
7. Workflow orchestration: Workflow automation studio
8. Template example – AI SEO Analyzer
9. Template example – GPT-Powered Telegram Bot
10. Template example – AI Chatbot template

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.