Andrii Bidochko

• Updated: July 11, 2026
• 7 min read

When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals

Direct Answer

The paper introduces a large‑scale empirical audit of two widely‑used confidence proxies for large language models (LLMs): self‑consistency (a model’s agreement with its own sampled answers) and cross‑model agreement (different models converging on the same answer). It shows that while agreement correlates positively with correctness, the signal is weak, highly regime‑dependent, and can be dangerously misleading for the most capable frontier models.

Background: Why This Problem Is Hard

Enterprises increasingly rely on “LLM‑as‑judge” pipelines to rank, filter, or even approve AI‑generated content. In practice, these pipelines often amplify a simple heuristic: if multiple independent judges (or multiple samples from the same judge) agree, the answer is probably right. This heuristic underpins everything from automated code review to legal‑document drafting assistants.

However, agreement is not a guarantee of truth. Models inherit systematic biases from their training data, share architectural quirks, and can fall back on surface‑level heuristics such as “choose the first option” or “prefer longer strings.” When several models share these hidden priors, they may converge on the same wrong answer, creating a false sense of confidence. Existing evaluation frameworks rarely quantify how often this happens, leaving practitioners without a reliable way to calibrate model‑driven decisions.

Compounding the issue, most prior work treats self‑consistency as a binary confidence score, ignoring the nuanced ways agreement interacts with model scale, prompting style, and task difficulty. Without a systematic, cross‑model analysis, it is impossible to know whether agreement is a trustworthy proxy in a given deployment scenario.

What the Researchers Propose

The authors propose a rigorous auditing framework that treats agreement as a conditional confidence signal rather than a universal truth indicator. Their approach consists of three conceptual components:

• Runner Cluster: A collection of independent “runners” (model‑prompt‑sampling agents) that each generate a fixed number of answer samples for a shared set of test items.
• Agreement Metric Suite: Quantitative measures that capture (a) self‑consistency (intra‑runner agreement) and (b) cross‑model agreement (inter‑runner convergence) across varying model tiers, prompting templates, and sampling temperatures.
• Hierarchical Bootstrap Evaluation: A statistical resampling technique that respects the natural clustering of items (e.g., questions from the same exam) and runners, yielding robust correlation estimates between agreement and ground‑truth correctness.

By decoupling the measurement of agreement from any single model’s output, the framework can reveal systematic over‑confidence, identify regimes where agreement is a useful proxy, and expose hidden failure modes that would otherwise remain invisible.

How It Works in Practice

The workflow can be visualized as a four‑stage pipeline:

1. Dataset Selection: Curated high‑stakes multiple‑choice benchmarks (GPQA Diamond and AIME) are chosen for their difficulty and well‑defined answer keys.
2. Runner Deployment: Fifty independent runners are instantiated for each model tier (e.g., mid‑tier, frontier). Each runner draws K = 50 answer samples per question, using a mix of zero‑shot, few‑shot, and chain‑of‑thought prompts.
3. Agreement Computation: For every question, the system calculates the proportion of samples that converge on each option. Self‑consistency is the dominant proportion within a single runner; cross‑model agreement is the proportion across all runners of the same tier or across tiers.
4. Bootstrap Correlation Analysis: A hierarchical, runner‑clustered bootstrap resamples both questions and runners to estimate the Spearman correlation (ρ) between agreement scores and the “majority‑correctness” label (the answer that would be deployed in production).

What sets this approach apart is the explicit modeling of uncertainty at two levels—sample‑level variance within a model and variance across models—combined with a resampling strategy that respects the natural grouping of test items. This yields a nuanced picture of when agreement is a reliable proxy and when it is merely a shared bias.

Evaluation & Results

The authors conducted a massive cross‑runner study involving 53 runners, 265,000 answer samples, and two distinct knowledge‑intensive benchmarks:

• GPQA Diamond: A 1,000‑question dataset of graduate‑level physics problems.
• AIME: The American Invitational Mathematics Examination, featuring challenging high‑school math questions.

Key observations include:

• Positive but modest correlation: Across all configurations, agreement correlated with correctness with Spearman ρ ranging from 0.20 to 0.59. Even the strongest correlation fell short of a reliable confidence estimator.
• Regime‑dependent usefulness: Mid‑tier models (e.g., 13‑B parameter class) showed the highest marginal gain from using agreement to allocate compute resources. In contrast, the most advanced frontier model displayed an alarming over‑confidence: it agreed with itself on 77 % of GPQA entries, yet 48 % of those “high‑agreement” cases were wrong.
• Cross‑family consistency: A supplemental analysis on three Claude tiers reproduced the frontier over‑confidence pattern, indicating that the phenomenon is not limited to a single provider.
• Bias amplification: High agreement often coincided with option‑position priors (e.g., a tendency to select answer “C”) rather than genuine knowledge, suggesting that models can collectively reinforce superficial heuristics.

These findings collectively demonstrate that agreement is a conditional, not absolute, proxy for correctness. It can be a useful heuristic for certain model families and task regimes, but it becomes a liability when applied indiscriminately to the most capable models.

Why This Matters for AI Systems and Agents

For practitioners building AI‑driven agents, evaluation pipelines, or autonomous decision‑making systems, the study delivers three actionable insights:

1. Calibration over simplification: Relying solely on self‑consistency as a confidence score can lead to systematic over‑trust in frontier models. Instead, integrate agreement metrics with calibrated probability estimates (e.g., temperature‑scaled softmax) to obtain a more balanced risk profile.
2. Dynamic compute budgeting: In environments where inference cost is a bottleneck, agreement can guide selective sampling—allocate additional model calls only when initial agreement falls below a calibrated threshold, a strategy that proved most effective for mid‑tier models.
3. Cross‑model orchestration: When constructing ensembles or “mixture‑of‑experts” panels, designers should be aware that high cross‑model agreement does not guarantee correctness. Diversifying model families (e.g., mixing OpenAI, Claude, and Anthropic) can mitigate shared biases, but the study shows even cross‑family agreement can be misleading at the frontier.

These considerations directly inform the design of robust AI pipelines on platforms such as the UBOS platform overview, where developers can configure multi‑model workflows, set agreement thresholds, and monitor confidence drift in real time.

Moreover, agents that interact with end‑users—like AI marketing agents—must surface uncertainty to users rather than presenting a single “confident” answer. Transparent confidence reporting can improve trust and reduce the risk of costly missteps in high‑stakes domains such as finance, healthcare, or legal services.

What Comes Next

While the audit provides a solid empirical foundation, several open challenges remain:

• Granular bias diagnostics: Future work should dissect which linguistic or positional heuristics drive spurious agreement, enabling targeted debiasing interventions.
• Real‑time confidence adaptation: Developing lightweight, on‑the‑fly calibration methods that adjust agreement thresholds based on observed task difficulty could make agreement‑based budgeting more responsive.
• Broader domain coverage: Extending the framework to include open‑ended generation tasks (e.g., summarization, code synthesis) will test whether the observed over‑confidence persists beyond multiple‑choice settings.
• Human‑in‑the‑loop validation: Integrating human feedback loops to verify high‑agreement but low‑accuracy cases could create a safety net for critical deployments.

Practitioners interested in experimenting with these ideas can start by leveraging the Workflow automation studio to prototype custom agreement‑based routing logic, or explore the OpenAI ChatGPT integration for rapid prototyping of multi‑model ensembles.

In summary, agreement remains a valuable signal—but only when treated as a conditional cue, calibrated against model tier, task difficulty, and known bias patterns. By embedding these nuanced insights into evaluation pipelines, AI teams can avoid the trap of over‑confidence and build more reliable, trustworthy systems.

References

Ding, K. (2026). When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals. arXiv preprint arXiv:2607.08065.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.