Andrii Bidochko

• Updated: June 13, 2026
• 6 min read

AI, Take the Wheel: What Drives Delegation and Trust in Human-Computer Cooperative Question Answering?

Direct Answer

The paper AI, Take the Wheel introduces a dual‑decision framework that separates delegation (when to let an AI act autonomously) from adoption (how to use AI‑generated suggestions) in cooperative question‑answering tasks. By measuring 387 delegation choices and 1,440 adoption decisions across 24 human‑AI matches, the authors reveal systematic over‑ and under‑reliance patterns that directly impact the effectiveness of human‑AI teamwork.

Background: Why This Problem Is Hard

Human‑AI collaboration promises higher productivity, but it hinges on a fragile trust relationship. In real‑world settings—customer support, medical triage, or data‑driven decision making—users must constantly decide whether to accept an AI’s answer, ignore it, or hand the problem entirely over to the system. Two intertwined challenges make this decision process difficult:

• Fallibility of AI models: Even state‑of‑the‑art language models produce confident yet incorrect answers, especially when faced with ambiguous or out‑of‑distribution queries.
• Human bias and metacognition limits: People often overestimate their own knowledge, fall prey to confirmation bias, and misinterpret model confidence cues.

Existing research typically treats reliance as a single binary choice—either the user follows the AI or not. This simplification obscures the nuanced ways users interact with AI: they may let the system answer without seeing the output (delegation) or they may evaluate a suggested answer before deciding (adoption). Moreover, most prior studies evaluate delegation and adoption in isolation, using synthetic tasks or crowdsourced participants that do not reflect expert decision‑makers in high‑stakes environments.

What the Researchers Propose

Gor et al. propose a **two‑stage reliance model** that captures both delegation and adoption decisions within the same experimental ecosystem. The framework consists of three conceptual components:

1. Human Agent: An expert participant who can answer a question directly, request AI assistance, or delegate the entire decision.
2. AI Agent: A question‑answering model that returns an answer together with a self‑reported confidence score and, optionally, an evidence‑grounded explanation.
3. Game Engine: A competitive QA game that scores teams based on correctness, encouraging participants to balance speed, accuracy, and trust.

The key insight is that delegation and adoption are **decoupled yet interdependent**: a user may delegate when they lack confidence, but later over‑rely on a misleading AI suggestion during adoption. By logging both decision types, the authors can quantify how confidence calibration, explanation quality, and human bias jointly shape trust dynamics.

How It Works in Practice

The experimental workflow unfolds in four steps:

1. Question Presentation

The game engine presents a factual question (e.g., “What year did the first moon landing occur?”). The human participant sees the prompt and must decide within a limited time window.

2. Delegation Decision

Before any AI output appears, the participant chooses one of three actions:

• Answer themselves – rely on personal knowledge.
• Ask for AI assistance – request a suggestion but retain final control.
• Delegate – hand the question entirely to the AI without seeing the answer.

This stage isolates pure trust: the user must predict the AI’s reliability without any evidence.

3. AI Generation (if requested)

When the human opts for assistance, the AI returns:

• An answer string.
• A confidence score (0–100%).
• A short, evidence‑grounded explanation (e.g., citation snippets or reasoning steps).

4. Adoption Decision

Armed with the AI’s output, the participant decides to:

• Accept the suggestion as‑is.
• Modify the answer based on the explanation.
• Reject the suggestion and answer independently.

The final answer is recorded, and the team’s score is updated. By repeating this loop across 24 matches (23 humans, 16 AI agents), the study captures a rich dataset of trust signals.

Evaluation & Results

The authors evaluate three core hypotheses:

1. Human‑AI teams outperform either party alone.
2. Mis‑calibrated confidence leads to systematic over‑ and under‑reliance.
3. Confirmation bias amplifies under‑reliance when AI suggestions align with a human’s incorrect intuition.

Key quantitative findings (rounded for readability):

• Combined human‑AI teams achieved a 12% higher accuracy than the best standalone AI and a 9% boost over the best human alone.
• Participants missed 3.9% of correct AI suggestions (under‑reliance) and followed 1.7% of incorrect AI outputs (over‑reliance).
• When AI confidence was near chance (≈50%) and humans disagreed, the model’s confidence signal provided no predictive value.
• Confirmation bias manifested strongly: in 64.5% of cases where the AI’s answer matched the human’s initial (incorrect) guess, participants failed to switch to the correct AI suggestion.

These results demonstrate that **trust calibration—not just raw model performance—is the limiting factor** for effective collaboration. Even a highly accurate AI can be under‑utilized if users cannot interpret its confidence, and a modestly accurate AI can cause costly errors when users over‑trust it.

Why This Matters for AI Systems and Agents

For product managers, AI engineers, and enterprise architects, the study offers concrete takeaways that go beyond model selection:

• Confidence must be calibrated to human perception. Simple probability scores are insufficient; they should be mapped to intuitive risk categories (e.g., “high confidence – likely correct”).
• Evidence‑grounded explanations reduce confirmation bias. When users see the reasoning behind an answer, they are more likely to override a mistaken intuition.
• Interface design should separate delegation and adoption. Providing a clear “delegate” button versus an “ask for help” option lets users express distinct trust levels.
• Continuous feedback loops improve trust. Logging delegation/adoption outcomes enables on‑the‑fly model re‑weighting or confidence adjustment.

These insights align with the capabilities of the UBOS platform overview, which supports modular AI components, confidence calibration widgets, and explanation APIs that can be wired into enterprise workflows. By embedding calibrated confidence displays and evidence snippets, developers can reduce both over‑reliance and under‑reliance, driving higher overall system accuracy.

What Comes Next

While the study makes a strong case for dual‑stage trust modeling, several open challenges remain:

• Scalability to open‑domain, multi‑turn dialogues. The current QA game is single‑turn; extending the framework to conversational agents will require tracking trust over longer interaction histories.
• Personalized trust profiles. Users differ in risk tolerance and domain expertise. Future systems could learn individual calibration curves and adapt delegation prompts accordingly.
• Cross‑modal explanations. Visual or auditory evidence (e.g., charts, voice‑over) may further mitigate confirmation bias, especially in data‑intensive domains.
• Real‑world deployment studies. Testing the framework in live customer‑support or medical triage settings will reveal operational constraints such as latency, privacy, and regulatory compliance.

Addressing these gaps will likely involve tighter integration of trust‑aware modules into end‑to‑end AI pipelines. The Enterprise AI platform by UBOS already offers orchestration tools for model monitoring, confidence recalibration, and explanation generation, making it a natural testbed for the next generation of human‑AI collaboration research.

In the meantime, practitioners can start applying the paper’s recommendations by:

1. Implementing a clear delegation toggle in UI/UX designs.
2. Mapping raw confidence scores to user‑friendly risk tiers.
3. Providing concise, citation‑backed explanations alongside AI answers.
4. Collecting delegation/adoption logs to fine‑tune model confidence over time.

By treating trust as a measurable, adjustable component rather than an afterthought, organizations can unlock the full potential of AI assistants, from AI marketing agents that draft campaign copy to decision‑support bots that help analysts interpret financial reports.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.