Andrii Bidochko

• Updated: June 12, 2026
• 7 min read

Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Information

Direct Answer

The paper introduces Judge‑Then‑Solve (JTS), a trajectory‑level control framework that forces large reasoning models to decide whether a question is answerable before they begin any problem‑solving steps. By turning abstention into an explicit control decision, JTS closes the “detection‑to‑abstention gap” and dramatically improves safe, efficient inference in domains where missing information can be catastrophic.

Background: Why This Problem Is Hard

Reasoning‑oriented language models have become adept at multi‑step chain‑of‑thought (CoT) generation, yet they still inherit a fundamental blind spot: when a query lacks sufficient evidence, the model can recognize the insufficiency but continue reasoning anyway. This mismatch—detecting a gap in information without translating that detection into a refusal—creates a dangerous “detect‑but‑don’t‑abstain” failure mode. In high‑risk settings such as medical diagnosis, legal advice, or financial compliance, an unsupported answer can cause more harm than a polite “I don’t know.”

Existing abstention mechanisms typically treat refusal as a final answer class, training models to output a special token after the reasoning chain is complete. Those approaches suffer from two intertwined limitations:

• Late‑stage detection: The model must first generate a full reasoning trajectory before it can decide to abstain, wasting compute on a path that will be discarded.
• Inconsistent signaling: Because abstention is learned as just another output token, the model often produces contradictory signals—e.g., it may flag “insufficient information” early on but later fabricate an answer to satisfy the final‑answer loss.

Consequently, the detection‑to‑abstention gap remains a persistent safety bottleneck, especially as reasoning models scale and are deployed in autonomous agents that must make rapid, trustworthy decisions.

What the Researchers Propose

Judge‑Then‑Solve reframes abstention as a control decision rather than a terminal output token. The framework consists of two tightly coupled modules:

1. Judge Module: A lightweight classifier that evaluates the incoming query (and any retrieved context) and emits a binary “answerability” commitment—Proceed or Abort.
2. Solve Module: The standard reasoning engine (e.g., a dense or Mixture‑of‑Experts transformer) that only activates when the Judge returns Proceed. If the Judge says Abort, the system terminates immediately, returning an explicit abstention signal.

The key insight is that the abstention decision is made before any multi‑step reasoning begins, ensuring that the model never expends resources on a trajectory it will later reject. To teach the system this discipline, the authors combine two training stages:

• Supervised warm‑up: The Judge learns from human‑annotated answerability labels on a curated corpus of under‑specified and well‑specified questions.
• Missing‑premise reinforcement learning (RL): The system is exposed to synthetic “missing premise” scenarios where the correct behavior is to abort. A reward function balances consistency (the Judge’s decision must match the final outcome) and length‑shaping (penalizing unnecessary reasoning steps).

How It Works in Practice

The operational flow of JTS can be visualized as a decision‑flow diagram ({{IMAGE_PLACEHOLDER}}). In a production pipeline, the process unfolds as follows:

1. Input Reception: An external request—e.g., a medical query submitted via a chatbot—arrives at the system.
2. Context Retrieval (optional): A retrieval module fetches relevant documents or knowledge‑base entries. The retrieved snippets are concatenated with the original query.
3. Judge Evaluation: The combined text is fed to the Judge. Using a binary cross‑entropy loss, the Judge predicts Proceed if it believes the evidence is sufficient, otherwise Abort.
4. Control Branching:
   • If Abort, the pipeline returns a standardized abstention response (e.g., “I don’t have enough information to answer safely”). No further compute is spent.
   • If Proceed, the Solve module receives the same input and generates a CoT reasoning chain followed by a final answer.
5. Post‑Processing & Logging: For auditability, the system logs the Judge’s decision, the reasoning steps, and the final answer. This traceability is crucial for compliance in regulated industries.

What distinguishes JTS from prior abstention methods is the early‑stage gating of the reasoning engine. By decoupling detection from generation, the framework eliminates the “self‑reflexive” behavior where a model continues to hallucinate after flagging uncertainty. Moreover, the reinforcement learning stage explicitly rewards the model for “knowing when not to know,” shaping a more disciplined inference policy.

Evaluation & Results

The authors evaluated JTS on two families of reasoning models—dense Transformers and large‑scale Mixture‑of‑Experts (MoE) architectures—across three benchmark datasets that deliberately contain under‑specified questions:

• MedQA‑Insufficient: A medical multiple‑choice set where many items lack a key premise.
• Logical‑Gap: Synthetic logical puzzles with missing statements.
• Real‑World QA‑Missing: A web‑derived QA corpus where retrieval often fails to surface a necessary fact.

Key findings include:

• Abstention@Detection (A@D) near saturation: JTS achieved >95% alignment between the Judge’s “insufficient” flag and the final abstention, whereas baseline models lingered around 60%.
• Inference efficiency gains: By terminating unanswerable queries after the Judge step, average compute per query dropped by 38% for dense models and 42% for MoE models.
• Reduced hallucination: On answerable but difficult questions, JTS lowered the rate of unsupported self‑reflections by 22%, indicating a more focused reasoning trajectory.
• Safety uplift: In the medical benchmark, the number of harmful “confident but wrong” answers fell from 12% to 2% after applying JTS.

These results demonstrate that the detection‑to‑abstention gap is not an abstract theoretical issue; it has measurable impact on both model reliability and operational cost. The experiments also confirm that the missing‑premise RL objective successfully reshapes model behavior without sacrificing accuracy on fully specified queries.

Why This Matters for AI Systems and Agents

For practitioners building autonomous agents, the ability to refuse safely is as important as the ability to answer correctly. JTS provides a concrete, plug‑and‑play control layer that can be inserted into any existing reasoning pipeline, turning a “maybe” into a decisive “no.” This has several practical ramifications:

• Agent orchestration: In multi‑agent workflows, a downstream component can trust the abstention signal as a hard stop, preventing cascading errors.
• Compliance & audit trails: The explicit Judge decision creates a verifiable checkpoint, simplifying regulatory reporting for sectors like healthcare and finance.
• Cost optimization: Early termination reduces GPU hours, which directly translates to lower cloud bills—critical for startups scaling AI services.
• User experience: Delivering a clear “I don’t have enough information” message is preferable to a confident but incorrect answer, preserving trust.

Enterprises that already leverage the UBOS platform overview can integrate JTS as a micro‑service within their Workflow automation studio, allowing business logic to react to abstention events automatically. Similarly, developers building conversational assistants can pair JTS with the OpenAI ChatGPT integration to ensure that the assistant never fabricates answers when the knowledge base is silent.

What Comes Next

While JTS marks a significant step forward, several open challenges remain:

• Generalization to multimodal inputs: Extending the Judge to evaluate image, audio, or video evidence could broaden safety guarantees for vision‑language agents.
• Dynamic premise discovery: Future work might enable the system to request additional information (e.g., a follow‑up question) instead of outright abstaining, turning “I don’t know” into “What else do you need?”
• Robustness to adversarial prompts: Ensuring that malicious users cannot trick the Judge into proceeding on deliberately under‑specified queries is an ongoing security concern.

From an application standpoint, the framework is ready for integration into real‑world products. Companies interested in deploying safe AI assistants can explore the AI marketing agents that already incorporate abstention logic for brand‑safe content generation. For teams focused on rapid prototyping, the UBOS templates for quick start include a pre‑configured Judge‑Then‑Solve module, lowering the barrier to experiment with answerability control.

Finally, the research community is encouraged to benchmark JTS against emerging “self‑refine” and “self‑check” paradigms, and to share datasets that explicitly label answerability. Open collaboration will be essential to turn abstention from a niche safety feature into a standard component of every reasoning system.

References

Gu, R., Li, J., Wang, Y., Yue, Y., Xiao, H., Chen, Y., Wang, Y., Guo, C., Wei, P., Gu, J., & Cao, Y. (2026). Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Information. arXiv preprint arXiv:2605.28070v1.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.