Andrii Bidochko

• Updated: June 13, 2026
• 7 min read

Plan Before Search: Search Agents Need Plan

Direct Answer

The paper Plan Before Search: Search Agents Need Plan introduces a “Plan” paradigm that forces retrieval‑augmented agents to decompose a complex query into an ordered list of sub‑questions before any document is fetched. By anchoring each search step to a pre‑designed sub‑question, the approach eliminates the drift caused by early, partially relevant results and consistently improves multi‑hop question answering across model sizes.

Background: Why This Problem Is Hard

Retrieval‑augmented reasoning agents have become the backbone of many enterprise AI products, from customer‑support bots to research assistants. These agents typically follow a “search‑then‑reason” loop: they retrieve documents, feed them to a language model, and let the model generate the next query. In practice, this loop suffers from two intertwined bottlenecks.

• Dependency Blindness: Complex questions often require a chain of sub‑skills (e.g., entity extraction → temporal reasoning → synthesis). Existing pipelines treat each retrieval step independently, ignoring the logical dependencies that tie sub‑tasks together.
• Reward‑Only Training: Reinforcement learning (RL) fine‑tunes agents using a single reward signal (e.g., final answer correctness). This signal does not guide the agent on how to structure its internal plan, leading to unstable learning and divergent failure modes across model families.

Because the agent’s next query is conditioned on the documents retrieved so far, a single off‑target document can cascade into a series of irrelevant searches, dramatically reducing answer quality. Moreover, the prevailing training recipe—distilling a stronger teacher model into a smaller student—assumes that the teacher already possesses a robust planning capability, which is not guaranteed.

What the Researchers Propose

The authors propose a two‑stage framework called Plan that explicitly separates “planning” from “search.” The core idea is simple yet powerful: before any retrieval occurs, the agent generates a structured list of sub‑questions that capture the logical flow needed to answer the original query. Each sub‑question then serves as a fixed anchor for a subsequent retrieval step.

Key components of the framework are:

• Planner Module: A language model that, given the original question, outputs an ordered plan of sub‑questions. The planner is trained to maximize coverage of the underlying reasoning graph.
• Searcher Module: A retrieval engine (e.g., dense vector search) that receives a sub‑question and returns the most relevant documents.
• Reasoner Module: The same or a different language model that consumes the retrieved documents and produces an answer to the sub‑question, feeding the result forward if needed.

By decoupling the generation of the plan from the noisy retrieval process, the system can enforce a disciplined reasoning path, reducing the chance of “drift” and making the RL signal more informative.

How It Works in Practice

The operational workflow can be visualized as a three‑phase pipeline:

1. Planning Phase: The agent receives a user query such as “What were the economic impacts of the 2020 oil price crash on renewable energy investments in Europe?” The Planner outputs a plan:
   1. Identify the timeline of the 2020 oil price crash.
   2. Find macro‑economic indicators for Europe during that period.
   3. Locate reports linking oil price fluctuations to renewable energy investment trends.
   4. Synthesize the findings into a concise answer.
2. Search & Reason Phase: For each sub‑question, the Searcher queries a vector database (e.g., Chroma DB) and returns top‑k passages. The Reasoner then processes those passages, producing a short answer or intermediate fact. The output of step (a) becomes context for step (b), and so on.
3. Aggregation Phase: Once all sub‑answers are generated, a final synthesis step combines them into the final response presented to the user.

What distinguishes this approach from prior “search‑first” pipelines is the strict ordering imposed by the plan. The agent never re‑generates a new query based on partially retrieved documents; instead, it follows the pre‑approved sub‑question list, ensuring that each retrieval is purposeful and aligned with the overall reasoning graph.

Evaluation & Results

The authors evaluated the Plan paradigm on three families of language models ranging from 3 B to 14 B parameters, using standard multi‑hop QA benchmarks (e.g., HotpotQA, ComplexWebQuestions). The experimental protocol compared four configurations:

• Baseline RL agents without planning.
• Distilled agents that inherit a plan from a stronger teacher.
• Plan agents trained from scratch with a handcrafted plan.
• Self‑bootstrapped Plan agents (the authors’ final recipe).

Key findings include:

• Consistent Accuracy Gains: Across all model sizes, Plan agents outperformed baselines by 7–12 percentage points in exact match scores, demonstrating that structured planning is beneficial regardless of scale.
• Divergent Failure Modes: Identical reward signals caused small models to collapse into “search‑only” behavior, while larger models suffered from “plan‑overfitting” (overly rigid sub‑questions). This highlighted the need for model‑specific feasibility conditions such as sufficient initial entropy.
• Self‑Bootstrapping Success: By using a tiny seed model (≈300 M parameters) to generate filtered trajectories that already contain a valid plan, the authors could activate Plan in any target model without external distillation. This method matched or exceeded the performance of teacher‑distilled variants.

These results matter because they prove that a modest, self‑contained planning step can replace heavyweight teacher‑student pipelines, simplifying the engineering stack for production agents.

Why This Matters for AI Systems and Agents

For practitioners building enterprise‑grade AI assistants, the Plan paradigm offers three immediate advantages:

• Predictable Behavior: By fixing the reasoning skeleton ahead of time, developers can audit each sub‑question for compliance, bias, or data‑privacy concerns before any external call is made.
• Modular Orchestration: The three‑phase design maps cleanly onto existing workflow automation tools. For example, a Workflow automation studio can orchestrate the Planner, Searcher, and Reasoner as distinct micro‑services, enabling fine‑grained scaling and monitoring.
• Cost Efficiency: Since the Searcher only runs when a sub‑question is ready, unnecessary API calls to large language models are avoided, reducing token consumption and cloud spend.

From a product perspective, the ability to embed a reliable plan into an AI marketing agent means that campaign‑generation bots can follow a verified sequence (audience analysis → channel selection → creative drafting → performance forecast) without drifting into off‑topic content. Similarly, the Enterprise AI platform by UBOS can expose Plan as a first‑class primitive, allowing large organizations to enforce governance policies at each planning step.

What Comes Next

While the Plan framework marks a significant step forward, several open challenges remain:

• Dynamic Re‑Planning: Real‑world queries often evolve mid‑conversation. Future work should explore mechanisms for the Planner to revise its sub‑question list on the fly without breaking the stability guarantees.
• Cross‑Modal Planning: Extending the approach to multimodal inputs (images, audio) will require planners that can generate sub‑tasks spanning different data types.
• Scalable Knowledge Integration: Integrating external knowledge graphs into the planning stage could further improve coverage for niche domains.

Developers interested in experimenting with Plan can start by reviewing the UBOS platform overview, which provides ready‑made connectors for vector stores like Chroma DB integration and voice synthesis via ElevenLabs AI voice integration. For startups looking to prototype quickly, the UBOS for startups page outlines pricing tiers and sandbox environments that support self‑bootstrapped training pipelines. Detailed cost estimates and licensing options are available on the UBOS pricing plans page.

Conclusion

The “Plan Before Search” paper demonstrates that a modest, structured planning step can dramatically improve the reliability and performance of retrieval‑augmented agents across model scales. By decoupling reasoning from noisy retrieval, the approach mitigates drift, stabilizes reinforcement learning, and eliminates the need for heavyweight teacher distillation. For AI engineers and product teams, the framework offers a clear path to more controllable, cost‑effective, and governance‑friendly agents—key ingredients for the next generation of enterprise AI solutions.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.