Andrii Bidochko

• Updated: July 31, 2026
• 7 min read

GRASP: Granularity-Aware Search Policy – A Deep Dive

Direct Answer

GRASP (Granularity‑Aware Search Policy) is a reinforcement‑learning framework that teaches autonomous agents to choose the right type of retrieval—semantic, keyword, or paragraph‑level—at each step of a multi‑turn reasoning process. By dynamically adjusting search granularity, GRASP boosts both evidence recall and answer accuracy, addressing a core weakness of today’s static or single‑step retrieval‑augmented generation (RAG) pipelines.

Background: Why This Problem Is Hard

Retrieval‑augmented generation has become the de‑facto method for grounding large language models (LLMs) in up‑to‑date facts. Traditional RAG, however, treats retrieval as a one‑shot, “fetch‑then‑generate” operation. In real‑world scenarios—legal research, technical support, or multi‑hop question answering—agents must:

• Decide when to issue a new search versus when to continue reasoning on existing context.
• Select the right retrieval modality (lexical keyword match vs. semantic similarity) based on the current information need.
• Control context granularity so that only the most relevant sentences are introduced, avoiding token bloat that can derail LLM reasoning.

Existing approaches typically fall into three camps:

1. Static RAG pipelines that retrieve a fixed set of documents before generation, ignoring the evolving query.
2. Prompt‑engineered agents that rely on hand‑crafted heuristics to trigger searches, which are brittle across domains.
3. RL‑based retrieval selectors that choose between “retrieve” or “not retrieve” but lack fine‑grained actions for different search tools.

These methods struggle with multi‑step reasoning because they either overload the model with irrelevant passages or miss critical evidence that appears only after an intermediate inference. The problem is especially acute for enterprise AI systems that must balance latency, cost, and answer fidelity.

What the Researchers Propose

The GRASP framework reframes retrieval as a set of three complementary actions that an agent can invoke at any reasoning turn:

Semantic Search Action

Leverages dense vector similarity to pull sentences or paragraphs that are conceptually related to the current query, enabling broad exploration of the knowledge base.

Keyword Search Action

Executes traditional lexical matching (e.g., BM25) to surface evidence that contains specific entities or terminology, useful for pinpointing factual nuggets.

Paragraph‑Reading Action

Allows the agent to “open” a previously retrieved paragraph and read it sentence by sentence, effectively performing a local verification step before committing the content to the answer generation.

These actions are orchestrated by a reinforcement‑learning policy that receives a composite reward reflecting answer correctness, evidence grounding, complementary tool usage, and turn efficiency. The policy learns to “sketch” a reasoning path—first casting a wide net with semantic search, then narrowing focus with keyword search, and finally confirming details through paragraph reading.

How It Works in Practice

At a high level, GRASP operates as a loop of three stages: Observe → Act → Update. The workflow can be visualized in the diagram below.

1. Observation (State Construction)

• The LLM produces an intermediate reasoning step (e.g., “The capital of X is needed”).
• System metadata—current turn count, previously retrieved snippets, and confidence scores—are encoded into a state vector.

2. Action Selection (Policy Decision)

• A lightweight policy network evaluates the state and outputs a probability distribution over the three actions plus a “no‑search” option.
• During training, actions are sampled; during inference, the highest‑probability action is chosen.

3. Retrieval / Reading Execution

• Semantic search queries a dense index (e.g., FAISS) and returns top‑k sentence candidates.
• Keyword search runs a BM25 query against an inverted index, returning exact‑match passages.
• Paragraph reading selects a previously retrieved paragraph and extracts the next unread sentence, mimicking a human skimming process.

4. State Update & Generation

• New evidence is appended to the LLM’s context window.
• The model generates the next reasoning token or final answer, and the loop repeats until a stop condition (answer confidence, max turns) is met.

What sets GRASP apart is the explicit granularity control: the agent never pulls an entire document unless the policy deems it necessary, dramatically reducing token waste and keeping the reasoning chain tight.

Evaluation & Results

The authors benchmarked GRASP on two multi‑hop QA datasets: HotpotQA (open‑domain) and ComplexWebQuestions. Both require chaining together at least two pieces of evidence from disparate sources.

Experimental Setup

• Base LLM: a 13B instruction‑tuned transformer.
• Retrieval back‑ends: a dense vector store (semantic) and a BM25 index (keyword).
• Baselines: (a) single‑step static RAG, (b) prompting‑based agentic RAG (no RL), (c) RL selector with only “retrieve vs. not retrieve”.

Key Findings

• Retrieval Recall: GRASP achieved a 12‑point lift in sentence‑level recall over static RAG, indicating that the policy successfully uncovered evidence that single‑shot retrieval missed.
• Answer Accuracy: Exact‑match scores rose from 48% (prompted agent) to 61% with GRASP, narrowing the gap to human performance.
• Turn Efficiency: The average number of reasoning turns dropped by 18%, showing that the policy learns to avoid unnecessary searches.

Qualitative analysis revealed interpretable behaviors: agents started with semantic search to explore the topic space, switched to keyword search when a named entity surfaced, and used paragraph reading for final verification. Ablation studies confirmed that removing any of the three actions degraded performance, underscoring the importance of granularity awareness.

Why This Matters for AI Systems and Agents

For practitioners building enterprise‑grade AI assistants, GRASP offers a concrete recipe to make agents both smarter and more economical. By limiting retrieval to the smallest useful granule, token costs are reduced—a critical factor when operating on commercial LLM APIs. Moreover, the policy’s interpretability (skimming vs. scanning) aligns with audit requirements in regulated industries.

Integrating GRASP‑style orchestration into existing platforms can be done without a full model retraining. For example, the UBOS platform overview already supports plug‑in retrieval modules; swapping in a granularity‑aware policy would immediately improve answer fidelity. Similarly, the Workflow automation studio can schedule the three search actions as discrete workflow steps, giving product teams visual control over the agent’s reasoning path.

From a business perspective, the ability to surface precise evidence on demand translates into higher trust for AI‑driven decision support, whether in finance, healthcare, or customer service. Companies can also leverage the AI marketing agents use case to generate campaign copy that is automatically backed by the latest market research, reducing the risk of outdated or inaccurate messaging.

What Comes Next

While GRASP marks a significant step forward, several avenues remain open:

• Scalability to massive corpora: Current experiments used a few million passages; extending to web‑scale archives will require hierarchical indexing and possibly multi‑agent coordination.
• Cross‑modal evidence: Incorporating tables, code snippets, or audio transcripts could broaden applicability to domains like scientific literature or multimedia QA.
• Human‑in‑the‑loop feedback: Allowing users to correct or endorse retrieved snippets could further refine the policy through online RL.
• Integration with voice agents: Pairing GRASP with the ElevenLabs AI voice integration would enable spoken assistants that ask clarifying questions only when needed, conserving both bandwidth and user patience.

Future research may also explore meta‑learning approaches where a single policy can adapt to new domains with minimal fine‑tuning, or combine GRASP with tool‑use frameworks that invoke external APIs (e.g., calculators, databases) alongside retrieval.

For developers eager to experiment, the OpenAI ChatGPT integration provides a ready‑made LLM endpoint that can be wrapped with the GRASP policy logic, offering a low‑friction path from prototype to production.

Finally, the full technical details, code, and reproducibility checklist are available in the original GRASP paper. As the community builds on this work, we can expect a new generation of agents that reason more like humans—searching broadly, narrowing focus, and verifying evidence before speaking.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.