Agentic Multi-Source Grounding for Enhanced Query Intent Understanding – DoorDash Case Study

Direct Answer

The paper introduces an Agentic Multi‑Source Grounding framework that combines catalog retrieval and autonomous web‑search agents to resolve ambiguous, context‑sparse queries in a multi‑vertical marketplace. By emitting an ordered set of possible intents and applying a deterministic disambiguation layer, the system dramatically improves intent‑matching accuracy while remaining extensible to new domains and personalization signals.

Background: Why This Problem Is Hard

Marketplace platforms such as DoorDash host thousands of distinct business categories—restaurants, grocery items, retail products, and more. Users often submit short, vague queries like “Wildflower” that could refer to a restaurant chain, a floral arrangement, or a grocery product. Traditional classification pipelines face two fundamental challenges:

• Winner‑takes‑all bias: A single‑label classifier must pick one category, inevitably discarding legitimate alternative intents.
• Hallucination risk: Large language models (LLMs) trained on public data may generate results that do not exist in the platform’s inventory, leading to user frustration.

These failures are amplified for long‑tail queries that lack sufficient historical click or purchase signals. Existing solutions either rely on static rule‑based mappings—hard to scale—or on end‑to‑end neural classifiers that cannot be grounded in proprietary catalog data or real‑time web knowledge. The result is a persistent gap between user intent and the items presented, directly impacting conversion rates and user satisfaction.

What the Researchers Propose

The authors present a modular architecture called Agentic Multi‑Source Grounding (AMSG). At a high level, the system replaces the monolithic classifier with three cooperating agents:

1. Catalog Entity Retrieval Pipeline: A staged search over the platform’s own product and business catalog, returning a ranked list of candidate entities that match the query string.
2. Autonomous Web‑Search Agent: An LLM‑driven tool that issues a live web search when the catalog pipeline yields insufficient confidence, extracting up‑to‑date external references that may clarify the user’s intent.
3. Multi‑Intent Emission & Disambiguation Layer: Instead of a single label, the model outputs an ordered set of plausible intents. A deterministic rule engine—configurable with business policies and personalization signals—then selects the final intent or presents a disambiguation prompt to the user.

Crucially, the core inference model remains unchanged; grounding is achieved by feeding the retrieved entities and web snippets back into the LLM as context. This decoupling enables any marketplace to plug in its own data sources without retraining the underlying model.

How It Works in Practice

The end‑to‑end workflow can be visualized as a pipeline of four stages:

1. Query Reception: The user’s raw text query enters the system.
2. Catalog Retrieval: A fast, inverted‑index lookup returns the top‑k catalog entities (e.g., restaurant names, product SKUs). If the confidence score exceeds a predefined threshold, the pipeline proceeds to disambiguation.
3. Agentic Web Search (Cold‑Start Path): When catalog confidence is low, an autonomous agent constructs a web‑search prompt, executes the search via a safe API, and parses the top results for entities that match the query.
4. Intent Generation & Disambiguation: The LLM receives a combined context (catalog hits + web snippets) and emits an ordered list of intents, each annotated with a probability. A rule‑based disambiguation layer then applies business logic—such as “prefer in‑stock items” or “prioritize user’s past categories”—to select the final intent or trigger a clarification UI.

What sets this approach apart is the agentic nature of the web‑search component: it decides autonomously when to invoke external knowledge, how many results to fetch, and how to parse them, all without human‑in‑the‑loop supervision. This dynamic grounding mitigates hallucination while preserving the flexibility of LLM reasoning.

Evaluation & Results

The authors evaluated AMSG on DoorDash’s production search platform, focusing on two key metrics:

• Intent‑Matching Accuracy: The proportion of queries for which the system’s final intent aligns with human‑annotated ground truth.
• Conversion Lift: Measured as the increase in successful order completions after the intent is resolved.

Three experimental conditions were compared:

1. Baseline ungrounded LLM (no catalog or web grounding).
2. Legacy production classifier (single‑label, rule‑based).
3. Full AMSG system.

Key findings include:

• A 10.9‑percentage‑point improvement over the ungrounded LLM baseline.
• A 4.6‑percentage‑point gain relative to the legacy system.
• On long‑tail queries, catalog grounding contributed +8.3 pp, web‑search grounding added +3.2 pp, and the dual‑intent disambiguation layer provided an additional +1.5 pp.
• Overall accuracy reached 90.7 %—a 13.0 pp uplift over the baseline—while serving more than 95 % of daily search impressions in production.

These results demonstrate that grounding LLM inference in both proprietary and real‑time external sources can close the intent‑understanding gap without sacrificing latency or scalability.

Why This Matters for AI Systems and Agents

For engineers building AI‑driven search, recommendation, or conversational agents, the AMSG paradigm offers several practical advantages:

• Modular Extensibility: New data sources (e.g., partner catalogs, user‑generated content) can be added as separate agents without retraining the core model.
• Reduced Hallucination: By anchoring LLM outputs to verified entities, the system mitigates the risk of presenting unavailable inventory.
• Improved Long‑Tail Coverage: Autonomous web search fills knowledge gaps for rare or emerging queries, a common pain point in fast‑growing marketplaces.
• Policy‑Driven Disambiguation: Deterministic business rules ensure compliance with inventory constraints, regional regulations, or personalization strategies.
• Scalable Deployment: The architecture was proven at DoorDash’s scale, handling millions of daily queries with sub‑second latency.

Adopting a similar grounding strategy can help product teams deliver more reliable, context‑aware experiences while preserving the expressive power of foundation models. For deeper guidance on integrating agentic grounding into your platform, see our agentic grounding implementation guide.

What Comes Next

While the results are compelling, several open challenges remain:

• Privacy‑Preserving Web Access: Ensuring that autonomous agents respect user privacy and data‑use policies when querying external sources.
• Dynamic Policy Learning: Moving from static rule engines to learned policies that adapt to shifting business objectives.
• Cross‑Domain Generalization: Extending the framework to domains beyond marketplaces—such as healthcare or finance—where grounding sources differ dramatically.
• Robustness to Noisy Web Data: Developing better filtering mechanisms to avoid propagating misinformation from the open web.

Future research could explore hybrid retrieval‑augmented generation models that natively incorporate multiple grounding streams, or meta‑learning approaches that automatically calibrate when to invoke each agent. Companies interested in pioneering these next steps can partner with our team to prototype custom grounding pipelines; learn more at UBOS Partnerships.

References

For the full technical details, consult the original arXiv paper.