Andrii Bidochko

• Updated: June 20, 2026
• 7 min read

ROVER: Routing Object-Centric Visual Evidence for Grounded Multi-Image Reasoning

Direct Answer

ROVER (Routing Object‑Centric Visual Evidence for Grounded Multi‑Image Reasoning) is a lightweight, learnable plug‑in that equips multimodal large language models (MLLMs) with a dynamic visual‑working space. By routing object‑level evidence across images and reasoning steps, ROVER dramatically improves both answer accuracy and grounding precision without inflating inference cost.

Background: Why This Problem Is Hard

Modern MLLMs such as GPT‑4‑V, LLaVA, and Qwen‑VL excel at stitching together language and vision, yet they still stumble when a task demands precise object grounding across several images or video frames. The core difficulty stems from three intertwined bottlenecks:

• Fragmented visual context. Traditional grounding pipelines inject cropped patches or region‑of‑interest (RoI) embeddings directly into the language stream. This isolates objects from the surrounding scene, weakening holistic understanding and inter‑object relationships.
• Scaling inference cost. Each additional RoI multiplies the token length and the amount of visual processing, leading to quadratic growth in latency for multi‑image queries.
• Supervision overhead. Adaptive selection mechanisms that decide “which visual token to attend next” often rely on fine‑grained bounding‑box labels or handcrafted heuristics, both of which are expensive to collect and brittle in the wild.

These constraints limit the deployment of MLLMs in real‑world agents that must reason over product catalogs, surveillance footage, or multi‑step visual workflows. A more efficient, object‑centric routing strategy that preserves global scene awareness is therefore a pressing research need.

What the Researchers Propose

The authors introduce ROVER, a modular plug‑in that can be attached to any transformer‑based vision‑language backbone (they demonstrate it with Qwen2.5‑VL‑7B). ROVER operates on three tightly coupled principles:

1. Context aggregation. At each reasoning step, a step‑specific token triplet—comprising a query token, a key token, and a value token—is generated. These tokens summarize the language context up to that point.
2. Object‑centric differential attention. The triplet drives a lightweight attention module that extracts object‑level cues from the current image, compressing them into a shared visual working space.
3. History‑aware routing. The working space retains a memory of previously attended objects across all images. The routing mechanism selectively re‑injects the most relevant historical evidence for the next reasoning turn.

Crucially, ROVER does not require external bounding‑box supervision; the routing policy is learned end‑to‑end via a standard supervised‑fine‑tuning (SFT) objective, augmented with a grounding‑aware loss (GRPO). This design keeps the parameter budget modest while delivering a global, object‑centric view of the visual world.

How It Works in Practice

The end‑to‑end workflow can be broken down into four conceptual stages:

1. Input Normalization

All images associated with a query are encoded by a frozen vision encoder, producing a dense grid of patch embeddings. No cropping or RoI extraction occurs at this stage, preserving the full scene context.

2. Token‑Triplet Generation

For each reasoning step, the language decoder emits a triplet token (query, key, value). These tokens are concatenated with the textual tokens and fed into a cross‑modal attention layer.

3. Differential Object Attention

The triplet guides a differential attention head that computes object‑level relevance scores across the entire image grid. High‑scoring patches are pooled into a compact object vector and stored in the visual working space.

4. History‑Aware Routing

The working space maintains a chronological buffer of object vectors from all previous steps and images. A lightweight routing network evaluates similarity between the current triplet and buffered vectors, selecting the most pertinent historical evidence. The selected evidence is then merged back into the language model’s context for the next token prediction.

The following illustration visualizes the token‑triplet loop and the routing of object evidence across images:

What sets ROVER apart is its ability to reuse visual evidence without re‑encoding images or expanding the token stream. The routing decision is learned, not hard‑coded, allowing the system to discover non‑obvious cross‑image relationships (e.g., “the same brand logo appears in two different product photos”).

Evaluation & Results

The authors evaluated ROVER on two demanding multimodal benchmarks:

• MM‑GCoT – a chain‑of‑thought dataset that requires step‑wise grounding across multiple images.
• VideoEspresso – a video‑question answering suite where each query spans several frames and demands temporal grounding.

Key findings include:

• On MM‑GCoT, ROVER‑augmented Qwen2.5‑VL‑7B achieved a +4.8 % absolute gain in answer accuracy and a +14.6 % boost in grounding accuracy over the baseline without ROVER.
• On VideoEspresso, the same model recorded an +8.6 % increase in answer accuracy, demonstrating that the routing mechanism scales from static images to dynamic video streams.
• When the VideoEspresso‑trained model was transferred to unrelated benchmarks (e.g., VQA‑X, OCR‑V2), it consistently outperformed the base model by an average of +4.7 %, confirming the generality of the learned routing policy.

Importantly, these improvements came with no additional inference latency compared to the baseline, because ROVER reuses existing visual embeddings and adds only a few lightweight attention heads.

Why This Matters for AI Systems and Agents

For practitioners building AI agents that must interpret complex visual environments—think e‑commerce recommendation bots, autonomous inspection drones, or multimodal customer‑support assistants—ROVER offers three concrete advantages:

1. Efficient multi‑image reasoning. Agents can now query across product catalogs or surveillance feeds without exploding token counts, keeping response times suitable for real‑time interaction.
2. Improved grounding fidelity. Higher grounding accuracy translates to more trustworthy explanations (“the red button on the left panel is the cause of the error”), which is essential for compliance‑heavy sectors such as finance or healthcare.
3. Transferable visual memory. The visual working space acts like a short‑term memory that can be fine‑tuned on one domain (e.g., video tutorials) and then reused in another (e.g., technical support), reducing data collection costs.

These capabilities align directly with the needs of the UBOS platform overview, where developers orchestrate multimodal agents using a plug‑and‑play architecture. By integrating ROVER as a reusable component, UBOS users can enrich their agents with object‑centric reasoning while preserving the platform’s low‑latency guarantees.

What Comes Next

While ROVER marks a significant step forward, several open challenges remain:

• Scalability to higher‑resolution inputs. The current differential attention operates on a fixed patch grid; future work could explore hierarchical routing that adapts to variable image sizes.
• Explicit temporal modeling. For long video sequences, a more sophisticated memory decay or attention‑based summarization could further improve long‑range grounding.
• Cross‑modal supervision. Incorporating weak signals such as click‑through data or eye‑tracking could reduce the reliance on fully annotated grounding datasets.

From an application standpoint, ROVER opens doors to new product categories:

• AI‑driven AI marketing agents that can reference specific visual assets across a brand’s media library while composing campaign copy.
• Automated Workflow automation studio pipelines that need to verify visual compliance (e.g., logo placement) across thousands of assets in a single pass.
• Edge‑deployed inspection bots that can retain a compact visual memory of previously seen defects, enabling faster anomaly detection without cloud round‑trips.

Researchers are encouraged to experiment with the open‑source implementation (available alongside the ROVER paper on arXiv) and to explore hybrid routing strategies that combine object‑centric and scene‑level cues.

In summary, ROVER demonstrates that a modest, learnable routing layer can unlock powerful, grounded multi‑image reasoning for large language models, delivering tangible gains for both academic benchmarks and real‑world AI agents.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.