Andrii Bidochko

• Updated: August 22, 2026
• 8 min read

QV-PIC: Query-Aware Visual Position-Independent Caching for Efficient RAG Serving

Direct Answer

QV-PIC introduces a query‑aware, dual‑resolution visual caching framework that lets Retrieval‑Augmented Generation (RAG) systems reuse pre‑computed key‑value (KV) pairs across queries while preserving fine‑grained textual evidence. By combining low‑resolution global context with high‑resolution, relevance‑driven detail, QV-PIC cuts total time‑to‑first‑token (TTFT) by more than 80 % compared with naïve full‑prefill, delivering near‑text‑level quality with the speed benefits of visual compression.

Background: Why This Problem Is Hard

RAG pipelines have become the de‑facto method for grounding large language models (LLMs) in up‑to‑date knowledge. The typical workflow repeats three steps for every user query:

• Retrieve a set of relevant documents.
• Chunk the documents into manageable text spans.
• Prefill the LLM with the concatenated chunks before generating the answer.

When many queries target overlapping knowledge bases—think customer‑support bots, enterprise search assistants, or news summarizers—the same text chunks are prefixed over and over. This redundancy inflates compute costs, especially for transformer‑based models where each token incurs quadratic attention overhead.

Position‑Independent Caching (PIC) was proposed to break this cycle. PIC stores the KV activations that result from a chunk’s tokens, then reuses them regardless of where the chunk appears in the prompt. In theory, PIC eliminates the need to recompute the same chunk for every request.

In practice, two obstacles limit PIC’s impact:

1. Token volume. Textual chunks can still number in the thousands of tokens per query, so even cached KV matrices remain large and memory‑intensive.
2. Cache quality degradation. When the same KV cache is injected at a different position, subtle context mismatches arise, leading to hallucinations or loss of factual precision.

Researchers have tried to compress text into visual tokens—rendering paragraphs as images and feeding them to multimodal LLMs. Visual tokens are far fewer than raw text tokens, offering a tempting route to shrink cache size. However, the visual route introduces a new quality gap: the rendering process discards fine‑grained token‑level evidence, and the model’s visual encoder often struggles to align image patches with the original textual semantics. Existing PIC repair techniques focus on selective recomputation, which re‑introduces online latency and still cannot recover the lost textual detail.

What the Researchers Propose

QV-PIC (Query‑Aware Visual Position‑Independent Caching) tackles both the size and quality problems with a two‑pronged strategy:

• Offline dual‑resolution cache compilation. Instead of a single visual cache, QV-PIC builds two caches for each document chunk:
  • A low‑resolution visual cache that captures the global layout and coarse semantics of the rendered image.
  • A high‑resolution visual cache that preserves detailed token‑level information for the most query‑relevant portions of the chunk.
• Online query‑aware cache reuse. At inference time, the system scores each cached chunk against the incoming query, then selectively merges the low‑resolution global context with high‑resolution patches that exceed a cumulative relevance threshold. This dynamic composition respects the model’s native chat‑template prefix, eliminating the need for any on‑the‑fly recomputation.

Key components of the QV-PIC pipeline include:

1. Template‑Aligned Renderer. Generates images using the exact chat‑template prefix the LLM expects, ensuring that the visual cache aligns with the model’s native positional embeddings.
2. Dual‑Resolution Encoder. A multimodal encoder that processes both low‑ and high‑resolution images, producing separate KV caches that can be merged later.
3. Relevance Scorer. A lightweight cross‑encoder that estimates how much each chunk contributes to answering the current query.
4. Cache Composer. A runtime module that stitches together the appropriate KV slices based on relevance scores, preserving the original token order without additional forward passes.

How It Works in Practice

The QV-PIC workflow can be divided into three phases: preparation, scoring, and generation.

1. Preparation (Offline)

• Documents are split into fixed‑size chunks (e.g., 512 tokens).
• Each chunk is rendered twice:
  • Low‑resolution rendering (e.g., 224 × 224 px) captures the whole paragraph.
  • High‑resolution rendering (e.g., 448 × 448 px) focuses on sub‑regions identified by a heuristic tokenizer that flags named entities, numbers, and code snippets.
• The multimodal encoder processes both images, storing the resulting KV pairs in separate caches: KV_low and KV_high.
• All caches are indexed by chunk ID and stored in a fast key‑value store (e.g., Redis or Chroma DB).

2. Scoring (Online, per query)

• The incoming user query is embedded using the same LLM encoder.
• The relevance scorer computes a similarity score for every cached chunk, producing a ranked list.
• A cumulative relevance budget (e.g., 70 % of total score) determines which high‑resolution patches are worth pulling into the final prompt.

3. Generation (Online)

• The system assembles a prompt that starts with the model’s native chat‑template prefix.
• It inserts KV_low for all selected chunks, guaranteeing that the model sees the full context.
• For the top‑scoring sub‑regions, the corresponding KV_high slices replace the low‑resolution equivalents, restoring token‑level fidelity where it matters most.
• The composed KV matrix is fed directly into the transformer’s attention layers, bypassing the need to re‑tokenize or recompute the cached portions.
• The LLM then generates the answer, starting from the first token after the cached prefix.

What sets QV-PIC apart is that the entire cache reuse step is a pure lookup and merge operation—no additional forward passes, no gradient calculations, and no latency‑inducing recomputation. By aligning the visual cache with the model’s chat template, QV-PIC also eliminates the contextual mismatch that plagued earlier visual PIC attempts.

Evaluation & Results

The authors benchmarked QV-PIC on six heterogeneous RAG tasks, ranging from open‑domain QA to code documentation retrieval. The evaluation protocol measured two primary axes:

• Answer quality. Reported as F1 score against ground‑truth answers.
• Latency. Total time‑to‑first‑token (TTFT) measured from query receipt to the first generated token.

Key findings include:

In plain language, QV-PIC closed more than 90 % of the quality gap between visual and text PIC while delivering an order‑of‑magnitude speedup over full pre‑fill. The relevance‑aware high‑resolution patch selection proved especially effective on tasks that required precise numeric or code extraction, where the visual compression alone would have omitted critical symbols.

Beyond raw numbers, the experiments demonstrated that QV-PIC scales gracefully: as the document corpus grows, cache size grows linearly with the number of chunks, but the low‑resolution component keeps memory footprints modest. Moreover, the offline compilation step can be parallelized across GPUs, making the approach viable for production pipelines that refresh their knowledge base nightly.

Why This Matters for AI Systems and Agents

For engineers building AI agents that must answer queries in real time—whether in customer‑service chatbots, internal knowledge assistants, or autonomous research agents—the latency‑quality trade‑off is a daily pain point. QV-PIC offers a concrete path to break that trade‑off:

• Reduced infrastructure costs. By cutting KV recomputation, GPU utilization drops dramatically, translating into lower cloud spend.
• Higher throughput. Agents can serve more concurrent users without scaling out additional hardware.
• Preserved factual fidelity. The high‑resolution patches ensure that critical details (numbers, code snippets, legal clauses) survive the visual compression step.
• Seamless integration. Because QV-PIC works with the model’s native chat template, existing prompt‑engineering pipelines need only a cache‑lookup wrapper.

Practically, a team could plug QV-PIC into an UBOS platform overview deployment, combine it with the Chroma DB integration for fast cache retrieval, and expose the service through the Telegram integration on UBOS for end‑user access. The result is an AI agent that feels instantaneous while still grounding its answers in up‑to‑date, high‑quality evidence.

What Comes Next

While QV-PIC marks a significant step forward, several open challenges remain:

• Dynamic knowledge updates. Current caches are compiled offline; integrating incremental updates without full recompilation is an active research direction.
• Multimodal query handling. Extending the relevance scorer to handle image or audio queries could broaden QV-PIC’s applicability.
• Cache security. Storing KV pairs for proprietary documents raises confidentiality concerns; encryption‑aware caching is a promising avenue.
• Model‑agnostic visual encoders. QV-PIC relies on a specific multimodal encoder; generalizing to other architectures would increase adoption.

Future work may also explore tighter coupling between the relevance scorer and the LLM’s own attention patterns, allowing the model to request additional high‑resolution patches on the fly. Such a feedback loop could push visual caching even closer to the quality of pure text caching while retaining its speed advantage.

Enterprises interested in experimenting with QV-PIC can start by prototyping on the Enterprise AI platform by UBOS, which already supports multimodal model serving and cache management. Start‑ups may find the UBOS for startups program a low‑friction entry point, while developers looking for concrete examples can explore the Openclaw (Clawdbot, MoltBot) tools that showcase cache‑aware agent orchestration.

Call to Action

If you’re building RAG‑powered agents and want to shave latency without sacrificing answer quality, consider integrating QV-PIC into your stack. Visit the UBOS homepage for a full suite of AI infrastructure tools, or read the original QV-PIC paper for deeper technical details.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.