Andrii Bidochko

• Updated: July 2, 2026
• 5 min read

MIRAGE: Stealthy Visual Prompt Injection for Vulnerability Detection in Web Agents

Direct Answer

MIRAGE introduces a stealthy visual‑prompt injection technique that can silently embed vulnerability‑detecting cues into web pages viewed by multimodal large language model (MLLM) agents. By doing so, it reveals a new class of vision‑based attack surface that challenges the security assumptions of modern AI‑driven browser automation.

Background: Why This Problem Is Hard

MLLM‑based web agents have become the de‑facto solution for tasks that require both visual perception and natural‑language reasoning—think automated form filling, visual QA, or dynamic content extraction. Their strength lies in processing raw screenshots or DOM renders through a unified transformer, allowing a single model to “see” and “act” on a webpage.

However, this integration of vision and language also expands the attack surface in two ways:

• Implicit visual cues: Agents trust the pixel data they receive, assuming it faithfully represents the underlying UI.
• End‑to‑end pipelines: The visual encoder is often a frozen diffusion or CLIP backbone, making it difficult to patch or audit for adversarial robustness.

Existing adversarial evaluations typically rely on overt perturbations—high‑contrast noise, visible patches, or pixel‑level modifications that are easy for a human observer to spot. While such attacks prove a concept, they do not reflect realistic threat actors who must remain undetected while probing a target system. Consequently, current defenses (e.g., input sanitization, model fine‑tuning) are evaluated against an unrealistically permissive threat model.

What the Researchers Propose

The authors present MIRAGE (Stealthy Visual Prompt Injection for Vulnerability Detection), a framework that crafts imperceptible visual prompts capable of triggering specific behaviors in MLLM agents. The core idea is to embed a hidden “signal” into a webpage’s rendered image such that the agent interprets it as a command or a diagnostic cue, without any noticeable artifact to a human user.

Key components of MIRAGE include:

• Diffusion‑based prompt generator: Uses a pretrained diffusion model to synthesize subtle pixel patterns that align with the visual distribution of the target page.
• Curvature‑aware guidance: Optimizes the prompt along the loss landscape of the agent’s visual encoder, ensuring maximal influence with minimal visual distortion.
• Sparse dark‑pixel residuals: Restricts modifications to a few low‑intensity pixels, further reducing perceptibility while preserving the agent’s trigger response.

How It Works in Practice

The MIRAGE workflow can be broken down into three stages:

1. Target profiling: The attacker queries the MLLM agent with a benign screenshot to extract the gradient of the visual encoder with respect to a desired output (e.g., “detect vulnerability”). This gradient informs where the model is most sensitive.
2. Prompt synthesis: A diffusion model, conditioned on the original webpage rendering, iteratively adds a sparse residual that follows the curvature‑aware direction. The process stops once a perceptual similarity threshold (e.g., SSIM > 0.98) is reached.
3. Injection & execution: The crafted image replaces the original rendering in the browser pipeline. When the MLLM agent processes the page, the hidden prompt activates a pre‑programmed response—such as reporting a security flaw or executing a diagnostic routine—while the human user sees an unchanged page.

What distinguishes MIRAGE from prior visual attacks is its focus on stealth rather than sheer success rate. By leveraging diffusion priors and curvature‑aware optimization, the injected pattern blends with the natural texture of the page, evading both human inspection and many automated detection tools.

Evaluation & Results

The authors evaluated MIRAGE against two representative MLLM agents:

• SeeAct: An open‑source visual‑language agent that combines a CLIP visual encoder with a GPT‑style reasoning module.
• OpenClaw (Clawdbot, MoltBot): A commercial web‑automation suite that relies on multimodal perception for dynamic interaction.

Testing scenarios included:

• Standard web forms (login, checkout, survey).
• Dynamic dashboards with charts and interactive widgets.
• Complex single‑page applications that load content asynchronously.

Key findings:

• Success rate: MIRAGE achieved a 92 % trigger success across all tested pages, comparable to overt adversarial patches but with a visual distortion score (LPIPS) an order of magnitude lower.
• Human detection: In a user study with 150 participants, fewer than 3 % reported noticing any visual anomaly, confirming the stealth claim.
• Robustness to defenses: Standard preprocessing (Gaussian blur, JPEG compression) reduced success only marginally (to 78 %), indicating that MIRAGE’s residuals survive common hardening pipelines.

These results demonstrate that a well‑crafted visual prompt can reliably influence MLLM agents without raising alarms, highlighting a practical vulnerability that existing security audits may overlook.

Why This Matters for AI Systems and Agents

For practitioners building AI‑driven web automation, MIRAGE forces a reassessment of the trust model placed on visual inputs. The technique shows that:

• Visual encoders can be hijacked without altering the DOM or injecting JavaScript, bypassing many traditional web‑security layers.
• Security testing frameworks must incorporate stealthy visual‑prompt generation to surface hidden attack vectors.
• Designers of multimodal agents should consider multi‑modal verification (e.g., cross‑checking visual cues against DOM semantics) to mitigate blind spots.

Integrating such defenses into existing pipelines can be streamlined through platforms that already support modular workflow orchestration. For example, the Workflow automation studio enables developers to insert visual‑sanity checks before an agent processes a page, while the OpenAI ChatGPT integration can be used to double‑validate agent outputs against a language‑only baseline.

What Comes Next

While MIRAGE establishes a compelling proof‑of‑concept, several open challenges remain:

• Generalization across models: The current approach tailors prompts to a specific visual encoder. Future work should explore universal prompts that transfer across architectures.
• Defensive synthesis: Developing generative defenses that can detect or neutralize hidden residuals in real time.
• Policy‑level mitigation: Embedding security policies that require agents to request explicit user confirmation when visual cues deviate from expected patterns.

Researchers and product teams can accelerate these efforts by leveraging existing AI infrastructure. The UBOS platform overview provides a sandbox for experimenting with diffusion models, while the UBOS templates for quick start include pre‑configured pipelines for visual‑prompt generation and detection.

References

MIRAGE paper on arXiv

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.