- Updated: June 25, 2026
- 6 min read
Mind the Noise: Sensitivity of Transformer-based Interaction-Aware Trajectory Prediction Models to Noisy Data
Direct Answer
The paper “Mind the Noise: Sensitivity of Transformer‑based Interaction‑Aware Trajectory Prediction Models to Noisy Data” shows that state‑of‑the‑art transformer models for predicting the future paths of surrounding agents degrade dramatically when fed realistic sensor or V2X noise. Even modest perturbations can cut prediction accuracy by 30 %, while extreme but plausible noise can erase nearly three‑quarters of the performance gain.
Background: Why This Problem Is Hard
Autonomous vehicles (AVs) rely on trajectory prediction to anticipate the motion of pedestrians, cyclists, and other cars. Modern interaction‑aware predictors use attention‑based transformers to model the complex, time‑varying relationships among dozens of agents. These models have been benchmarked on pristine datasets such as Waymo Open Motion or Argoverse, where raw sensor streams are cleaned, objects are perfectly tracked, and V2X messages are assumed error‑free.
In the real world, however, perception pipelines introduce latency, mis‑detections, and localization drift. V2X communications—especially those received from low‑cost roadside units—add quantization errors, packet loss, and time‑stamp jitter. The gap between laboratory‑grade data and on‑road data creates a hidden vulnerability: a model that looks flawless in simulation can become unsafe when deployed.
Existing research has largely sidestepped this issue. Most papers focus on improving interaction modeling, multimodal output, or long‑range forecasting, while treating input states as immutable ground truth. Consequently, the community lacks systematic evidence about how noise propagates through transformer attention layers and how it ultimately harms safety‑critical decisions.
What the Researchers Propose
The authors introduce a systematic sensitivity analysis framework for transformer‑based trajectory predictors. Rather than redesigning the model architecture, they inject controlled, statistically realistic noise into the input state vectors (position, velocity, heading) and observe the downstream impact on prediction quality. The framework consists of three core components:
- Noise Generator: A parameterized module that adds Gaussian and uniform perturbations calibrated to match typical perception error distributions and V2X communication uncertainties.
- Baseline Transformer Predictor: A publicly available interaction‑aware model that employs multi‑head self‑attention to fuse agent histories and produce multimodal future trajectories.
- Evaluation Suite: A set of metrics—minimum average displacement error (ADE), final displacement error (FDE), and collision‑rate increase—that quantify degradation across low, medium, and high noise regimes.
By keeping the predictor unchanged, the study isolates the effect of input noise, providing a clear, reproducible benchmark for robustness.
How It Works in Practice
The workflow can be visualized as a three‑stage pipeline:
- Data Ingestion: Raw sensor detections and V2X messages are collected from a high‑fidelity dataset. Each agent’s state at time t is represented as a 6‑dimensional vector (x, y, vx, vy, heading, timestamp).
- Noise Injection: The Noise Generator samples perturbations from predefined distributions. For low‑noise scenarios, standard deviations are set to 0.05 m for position and 0.02 rad for heading; medium noise doubles these values, and high noise triples them, reflecting worst‑case urban conditions.
- Prediction & Scoring: The perturbed states feed the baseline transformer, which outputs a set of plausible future trajectories. The Evaluation Suite then compares these predictions against the ground‑truth trajectories (which remain noise‑free) to compute error metrics.
This approach differs from prior robustness studies that either retrain models on noisy data or add adversarial attacks at the pixel level. Here, the focus is on the *semantic* noise that AV systems actually encounter, making the findings directly actionable for engineers.
Evaluation & Results
The authors evaluated the framework on two industry‑scale datasets: Waymo Open Motion (100 km of urban driving) and Argoverse 2 (30 km of mixed traffic). They measured performance under three noise intensities:
| Noise Level | Average ADE Increase | Average FDE Increase | Collision‑Rate Spike |
|---|---|---|---|
| Low | ≈ 30 % | ≈ 28 % | + 12 % |
| Medium | ≈ 85 % | ≈ 80 % | + 35 % |
| High | ≈ 290 % | ≈ 275 % | + 78 % |
In plain language, a modest 0.05 m positional jitter already inflates the average displacement error by more than one‑third. When the noise reaches realistic worst‑case levels—such as those observed in dense city canyons with intermittent V2X packets—the error balloons to nearly four times the baseline, and the likelihood of predicting a collision‑course trajectory spikes dramatically.
These results are significant because they demonstrate a non‑linear relationship between input uncertainty and prediction reliability. Small improvements in sensor calibration or V2X packet filtering can therefore yield outsized safety benefits.
Why This Matters for AI Systems and Agents
For practitioners building autonomous driving stacks, the study delivers three immediate takeaways:
- Training Data Must Mirror Deployment Conditions: Relying solely on cleaned datasets creates a blind spot. Augmenting training pipelines with realistic noise—using the same generator described in the paper—can improve model resilience.
- Evaluation Protocols Need a Noise Dimension: Benchmarks should report performance across a spectrum of sensor fidelity, not just a single “clean” score. This aligns with safety standards that require worst‑case analysis.
- System‑Level Mitigation Is Essential: Even the most robust predictor benefits from upstream filtering, sensor fusion, and confidence‑aware V2X handling. Integrating a Workflow automation studio to orchestrate real‑time noise estimation can close the gap between perception and planning.
In the broader AI‑agent ecosystem, the findings echo a recurring theme: agents that act on imperfect observations must be designed with explicit uncertainty handling. Whether the agent is a self‑driving car, a warehouse robot, or a conversational AI interpreting noisy user input, the same principle applies—model performance is only as good as the fidelity of its inputs.
What Comes Next
While the paper establishes a clear vulnerability, several avenues remain open for exploration:
- Robust Architecture Design: Future work could embed noise‑aware attention mechanisms that weigh each agent’s confidence, similar to Bayesian transformers.
- Domain‑Specific Data Augmentation: Generating synthetic noisy trajectories that reflect city‑specific V2X error profiles could further narrow the simulation‑to‑reality gap.
- Cross‑Modal Fusion Strategies: Combining lidar, radar, and camera data with V2X messages in a unified uncertainty model may mitigate the impact of any single noisy source.
- Real‑World Field Trials: Deploying the sensitivity framework on a fleet of test vehicles would validate the laboratory findings and uncover additional failure modes.
Practitioners interested in prototyping these ideas can leverage the UBOS platform overview to spin up modular pipelines that integrate sensor simulators, noise generators, and transformer models without writing extensive boilerplate code. For teams focused on rapid experimentation, the UBOS templates for quick start provide pre‑configured notebooks that already include the noise‑injection utilities described in the study.
Finally, organizations looking to commercialize robust trajectory prediction should consider joining the UBOS partner program, which offers co‑development resources, dedicated support, and access to a community of safety‑critical AI engineers.
Visual Insight
The diagram below illustrates how increasing sensor noise warps the predicted future paths of surrounding agents. The clean baseline (left) shows tight, accurate trajectories, while the high‑noise case (right) spreads predictions widely, increasing the risk of unsafe maneuvers.
