Andrii Bidochko

• Updated: July 14, 2026
• 7 min read

Shift & Drift: A Zero-Shot Benchmark for Generalizable and Robust Autonomous Driving Motion Planning

Direct Answer

The paper Shift & Drift: A Zero‑Shot Benchmark for Generalizable and Robust Autonomous Driving Motion Planning introduces a dual‑track benchmark that evaluates motion planners under two orthogonal distribution‑shift axes: semantic shift across unseen urban topologies and state‑distribution drift caused by stochastic execution perturbations. By exposing planners to zero‑shot, cross‑city scenarios and realistic actuation noise, the benchmark reveals a trade‑off between imitation‑learning fidelity and closed‑loop resilience, offering the community a rigorous tool for measuring real‑world readiness.

Background: Why This Problem Is Hard

Autonomous driving systems must operate safely in an open‑world environment where road layouts, traffic participants, and vehicle dynamics constantly evolve. Current motion‑planning pipelines are typically trained on large, curated datasets such as nuPlan, which capture a narrow slice of the global driving distribution—mostly North American and Singaporean urban patterns. When a planner encounters a city with different lane markings, pedestrian densities, or cyclist behaviors, performance can degrade dramatically.

Two intertwined challenges make robust evaluation difficult:

• Semantic Shift: The visual and structural semantics of a road network (e.g., intersection geometry, signage conventions) differ across regions. Existing benchmarks rarely test planners on cities that were never seen during training, leaving a blind spot for cross‑regional deployment.
• State‑Distribution Drift: Even in familiar environments, small actuation errors—sensor latency, wheel slip, or controller noise—accumulate over time, pushing the vehicle into states that were not represented in the training distribution. Most offline metrics ignore this closed‑loop feedback loop.

Because most research focuses on in‑distribution (ID) metrics, the community lacks a systematic way to quantify how planners generalize to novel topologies or recover from execution perturbations. This gap hampers the transition from simulation‑only validation to real‑world, safety‑critical deployment.

What the Researchers Propose

The authors present Shift & Drift, a two‑track benchmark designed to stress‑test motion planners along the semantic and state‑distribution axes simultaneously.

Semantic Shift Track

They built a conversion pipeline that ingests the DeepScenario Open 3D aerial dataset—originally a high‑fidelity city‑scale point cloud—and translates it into the nuPlan simulation environment. This pipeline preserves road geometry, traffic‑light logic, and dynamic agent trajectories while re‑formatting them for nuPlan’s API.

State‑Distribution Drift Track

To emulate execution noise, the benchmark injects stochastic perturbations into the ego vehicle’s dynamics at each planning step. The noise is temporally correlated, mimicking realistic actuator drift, sensor jitter, and environmental disturbances. Planners are evaluated in a closed‑loop fashion, where each perturbed action influences the next observation.

Together, the two tracks create a comprehensive testbed that measures both “can the planner understand a new city?” and “can the planner stay safe when its own actions are imperfect?”

How It Works in Practice

The benchmark consists of four major components that interact in a loop:

1. Scenario Generator: Pulls a city‑scale map from the DeepScenario dataset, runs the conversion pipeline, and produces a nuPlan‑compatible scenario file.
2. Planner Interface: Exposes a standard API (plan(observation) → trajectory) so that any planner—imitation‑learning, reinforcement‑learning, or hybrid—can be swapped without code changes.
3. Noise Engine: Applies a configurable stochastic model to the vehicle’s control commands (steering, throttle, brake). The model can be tuned to represent different levels of actuation fidelity.
4. Metrics Aggregator: Collects safety (collision count, near‑misses), progress (distance covered, route completion), and robustness (drift magnitude, recovery time) statistics for each run.

During a test episode, the scenario generator feeds the current world state to the planner. The planner returns a trajectory, which the noise engine perturbs before execution in the simulator. The resulting state is fed back to the planner for the next step, forming a closed‑loop cycle that continues until the route ends or a safety violation occurs.

What sets this approach apart is the zero‑shot nature of the semantic shift: planners are never exposed to the target cities during training, yet they must operate on them directly. Moreover, the drift component is not a post‑hoc analysis; it is baked into the simulation loop, forcing planners to demonstrate genuine recovery behavior.

Evaluation & Results

The authors evaluated three representative planners:

• IL‑Planner: An imitation‑learning model trained on the nuPlan dataset (behavior cloning with data augmentation).
• RL‑Planner: A reinforcement‑learning agent trained via curriculum learning on the same dataset.
• Hybrid‑Planner: A combination of IL pre‑training followed by RL fine‑tuning.

Scenarios Tested

Shift & Drift includes 1,182 scenarios spanning four German cities (Berlin, Munich, Hamburg, Cologne) and San Francisco, each featuring dense pedestrian‑cyclist interactions. For the drift track, three noise levels (low, medium, high) were applied, with temporally correlated Gaussian perturbations to steering and throttle.

Key Findings

• Semantic Shift: The IL‑Planner achieved high success rates on ID benchmarks (≈ 92 % route completion) but dropped sharply on unseen German cities (≈ 58 %). Failure modes were dominated by unsafe pedestrian crossings and mis‑interpreted lane markings.
• State‑Distribution Drift: Under medium noise, the IL‑Planner’s collision rate rose by 27 % compared to the ID setting, and it exhibited persistent lateral drift that required manual intervention to correct.
• RL‑Planner Resilience: The RL‑Planner maintained a more graceful degradation. Even under high noise, its collision rate increased by only 12 % and route completion stayed above 80 % across all cities.
• Hybrid Advantage: The hybrid approach combined the IL‑Planner’s strong nominal performance with the RL‑Planner’s robustness, achieving the best overall trade‑off (≈ 75 % completion on shifted cities, modest drift impact).

These results demonstrate an empirical trade‑off: models that excel at mimicking human demonstrations may overfit to the training distribution, while reinforcement‑learning agents, trained to maximize long‑term reward under stochastic dynamics, tend to be more robust to both semantic and state‑distribution shifts.

Why This Matters for AI Systems and Agents

For practitioners building autonomous‑driving stacks, Shift & Drift offers a concrete, reproducible yardstick for two critical deployment questions:

• Generalization Across Geographies: Companies can now quantify how a planner trained on North American data will behave in European or Asian cities before costly field trials.
• Closed‑Loop Safety Under Real‑World Noise: By embedding actuation perturbations directly into the evaluation loop, developers can identify recovery strategies (e.g., fallback controllers, safety envelopes) early in the development cycle.

Beyond autonomous vehicles, the benchmark’s dual‑track philosophy can be adapted to any embodied AI system—delivery robots, warehouse automation, or aerial drones—where both environmental novelty and execution uncertainty are inevitable.

Integrating such rigorous testing into continuous integration pipelines aligns with modern UBOS platform overview practices, enabling teams to automate benchmark runs, track regression, and surface failure modes before they reach production.

What Comes Next

While Shift & Drift marks a significant step forward, several open challenges remain:

• Richer Sensor Modalities: Current scenarios rely on lidar‑centric perception. Adding camera‑based semantic segmentation could expose planners to visual domain shifts.
• Adaptive Noise Models: Real‑world disturbances are often non‑Gaussian (e.g., tire blowouts). Future work could incorporate physics‑based fault injection.
• Benchmark Expansion: Including rural or highway environments would test planners’ ability to handle vastly different traffic dynamics.
• Tooling Integration: Embedding the benchmark into a Workflow automation studio would let teams schedule nightly runs, compare versions, and generate automated reports.

Researchers interested in extending the benchmark can contribute new city maps via the open conversion pipeline, or propose alternative drift models that reflect specific vehicle platforms.

For organizations looking to adopt a robust evaluation framework today, the Enterprise AI platform by UBOS provides out‑of‑the‑box support for large‑scale simulation, data management, and result visualization, making it straightforward to plug in Shift & Drift scenarios and start measuring real‑world readiness.

Conclusion

Shift & Drift delivers a meticulously crafted, zero‑shot benchmark that simultaneously probes semantic generalization and closed‑loop robustness. By exposing the hidden fragilities of imitation‑learning planners and highlighting the resilience of reinforcement‑learning agents, the benchmark equips the autonomous‑driving community with a practical tool for moving beyond in‑distribution performance metrics. As the industry pushes toward global deployment, such rigorous stress‑testing will be essential for building trustworthy, safety‑critical motion‑planning systems.

Explore the benchmark, contribute new scenarios, and integrate the evaluation pipeline into your development workflow to future‑proof your autonomous driving solutions.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.