Andrii Bidochko

• Updated: July 1, 2026
• 7 min read

MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planning

Direct Answer

MAGNIFIED introduces a reinforcement‑learning fine‑tuning (RLFT) pipeline that transforms multimodal large language models (MLLMs) into motion‑planning agents for autonomous driving. By rewarding token‑level predictions that correspond to safe, efficient trajectories, the method bridges the gap between language‑centric pre‑training and the concrete planning objectives required on real roads.

Background: Why This Problem Is Hard

Autonomous‑vehicle (AV) systems must translate high‑dimensional sensor inputs—camera images, LiDAR point clouds, and map rasters—into a sequence of control actions that respect traffic laws, passenger comfort, and collision avoidance. Traditional planning stacks rely on handcrafted cost functions and optimization pipelines that are brittle when faced with novel scenarios.

Multimodal large language models have shown impressive semantic reasoning, but they are trained on next‑token prediction objectives. That objective encourages the model to mimic the statistical distribution of text, not to consider the downstream consequences of a predicted token chain. In a planning context, a single token error can cascade into an unsafe trajectory, while the model receives no direct signal that it has “given space” to a nearby cyclist or avoided an off‑road excursion.

Existing approaches to adapt LLMs for control either freeze the language backbone and add a downstream policy head, or fine‑tune with supervised trajectory data. Both strategies inherit the “imitation‑only” bias: the model learns to reproduce training trajectories but lacks a mechanism to explore alternative actions that might improve safety or efficiency. Moreover, supervised fine‑tuning (SFT) cannot easily incorporate complex, multi‑step reward structures such as “minimize lane changes while maintaining a minimum clearance from other agents.”

What the Researchers Propose

The MAGNIFIED framework reframes motion planning as a token‑level reinforcement‑learning problem. Instead of treating the language model as a static predictor, the authors attach a reward function to each generated token, where the token sequence is deterministically mapped to a vehicle trajectory. The core components are:

• Token‑to‑Trajectory Mapper: A deterministic decoder that converts a series of X‑Y coordinate tokens into a continuous path in the vehicle’s coordinate frame.
• Planning Reward Engine: A set of metrics (collision avoidance, lane‑keeping, off‑road penalty, overlap with other agents) that evaluate the decoded trajectory and produce a scalar reward per token.
• RL Fine‑Tuning Loop: Proximal Policy Optimization (PPO) or a similar policy gradient method updates the MLLM’s weights to maximize the cumulative token‑level reward.

By aligning the language model’s objective with planning‑centric rewards, MAGNIFIED enables the model to “think ahead” in token space, effectively learning to generate trajectories that satisfy safety and comfort constraints rather than merely echoing the training distribution.

How It Works in Practice

Step‑by‑Step Workflow

1. Input Fusion: The system ingests a rasterized bird’s‑eye‑view (BEV) of the surrounding environment, enriched with semantic layers (road geometry, traffic signs, dynamic agent positions). This visual context is tokenized and concatenated with a textual prompt describing the driving task (e.g., “navigate to the next intersection while yielding to on‑coming traffic”).
2. Initial Supervised Fine‑Tuning (SFT): A baseline MLLM is trained on a large corpus of paired BEV‑prompt and trajectory‑token examples. The objective remains next‑token prediction, establishing a strong language‑to‑trajectory mapping.
3. Token Generation: During RLFT, the model autoregressively emits tokens representing successive (x, y) coordinates. After each token, the mapper updates the partial trajectory.
4. Reward Computation: The planning reward engine evaluates the partial trajectory against the current traffic scene. Rewards are decomposed per token to provide fine‑grained feedback (e.g., a token that brings the vehicle too close to a pedestrian incurs a negative reward).
5. Policy Update: Using PPO, the model’s policy network is adjusted to increase the probability of high‑reward token sequences while preserving language fluency.
6. Iterative Refinement: The loop repeats across thousands of simulated episodes drawn from the Waymo Open Motion Dataset, gradually shifting the model’s distribution toward safer, more efficient plans.

Key Differentiators

• Token‑Level Credit Assignment: Unlike episode‑level RL where the reward is only observed at the end, MAGNIFIED distributes credit to each token, accelerating convergence.
• Preservation of Multimodal Understanding: The language backbone remains intact, allowing the model to still leverage rich semantic cues (e.g., “school zone”) while optimizing for motion.
• Seamless Integration with Existing Datasets: By treating trajectories as text, the approach can reuse large‑scale driving logs without bespoke annotation pipelines.

Evaluation & Results

The authors benchmarked MAGNIFIED on the Waymo Open Motion Dataset (WOMD), a public repository containing high‑resolution sensor data and expert‑annotated trajectories for diverse traffic scenarios. The experimental setup featured:

• Rasterized BEV images (256 × 256 pixels) encoding lane markings, drivable area, and dynamic agents.
• Tokenized trajectories expressed as a sequence of 0.5‑meter grid coordinates, capped at 40 tokens per episode.
• A baseline SFT model trained solely on next‑token loss.
• MAGNIFIED RLFT applied for 200k gradient steps using PPO.

Key performance indicators included:

• Overlap Rate: Frequency with which the planned path intersected the predicted future footprints of other agents.
• Off‑Road Rate: Percentage of trajectory points that fell outside the drivable lane network.
• Comfort Metric: Average lateral acceleration, reflecting smoothness of lane changes.

After RL fine‑tuning, MAGNIFIED achieved a **10.5 % reduction in overlap rate** and a **38.9 % reduction in off‑road rate** compared with the SFT baseline. Comfort metrics also improved modestly, indicating that the model learned to avoid abrupt maneuvers while respecting safety constraints. These gains demonstrate that token‑level reinforcement can reshape a language model’s behavior from pure imitation to proactive, rule‑aware planning.

Why This Matters for AI Systems and Agents

For practitioners building autonomous‑driving stacks, MAGNIFIED offers a blueprint for turning any powerful multimodal LLM into a planning‑aware component without discarding its linguistic strengths. The approach reduces the engineering overhead of hand‑crafting cost functions, because the reward design can be expressed in familiar planning terms (collision, lane‑keeping, comfort) and directly attached to the model’s output.

Beyond AVs, the token‑level RL paradigm can be transplanted to other domains where language models generate sequential control signals—robotic manipulation, drone navigation, or even workflow orchestration. By exposing the model to a reward signal that reflects real‑world constraints, developers can achieve safer, more reliable agents while still leveraging the zero‑shot reasoning capabilities of large pretrained models.

Organizations looking to prototype such agents can accelerate development on the UBOS platform overview, which provides a modular environment for integrating vision, language, and reinforcement‑learning pipelines.

What Comes Next

While MAGNIFIED marks a significant step forward, several open challenges remain:

• Scalability to Real‑Time Deployment: The current RL loop runs in simulation; translating the policy to on‑vehicle inference with strict latency budgets will require model compression and hardware‑aware optimization.
• Reward Shaping Complexity: Designing a comprehensive reward that balances safety, efficiency, and passenger comfort is non‑trivial. Future work could explore hierarchical rewards or multi‑objective RL to better capture trade‑offs.
• Generalization Across Domains: Extending token‑level RL to other modalities (e.g., 3‑D point clouds) or to multi‑agent coordination scenarios will test the limits of the current mapping strategy.
• Human‑In‑the‑Loop Feedback: Incorporating driver or passenger preferences as additional reward signals could personalize the planning behavior.

Researchers and product teams interested in exploring these directions may find synergy with Enterprise AI platform by UBOS, which supports large‑scale model training, simulation environments, and continuous deployment pipelines.

For a deeper dive into the original methodology, consult the MAGNIFIED paper on arXiv. The community is encouraged to reproduce the experiments, experiment with alternative reward formulations, and share findings that could accelerate the adoption of language‑driven planning across autonomous systems.

Call to Action

Ready to experiment with multimodal LLMs and reinforcement learning in your own projects? Explore the UBOS solutions for SMBs to get started quickly, or join the UBOS partner program for deeper collaboration and access to advanced tooling.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.