Andrii Bidochko

• Updated: July 11, 2026
• 7 min read

Grounding Spatial Relations in a Compact World Model: Instruction Leakage and a Goal-Free Dynamics Fix

Direct Answer

The paper Grounding Spatial Relations in a Compact World Model: Instruction Leakage and a Goal‑Free Dynamics Fix introduces a compact, goal‑conditioned world model that can correctly ground spatial language (e.g., “put the red block left of the blue block”) while exposing a hidden “instruction leakage” problem and offering a goal‑free dynamics correction to eliminate it. This matters because it reveals a subtle source of over‑optimistic performance in language‑grounded agents and provides a practical fix that brings perception‑level grounding back into focus.

Background: Why This Problem Is Hard

Spatial relation grounding sits at the intersection of language understanding, perception, and planning. Real‑world agents—robots, virtual assistants, or game AI—must translate a textual instruction into a concrete arrangement of objects in a scene. The difficulty stems from three intertwined challenges:

• Reference sparsity: Natural language often uses relational descriptors (“left of”, “behind”) without naming every object, forcing the model to infer implicit anchors.
• World model compactness: Maintaining a full 3‑D representation of every object is computationally expensive; compact models aim to store only the information needed for the current goal.
• Evaluation leakage: When a model is conditioned on the goal sentence, it can inadvertently “cheat” by echoing the instruction rather than truly perceiving the scene.

Existing approaches—such as dense voxel grids, transformer‑based scene encoders, or hierarchical planners—either sacrifice scalability or rely on large amounts of supervision. Moreover, most benchmarks report high accuracy without probing whether the model truly grounds the relation or simply reproduces the instruction text. This hidden bias, termed “instruction leakage,” has gone largely unnoticed, leading researchers to overestimate the reliability of language‑grounded agents.

What the Researchers Propose

The authors present a three‑component framework designed to expose and fix instruction leakage while preserving the efficiency of a compact world model:

1. Goal‑Conditioned World Encoder (GCWE): A lightweight neural module that ingests raw visual observations and a textual goal, producing a concise latent representation that should contain only the information needed to achieve the goal.
2. Relation Read‑Out Head (RROH): A classifier that predicts spatial relations (e.g., “left of”, “above”) from the latent state. Its performance is the primary metric for grounding quality.
3. Goal‑Free Dynamics Module (GFDM): An auxiliary dynamics predictor that learns to forecast the next latent state without any access to the goal sentence. By training the GCWE to be useful for both goal‑conditioned and goal‑free predictions, the model is forced to separate pure perception from goal‑specific language cues.

Crucially, the framework introduces a diagnostic test: measuring relation read‑out accuracy when the goal sentence is replaced with a random string. A large drop indicates genuine grounding; a small drop signals leakage.

How It Works in Practice

The operational workflow can be broken down into four stages, each corresponding to a concrete component in a typical AI agent pipeline:

1. Observation Ingestion

The agent captures a RGB‑D frame of the environment. A convolutional backbone extracts a set of object‑level embeddings, each tagged with a class label (e.g., “red block”, “blue block”).

2. Goal Conditioning

The textual instruction is tokenized and passed through a transformer encoder. The resulting goal vector is concatenated with each object embedding, producing a goal‑aware latent map.

3. Compact World Modeling

The GCWE compresses the goal‑aware map into a fixed‑size latent vector z. This vector is deliberately small (e.g., 128 dimensions) to enforce compactness.

4. Dual‑Path Prediction

• Relation Read‑Out: The RROH reads z and outputs a probability distribution over spatial relations for each object pair.
• Goal‑Free Dynamics: The GFDM predicts the next latent vector z′ using only the current z, ignoring the goal. The loss from this prediction is back‑propagated into the GCWE, encouraging it to retain scene‑intrinsic information.

By jointly optimizing the relation read‑out loss and the goal‑free dynamics loss, the model learns to keep perception‑only features separate from goal‑specific cues. This dual‑path architecture is the key differentiator from prior work, which typically trains a single goal‑conditioned predictor and thus remains vulnerable to instruction leakage.

Evaluation & Results

The authors evaluate the framework on two synthetic tabletop environments that feature multiple colored blocks and a set of relational commands. The experimental protocol includes three test conditions:

1. Standard Goal‑Conditioned Test: The model receives the correct instruction and must predict the spatial relation.
2. Instruction Leakage Test: The same model is fed a random string instead of the true instruction; performance drop quantifies leakage.
3. Goal‑Free Dynamics Test: The GFDM’s prediction error is measured to ensure the latent space captures scene dynamics without goal information.

Key findings:

• When trained without the GFDM, the relation read‑out accuracy reaches 0.90 on the standard test but only falls to 0.68 under the leakage test—indicating that a large portion of the success comes from copying the instruction.
• With the goal‑free dynamics fix, standard accuracy remains high (0.88) while leakage accuracy drops dramatically to 0.31, confirming that the model now relies on visual perception.
• The GFDM predicts future latent states with a mean squared error comparable to a dedicated scene‑only predictor, demonstrating that the compact representation retains sufficient dynamics information.

These results matter because they show that a compact, goal‑conditioned world model can be both efficient and trustworthy, provided that a goal‑free dynamics regularizer is applied.

Why This Matters for AI Systems and Agents

For practitioners building AI agents that must interpret spatial language, the paper offers three actionable insights:

• Detect hidden leakage early: By swapping the goal sentence with a random token sequence during validation, engineers can surface over‑fitting to language rather than perception.
• Adopt dual‑path training: Incorporating a goal‑free dynamics loss forces the latent world model to encode genuine scene information, improving robustness when the language input is noisy or ambiguous.
• Maintain compactness without sacrificing fidelity: The 128‑dimensional latent vector demonstrated that agents can operate under strict memory or latency constraints—critical for edge devices, robotics, or real‑time game AI.

These principles translate directly into better‑behaving agents on the UBOS platform overview, where developers often combine language models with visual perception modules. By integrating a goal‑free dynamics component, teams can build more reliable “AI marketing agents” that understand spatial cues in visual ad creatives, or enhance the Workflow automation studio with vision‑aware triggers that react to object placement rather than just textual commands.

What Comes Next

While the study makes a compelling case for goal‑free dynamics, several open challenges remain:

• Real‑world complexity: The experiments use synthetic blocks; extending the approach to cluttered, occluded scenes with varied object categories will test the scalability of the compact latent space.
• Multi‑step instructions: Current work focuses on single relational commands. Future research should explore how the dynamics fix interacts with hierarchical planning over longer instruction sequences.
• Cross‑modal grounding: Integrating audio cues or tactile feedback could further stress‑test the separation between perception and language.

Potential applications include:

• Robotic manipulation systems that need to verify spatial constraints before acting, reducing the risk of collisions.
• Virtual environment editors where designers type relational commands and receive immediate, perception‑validated feedback.
• Augmented reality assistants that overlay instructions on real‑world objects while ensuring the grounding is based on visual data, not just textual parsing.

Developers interested in experimenting with the goal‑free dynamics fix can start by prototyping on the UBOS solutions for SMBs, leveraging the platform’s modular architecture to swap in a GFDM component alongside existing vision and language stacks.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.