Andrii Bidochko

• Updated: June 21, 2026
• 8 min read

Tool Forge: A Validation-Carrying Toolchain for Governed Agentic Execution

Direct Answer

Tool Forge introduces a validation‑carrying toolchain that turns natural‑language capability intents into governed, sandbox‑verified tool artifacts, and then serves those artifacts to large‑language‑model (LLM) agents through a token‑efficient routing layer. By encapsulating intent, contracts, tests, and runtime evidence in a single “capsule,” the system dramatically reduces the context overhead for agents while guaranteeing that only vetted tools are executed.

Background: Why This Problem Is Hard

Enterprises are increasingly relying on LLM‑driven agents to automate operational work—calling APIs, moving files, and orchestrating multi‑step workflows. The “tool layer” that bridges an agent’s textual reasoning with concrete actions is traditionally built in one of two ways:

• Hand‑crafted integrations: Engineers write bespoke wrappers for each API, which leads to brittle code, duplicated effort, and limited scalability.
• Static schema catalogs: A flat list of JSON schemas is injected into the model’s context, allowing the agent to select a tool by name. This approach inflates prompt size, forces the model to memorize large catalogs, and offers no runtime guarantees that the selected tool behaves as advertised.

Both approaches suffer from a lack of governance. In regulated industries, an agent must not only pick the right tool but also prove that the tool complies with security policies, dependency constraints, and credential bindings before execution. Existing pipelines provide little to no automated validation, leaving organizations to rely on manual code reviews or ad‑hoc testing—processes that are error‑prone and costly.

Moreover, LLM prompts have strict token limits. Loading an entire catalog of dozens or hundreds of tool schemas can consume 30‑40% of the available context, leaving insufficient room for the actual user query and the model’s reasoning. This token bloat reduces response quality and increases latency, making real‑time agentic execution impractical at scale.

What the Researchers Propose

The authors present Tool Forge, a modular framework that treats each tool as a self‑contained capsule. A capsule bundles together:

• Human‑readable intent (what the tool is meant to achieve).
• A formal capability contract describing inputs, outputs, and side‑effects.
• The actual implementation (code, script, or API endpoint).
• A dependency policy that enumerates required libraries, runtime versions, and network permissions.
• Automated tests and runtime validation evidence that prove the tool meets its contract.
• Metadata for lifecycle state, credential bindings, and routing information.

Two core components orchestrate these capsules:

1. The Validation Pipeline: When a developer submits a new capability intent, the pipeline automatically generates the capsule, runs unit‑style tests in an isolated sandbox, and records cryptographic evidence of successful validation.
2. The Router: Instead of loading every tool schema into the LLM prompt, the Router provides intent‑scoped “tool sessions.” An agent asks the Router for a tool that matches a specific intent, receives a lightweight token reference, and then invokes the validated implementation directly.

This separation of concerns—validation vs. routing—enables agents to operate with a minimal, purpose‑driven context while still benefiting from rigorous governance.

How It Works in Practice

Step‑by‑Step Workflow

1. Capability Definition: A product manager writes a natural‑language description such as “extract customer sentiment from a CSV file.”
2. Capsule Generation: Tool Forge’s compiler parses the intent, infers required inputs/outputs, and scaffolds a capsule skeleton.
3. Implementation & Tests: Engineers fill in the implementation (e.g., a Python script using pandas) and author declarative tests that verify correct sentiment extraction on sample data.
4. Sandbox Validation: The capsule is executed in a secure container. If all tests pass, the system records a signed validation artifact and marks the capsule as “ready.”
5. Catalog Registration: The validated capsule is stored in a central registry with routing metadata (intent tags, version, credential requirements).
6. Agent Query: An LLM agent receives a user request, extracts the high‑level intent, and asks the Router for a matching tool session.
7. Token‑Efficient Routing: The Router returns a compact token that references the pre‑validated capsule, avoiding the need to embed the full schema.
8. Execution: The agent invokes the capsule via a sandboxed runtime, passing user data. The runtime checks credential bindings and dependency policies before execution.
9. Result Delivery: The tool returns structured output, which the agent incorporates into its final response to the user.

Key Differentiators

• Governance‑by‑Design: Validation evidence travels with the capsule, ensuring that only vetted tools can be called.
• Token Economy: By exposing intent‑scoped sessions, the Router reduces prompt size by >99%, freeing tokens for richer reasoning.
• Lifecycle Management: Capsules carry versioning and deprecation flags, allowing safe rollout and rollback of tool updates.
• Credential Isolation: Each capsule declares required secrets; the runtime injects them only when the policy permits, preventing credential leakage.

Evaluation & Results

The authors benchmarked Tool Forge on two fronts: routing efficiency and end‑to‑end generation fidelity.

Routing Benchmark

• Test Set: 83 distinct intent‑matching cases covering file manipulation, API calls, and data transformation.
• Metric: Micro‑F1 score measuring correct tool selection.
• Result: Aggregate micro‑F1 of 0.901, indicating that the Router reliably maps intents to the appropriate capsule.
• Context Reduction: Estimated tool‑context size dropped by 99.2% compared with naïve full‑catalog exposure.

End‑to‑End Generation Probe

• Scenario: 25 local‑tool tasks (e.g., CSV summarization, image resizing, email drafting).
• Outcome: Tool Forge generated 25 complete tool bundles, achieving a micro‑F1 of 0.940 against deterministic acceptance checks.
• Live Validation: 23 of the 25 bundles passed sandbox execution without errors, demonstrating practical viability.

These results are presented as an initial systems benchmark rather than a claim of state‑of‑the‑art performance. Nonetheless, they illustrate that a validation‑carrying pipeline can coexist with high‑accuracy intent routing while dramatically shrinking the token footprint.

Why This Matters for AI Systems and Agents

For organizations building AI‑driven assistants, the Tool Forge paradigm offers three concrete advantages:

• Reduced Prompt Bloat: Agents can focus on reasoning and user interaction instead of memorizing extensive tool catalogs, leading to faster inference and lower API costs.
• Built‑in Compliance: Validation artifacts serve as audit trails, satisfying regulatory requirements for traceability and security in sectors such as finance, healthcare, and government.
• Scalable Tool Onboarding: New capabilities can be added by simply submitting a natural‑language intent; the pipeline handles capsule creation, testing, and registration without manual schema updates.

These benefits align directly with the needs of modern AI orchestration platforms. For example, the UBOS platform overview emphasizes modular tool integration and governance, making Tool Forge a natural fit for extending its workflow capabilities. Likewise, the Workflow automation studio can leverage validated capsules to construct reliable, low‑latency pipelines that scale across enterprise workloads.

From a developer perspective, the capsule model reduces the cognitive load of maintaining separate test suites, documentation, and security policies. All of these artifacts are co‑located, versioned, and automatically enforced at runtime, freeing engineering teams to concentrate on higher‑level product features.

What Comes Next

While Tool Forge demonstrates a compelling proof‑of‑concept, several open challenges remain:

• Adversarial Routing: Malicious actors could craft intents that deliberately confuse the Router. Future work must incorporate robust intent verification and anomaly detection.
• Broader API Grounding: Extending the framework to support dynamic discovery of third‑party APIs (e.g., SaaS services) will require standardized capability contracts and automated credential provisioning.
• Sandbox Isolation Enhancements: Current container‑based sandboxes provide a baseline security guarantee, but more granular resource controls (e.g., CPU throttling, network egress filtering) are needed for high‑risk environments.
• Cross‑System Evaluation: Benchmarking against heterogeneous agent ecosystems (e.g., OpenAI ChatGPT, Claude) will help quantify the universal applicability of the routing model.

Addressing these gaps could unlock new use cases such as:

• Enterprise‑wide AI assistants that automatically comply with internal policy engines.
• Marketplace models where third‑party developers publish validated capsules, and organizations consume them without bespoke integration work.
• Real‑time AI‑driven customer support that safely invokes privileged tools (e.g., CRM updates) under strict governance.

Companies interested in experimenting with validated toolchains can start by exploring the Enterprise AI platform by UBOS, which already supports sandboxed execution and credential management. For smaller teams, the UBOS solutions for SMBs provide a lightweight entry point to adopt capsule‑based tooling without extensive infrastructure overhead.

In summary, Tool Forge offers a systematic, governance‑first approach to bridging LLM reasoning with real‑world actions. As AI agents become central to business processes, frameworks that combine validation, token efficiency, and modular routing will be essential to maintain trust, compliance, and scalability.

For a deeper dive into the technical details, read the original Tool Forge paper on arXiv.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.