Andrii Bidochko

• Updated: July 10, 2026
• 7 min read

Formal Disco: Scalable Open-Ended Generation of Formally Verified Programs

Direct Answer

Formal Disco is a distributed orchestration framework that lets large language models (LLMs) collaboratively generate, verify, and evolve formally verified programs at scale. By turning verification‑aware code synthesis into an open‑ended, multi‑agent workflow, the system creates massive synthetic datasets that close the long‑standing data gap for languages such as Dafny, Verus, and Frama‑C.

Background: Why This Problem Is Hard

Formal verification offers the strongest safety guarantees for software, yet its adoption is throttled by two intertwined bottlenecks:

• Data scarcity: Verification‑oriented languages have tiny corpora of human‑written examples, making it difficult to train or fine‑tune LLMs that understand the subtle proof obligations these languages require.
• Feedback latency: Compilers and automated provers return dense, low‑level error messages that are hard for generic LLMs to interpret without specialized prompting or iterative correction loops.

Existing approaches either rely on hand‑crafted benchmarks (e.g., the SV‑COMP suite) that cover a narrow slice of real‑world programs, or they use static prompting tricks that cannot scale beyond a few dozen examples. As AI agents become capable of writing production code, the mismatch between code generation speed and verification reliability widens, creating a risk of “code‑fast, verify‑slow” pipelines that undermine trust in AI‑produced software.

What the Researchers Propose

The authors introduce Formal Disco, a modular, distributed system that coordinates three specialized LLM‑based worker types to produce verified programs from scratch and iteratively improve them:

Initiators

These agents ingest random README files, API docs, or informal specifications from open‑source repositories. Their task is to sketch a minimal, verification‑ready program that aligns with the high‑level intent of the source material.

Fixers

Fixers receive the initiator’s draft together with compiler and verifier diagnostics. Using these signals, they propose concrete edits—adding invariants, adjusting pre‑conditions, or refactoring code—to resolve proof failures.

Extenders

Once a program passes verification, extenders take the stable artifact and generate functional patches: new features, alternative implementations, or performance‑oriented variations. This step expands the dataset’s breadth without sacrificing correctness.

All interactions are logged as traceable “agent‑generated” records, which serve two purposes: (1) they provide a rich supervision signal for distilling stronger models, and (2) they enable self‑improvement loops where later generations learn from earlier successes and failures.

How It Works in Practice

The workflow can be visualized as a continuous pipeline:

1. Task broadcasting: A central dispatcher samples a random README or documentation snippet and publishes the task to a pool of initiators.
2. Program sketching: An initiator generates a candidate program in one of the supported verification languages (Dafny, Verus, Frama‑C).
3. Verification loop: The candidate is fed to a language‑specific verifier. Errors are captured and fed back to a fixer.
4. Iterative fixing: The fixer may run multiple rounds, each time narrowing the error set until the verifier reports success.
5. Extension phase: A verified program is handed to an extender, which proposes one or more patches that preserve proof obligations.
6. Trace recording: Every prompt, model output, and verification signal is stored in a structured log, forming a “synthetic proof trace.”
7. Distillation & fine‑tuning: The accumulated traces are used to fine‑tune a base LLM, creating a verification‑aware model that can act as a stronger initiator, fixer, or extender in future cycles.

What sets Formal Disco apart from prior data‑generation pipelines is its maximum entropy principle. By explicitly encouraging the model to produce high‑entropy (i.e., diverse) programs during supervised fine‑tuning, the system avoids mode collapse and ensures that the synthetic corpus covers a wide spectrum of algorithmic patterns, data structures, and proof techniques.

Evaluation & Results

The research team evaluated Formal Disco across three dimensions: dataset quality, model performance, and scalability.

Dataset Quality

• Generated 2.3 M verified programs: 1.1 M in Dafny, 0.8 M in Verus, and 0.4 M in Frama‑C.
• Entropy analysis showed a 27 % increase in syntactic variety compared to a baseline “single‑agent” generator.
• Manual inspection of a random 500‑sample subset revealed a 94 % correctness rate (verified proofs and functional intent alignment).

Model Performance

Fine‑tuned open‑source LLMs (7B‑parameter) were benchmarked against Claude Opus 4.5 on verification‑relevant tasks such as invariant synthesis and post‑condition generation. The distilled models matched or exceeded Claude’s scores on 8 out of 10 tasks, demonstrating that synthetic data can rival human‑curated corpora for training verification‑aware agents.

Scalability

Formal Disco was deployed on a Kubernetes cluster with 120 worker pods, achieving a sustained throughput of ~150 verified programs per minute. The system’s modular design allowed horizontal scaling of each worker class independently, confirming that the architecture can grow with demand.

Overall, the experiments validate that a coordinated multi‑agent system can produce high‑quality, diverse, and provably correct code at a scale previously unattainable for formal methods research.

Why This Matters for AI Systems and Agents

For practitioners building AI‑driven development tools, Formal Disco offers a concrete pathway to embed formal guarantees directly into the code‑generation loop:

• Reduced reliance on human‑written specs: By synthesizing verified programs from informal documentation, agents can bootstrap safety checks without exhaustive manual annotations.
• Improved evaluation metrics: Synthetic proof traces provide a rich, reproducible benchmark for measuring an agent’s verification competence, complementing traditional functional correctness tests.
• Modular orchestration: The three‑role design maps cleanly onto existing agent‑frameworks, enabling developers to plug in custom LLMs or domain‑specific verifiers.
• Enterprise integration potential: Organizations can feed internal API docs into initiators, automatically generate formally verified wrappers, and thus tighten security boundaries for critical services.

These capabilities align with the broader trend of UBOS platform overview, where AI agents are orchestrated through a unified workflow engine. By feeding Formal Disco’s synthetic datasets into UBOS’s Workflow automation studio, teams can create end‑to‑end pipelines that start with natural‑language requirements and finish with provably correct code ready for deployment.

Moreover, the availability of large, verified code corpora unlocks new research avenues for AI marketing agents that need to generate trustworthy scripts for data‑privacy compliance, or for Enterprise AI platform by UBOS customers seeking to automate compliance‑heavy software development.

What Comes Next

While Formal Disco marks a significant step forward, several open challenges remain:

• Cross‑language generalization: Extending the framework to languages beyond the current trio (e.g., Coq, Lean) will test the adaptability of the maximum entropy fine‑tuning regime.
• Human‑in‑the‑loop validation: Incorporating expert review at strategic checkpoints could further raise the trust ceiling for safety‑critical domains.
• Dynamic specification extraction: Automating the conversion of evolving API contracts into verification‑ready specifications would make the system responsive to rapid product changes.
• Resource efficiency: Reducing the compute overhead of repeated verification cycles, perhaps via learned proof‑prediction models, could lower the barrier for smaller teams.

Future research may also explore tighter integration with OpenAI ChatGPT integration to let conversational agents request verified code snippets on demand, or with ChatGPT and Telegram integration for real‑time, verification‑aware code assistance in collaborative chat environments.

In the meantime, the released datasets and fine‑tuned models are publicly available, inviting the community to build on top of Formal Disco’s foundation. By democratizing access to massive, high‑quality verification data, the work paves the way for a new generation of AI agents that can not only write code but also prove that the code is correct.

For a deeper dive into the methodology and experimental details, consult the original Formal Disco paper.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.