Andrii Bidochko

• Updated: July 12, 2026
• 7 min read

Compete Then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond Imitation

Direct Answer

The paper introduces a compete‑then‑collaborate framework where four frontier‑level large language models (Claude, Codex‑GPT, Grok, Gemini) are first ranked by an execution‑based judge and then jointly construct a verifiable curriculum that teaches a smaller coding model (Qwen2.5‑Coder). This approach shows that pooling teacher answers for imitation is less effective than using the teachers to build a rigorously verified learning environment, leading to measurable gains on hard coding challenges.

Background: Why This Problem Is Hard

Large language models (LLMs) have become de‑facto teachers for downstream, smaller models. The prevailing paradigm—knowledge distillation or supervised fine‑tuning (SFT) on teacher‑generated data—assumes that more teacher output equals better student performance. In practice, three intertwined bottlenecks undermine this assumption:

• Quality ambiguity: Frontier models often produce multiple plausible solutions, and there is no objective way to decide which one is “best” without running the code.
• Judge bias: Existing multi‑teacher pipelines rely on a language‑model judge that is itself biased toward the style and content of its own outputs, inflating perceived teacher quality.
• Verification gap: Most distillation pipelines ignore execution‑level verification (unit tests, stdin‑stdout checks), so students may learn from syntactically correct but semantically wrong code.

These issues matter because enterprises increasingly depend on compact, high‑performing coding assistants for code completion, bug fixing, and automated testing. If the distilled student inherits systematic errors, downstream productivity suffers, and the promise of “tiny but mighty” AI agents remains unfulfilled.

What the Researchers Propose

The authors present a two‑stage compete‑then‑collaborate framework that re‑thinks the teacher‑student relationship:

1. Competition phase: Four frontier AI teachers independently solve a coding problem. Their solutions are fed to an execution‑based judge that runs the code against hidden unit tests and stdin‑stdout checks. The judge ranks teachers head‑to‑head, applying fairness controls (e.g., random ordering, time limits) to avoid systematic bias.
2. Collaboration phase: The top‑ranked teachers jointly construct a verifiable curriculum. This curriculum consists of problem statements, verified solutions, and associated test suites that the student can interact with. The curriculum is then used either for direct supervised fine‑tuning or as a reinforcement‑learning environment with verifiable rewards (RLVR).

Key components include:

• Frontier teachers: Claude, Codex‑GPT, Grok, Gemini—each representing a distinct architectural family.
• Execution‑based judge: A deterministic sandbox that evaluates correctness, not just plausibility.
• Student model: Qwen2.5‑Coder, a 7B/32B parameter code‑focused LLM.
• Curriculum generator: A coordination module that merges verified solutions into a coherent learning path.

How It Works in Practice

Conceptual Workflow

1. Problem ingestion: A coding prompt (e.g., “Implement a binary search tree”) is broadcast to all teachers.
2. Independent generation: Each teacher writes code without seeing the others’ outputs.
3. Execution verification: The sandbox runs each snippet against a hidden test suite. Pass/fail outcomes, runtime errors, and performance metrics are recorded.
4. Head‑to‑head ranking: Teachers are paired in a round‑robin tournament; the judge awards points for correctness, efficiency, and adherence to style constraints.
5. Curriculum assembly: The top‑ranked teachers contribute their verified solutions. The curriculum builder extracts reusable patterns, edge‑case handling, and test cases, forming a step‑by‑step learning module.
6. Student training: The student model either (a) undergoes SFT on the verified solutions or (b) interacts with the curriculum in an RLVR loop where each successful execution yields a reward signal.
7. Evaluation: After training, the student is re‑tested on a held‑out set of standard and competition problems.

Interaction Between Components

The execution‑based judge acts as the impartial arbiter, ensuring that teacher ranking is grounded in observable behavior rather than linguistic similarity. The curriculum generator treats the ranked teachers as co‑authors, merging their strengths (e.g., Gemini’s algorithmic depth, Claude’s prompt engineering) into a single, verifiable teaching artifact. This separation of “competition” (to surface the best solutions) and “collaboration” (to synthesize a curriculum) is what differentiates the framework from traditional distillation pipelines that simply concatenate all teacher outputs.

Evaluation & Results

Test Scenarios

The researchers evaluated the framework on two families of coding benchmarks:

• Standard problems: 100 tasks from the MBPP (Mostly Basic Programming Problems) suite, which are well‑studied and have publicly available test cases.
• Competition problems: 40 novel, harder challenges sourced from recent coding contests (e.g., Codeforces, AtCoder) that are not part of any public benchmark.

Key Findings

• Teacher saturation on standard tasks: After self‑correction, all four teachers achieved near‑perfect scores (99‑100%). This indicates that for well‑defined problems, the execution‑based judge cannot meaningfully differentiate teacher quality.
• Differentiation on competition tasks: Gemini led with a 77% pass rate, Claude and Codex‑GPT tied at 69%, and Grok lagged at 50%. The competition phase therefore provides a reliable signal for selecting the most capable teachers on novel problems.
• SFT on verified solutions can hurt performance: Fine‑tuning Qwen2.5‑Coder on the verified curriculum reduced its MBPP‑test accuracy from 76.7% to 72.7% and its competition‑problem success from 5.9% to 2.9%.
• RLVR yields a 49% relative gain on hard problems: When the same curriculum was used as a reinforcement‑learning environment with verifiable rewards, the student’s competition‑problem success rose from 5.9% to 8.8% (peak), demonstrating that learning by doing—rather than pure imitation—extracts more value from the teachers.

These results collectively suggest that the value of multiple frontier teachers lies not in aggregating their answers for direct imitation, but in using them to construct a rigorously verified learning ecosystem where the student can practice, receive feedback, and improve iteratively.

Why This Matters for AI Systems and Agents

For practitioners building AI‑driven coding assistants, the study offers three actionable insights:

1. Verification‑first pipelines: Embedding execution‑level checks early in the data generation process prevents the propagation of syntactic but semantically incorrect code into student models.
2. Curriculum‑centric training: Treating teacher outputs as a curriculum rather than a static dataset enables reinforcement‑learning strategies that reward actual problem‑solving, aligning model objectives with real‑world developer workflows.
3. Orchestrated teacher ensembles: Deploying a competition stage to rank frontier models ensures that only the most capable solutions shape the curriculum, reducing noise and computational waste.

These principles dovetail with emerging enterprise AI platforms that emphasize UBOS platform overview for orchestrating heterogeneous model ensembles, and with Workflow automation studio for building verification‑driven pipelines. Moreover, the ability to generate a verifiable curriculum on‑the‑fly opens the door for AI marketing agents that can self‑improve by solving domain‑specific scripting tasks without human‑in‑the‑loop supervision.

What Comes Next

While the compete‑then‑collaborate framework marks a significant step forward, several limitations and open questions remain:

• Scalability of the execution sandbox: Running thousands of teacher solutions through a secure sandbox can be resource‑intensive. Future work could explore lightweight static analysis proxies that approximate execution outcomes.
• Curriculum diversity: The current curriculum focuses on single‑step problem‑solution pairs. Extending it to multi‑step projects, code refactoring, and documentation generation would broaden its applicability.
• Student model size vs. curriculum complexity: The study used a 7B/32B student. Investigating how smaller or larger students benefit from the same curriculum could inform model‑size‑aware training strategies.
• Generalization beyond coding: The verification‑centric approach could be adapted to other domains (e.g., data analysis, scientific computing) where unit tests or simulation results provide an objective correctness signal.

Addressing these challenges will likely involve tighter integration with platform‑level orchestration tools, such as the Enterprise AI platform by UBOS, which can provision sandbox resources on demand and manage curriculum versioning at scale. Researchers may also experiment with hybrid reward signals that combine execution success with code quality metrics (readability, efficiency) to produce more well‑rounded agents.

For teams eager to experiment, the authors released a reproducible on‑prem pipeline (NVIDIA GB10) with patches for the latest GRPO stack, making it feasible to replicate the study on private hardware.

References

• Kim, M. S. (2026). Compete Then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond Imitation. arXiv preprint arXiv:2607.08255.
• Additional benchmark sources: MBPP, Codeforces, AtCoder (publicly available datasets).

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.