- Updated: August 25, 2026
- 2 min read
Program Semantic Inequivalence Game with Large Language Models
Program Semantic Inequivalence Game with Large Language Models
Authors: Antonio Valerio Miceli‑Barone, Vaishak Belle, Ali Payani
Abstract: Large Language Models (LLMs) can achieve strong performance on everyday coding tasks, but they can fail on complex tasks that require non‑trivial reasoning about program semantics. Finding training examples to teach LLMs to solve these tasks can be challenging. In this work, we explore a method to synthetically generate code‑reasoning training data based on a semantic inequivalence game (SInQ): a generator agent creates program variants that are semantically distinct, derived from a dataset of real‑world programming tasks, while an evaluator agent has to identify input examples for which they behave differently. The agents train each other semi‑adversarially, improving their ability to understand the underlying logic of code.
Key Contributions
- Introduced the SInQ framework for automatic generation of semantically inequivalent program pairs.
- Demonstrated semi‑adversarial training between generator and evaluator agents to enhance code‑understanding capabilities.
- Released code and synthetic data for reproducibility and further research.
Results
- Improved cross‑language vulnerability detection (Lu et al., 2021) despite training exclusively on Python code.
- Substantial gains on the Python builtin identifier swap benchmark (Miceli Barone et al., 2023), where modern LLMs still struggle.
Impact
The SInQ approach provides a scalable way to generate high‑quality training data for LLMs, enabling better reasoning about program semantics and opening new avenues for research in code generation, vulnerability detection, and automated program analysis.
Read more about our methodology and download the dataset on the arXiv page. For related articles and resources, visit our blog and explore the SEO strategy we prepared for this paper.
Andrii Bidochko
CTO UBOS
Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.