Andrii Bidochko

• Updated: July 9, 2026
• 7 min read

Decentralized Aggregation of LLM Predictions via Wagering Mechanisms

Direct Answer

The paper introduces WALLA (Advantage‑Aligned Wagering Mechanisms for LLM Aggregation), a decentralized framework that lets multiple large language models (LLMs) submit predictions together with a self‑determined “wager” that serves as a weight in the final ensemble. By aligning each model’s wager with its expected advantage, WALLA guarantees incentive‑compatible reporting while matching the predictive power of centralized aggregation methods.

Background: Why This Problem Is Hard

Enterprises increasingly rely on ensembles of specialized LLMs—some fine‑tuned for legal text, others for code generation, still others with privileged access to proprietary databases. Combining these models promises higher accuracy, broader coverage, and resilience to individual failures. However, two practical obstacles have stalled widespread adoption:

• Privacy and ownership. Model owners often cannot expose weights, training data, or internal confidence scores to a central orchestrator without risking IP leakage.
• Strategic behavior. In a competitive ecosystem, a model might inflate its confidence or under‑report its prediction to manipulate the ensemble’s weighting, especially when monetary rewards are tied to outcomes.

Traditional aggregation techniques—simple averaging, weighted voting based on historical performance, or Bayesian model averaging—assume a trusted, fully observable environment. They either require a central authority to collect private signals or they break down when participants act strategically. As a result, current pipelines either sacrifice privacy for performance or accept sub‑optimal ensembles that are vulnerable to gaming.

What the Researchers Propose

WALLA reframes the aggregation problem as a wagering game inspired by prediction markets. Each participating LLM performs two actions:

1. Report a prediction. The model outputs its best guess for the target variable (e.g., the answer to a question or a probability distribution over future events).
2. Place a wager. Independently, the model submits a scalar amount that reflects how much “skin in the game” it is willing to put behind that prediction.

The ensemble’s final answer is a weighted combination where each model’s weight equals its wager. Crucially, the payout function—how much reward a model receives after the true outcome is revealed—includes a leave‑one‑out baseline. This baseline neutralizes any advantage a model could gain by simply mimicking the crowd, forcing the optimal strategy to be honest reporting of its true belief.

Key roles in the WALLA ecosystem:

• LLM agents. Autonomous services that generate predictions and learn a wagering policy.
• Wagering coordinator. A lightweight protocol that collects (prediction, wager) pairs, computes the weighted aggregate, and distributes payouts after the ground truth is known.
• Learning module. Each LLM updates its wager policy based on past payouts, without needing to adjust its underlying prediction model.

How It Works in Practice

The operational flow of WALLA can be broken down into four repeatable steps:

1. Query broadcast. A user or downstream system issues a request (e.g., “What is the projected revenue for Q4?”). The coordinator forwards the query to all registered LLM agents.
2. Prediction & wager submission. Each LLM independently computes its answer and, using its learned wagering policy, decides how much to stake. The pair (prediction, wager) is sent back to the coordinator.
3. Weighted aggregation. The coordinator aggregates predictions by normalizing wagers into a probability distribution and applying them as weights. The result is the ensemble output presented to the requester.
4. Outcome verification & payout. Once the true answer becomes available (e.g., after the quarter ends), the coordinator calculates each model’s net payout using the leave‑one‑out baseline. Models receive a reward proportional to the improvement their wagered prediction contributed beyond the baseline.

What distinguishes WALLA from prior market‑based approaches is the advantage‑aligned property: a model’s optimal wager is directly proportional to its expected scoring advantage, not to arbitrary confidence scores. This alignment emerges from the carefully designed payout function, which guarantees dominant‑strategy incentive compatibility—no model can benefit by misreporting its belief, regardless of what others do.

Evaluation & Results

The authors validated WALLA on two benchmark families:

• Open‑domain question answering. Datasets such as NaturalQuestions and TriviaQA were used, with heterogeneous LLMs ranging from open‑source 7B models to proprietary 70B systems.
• Forecasting tasks. Time‑series and probability‑forecast challenges (e.g., Good Judgment Open) where models had access to private auxiliary data.

Experimental protocol:

1. Each model was first trained on its own data, preserving privacy.
2. During aggregation, models submitted wagers learned via reinforcement learning on past payouts.
3. Baseline comparisons included simple averaging, performance‑based weighting (using historical accuracy), and a centralized oracle that could directly inspect model confidences.

Key findings:

• Predictive parity. WALLA’s ensemble accuracy matched the centralized oracle within a 0.5 % margin on QA tasks, and outperformed simple averaging by 3–5 % absolute.
• Robustness to strategic reporting. When a subset of models deliberately inflated their confidence, WALLA’s incentive‑compatible design prevented any gain, whereas performance‑based weighting suffered up to a 7 % drop.
• Bounded deficit. The mechanism’s worst‑case financial loss (deficit) remained under 2 % of total payouts, confirming the theoretical guarantee of a bounded budget.
• Uncertainty awareness. Models with higher epistemic uncertainty naturally placed lower wagers, leading to a self‑regularizing ensemble that down‑weights noisy contributors.

These results demonstrate that a fully decentralized, privacy‑preserving aggregation can achieve the same practical performance as a centrally controlled system while adding strategic safety nets.

Why This Matters for AI Systems and Agents

For AI practitioners building multi‑model pipelines, WALLA offers a plug‑and‑play protocol that solves three pain points simultaneously:

• Privacy‑first orchestration. Teams can expose only predictions and wagers, keeping model weights, training data, and proprietary APIs hidden.
• Incentive‑aligned collaboration. When multiple business units or external partners contribute models, the wagering mechanism ensures each party is rewarded fairly for genuine advantage, discouraging “gaming” of the ensemble.
• Dynamic weighting without retraining. Because wagers are learned separately, a model can improve its contribution simply by refining its wagering policy, avoiding costly re‑fine‑tuning of the base LLM.

These capabilities map directly onto emerging UBOS platform overview features such as decentralized workflow orchestration and secure model marketplaces. For example, an ChatGPT and Telegram integration could let a fleet of domain‑specific bots submit forecasts to a shared WALLA coordinator, delivering a single, trustworthy answer to end‑users while preserving each bot’s proprietary knowledge.

What Comes Next

While WALLA establishes a solid theoretical and empirical foundation, several avenues remain open for exploration:

• Scalability to thousands of agents. Current experiments involve up to a dozen models; extending the protocol to large federations will require efficient communication and payout computation.
• Multi‑task wagering. Adapting the mechanism to handle heterogeneous tasks (e.g., classification + generation) within a single aggregation round.
• Hybrid reward structures. Combining monetary payouts with non‑financial incentives (e.g., reputation scores) could broaden applicability in open‑source ecosystems.
• Robustness to delayed outcomes. Some forecasting domains reveal ground truth only after weeks or months; designing interim reward signals is an open challenge.

Addressing these challenges will unlock broader adoption in sectors such as finance, healthcare, and autonomous systems, where decentralized expertise and strict data governance are non‑negotiable. Companies interested in prototyping WALLA on their own model fleets can start by exploring the Workflow automation studio, which provides a low‑code environment for defining custom wagering contracts and integrating them with existing LLM endpoints.

References

Decentralized Aggregation of LLM Predictions via Wagering Mechanisms (arXiv)

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.