Andrii Bidochko

• Updated: June 16, 2026
• 6 min read

From Instructor to Collaborator: What a 90-Participant Study Reveals about Human-Agent Collaboration in a Mobile Serious Game

Direct Answer

The study introduces a large‑scale, within‑subjects experiment that compares a highly human‑like embodied conversational agent (ECA) with a low‑human‑like text‑only chatbot inside a mobile serious game about pre‑decimal UK currency. It demonstrates that users overwhelmingly prefer the more human‑like, spoken agent, and that this preference translates into higher usability scores and smoother mixed‑initiative collaboration.

Background: Why This Problem Is Hard

Human‑agent collaboration sits at the intersection of natural language processing, embodied interaction, and user experience design. While conversational AI has made strides in text‑based settings, translating those advances into embodied agents that can act as instructors, collaborators, or shopkeepers in real‑time, goal‑oriented tasks remains elusive. Existing research typically isolates one variable—either the modality (voice vs. text) or the role (assistant vs. tutor)—and evaluates agents in controlled lab environments that lack the ecological validity of everyday mobile applications.

Three core challenges hinder progress:

• Embodiment vs. usability trade‑offs: Adding a visual avatar and speech synthesis can increase perceived presence but also introduces latency, recognition errors, and design complexity.
• Role ambiguity: Users often have different expectations for an “Instructor” versus a “Collaborator,” yet most studies treat agents as a single monolithic entity.
• Mixed‑initiative breakdowns: When agents and humans both initiate dialogue, repairing misunderstandings without disrupting task flow is difficult to measure and optimize.

These bottlenecks matter because enterprises are increasingly embedding AI agents into mobile training, customer service, and gamified learning platforms. Understanding how embodiment, role, and dialogue dynamics interact is essential for building agents that are not only technically competent but also socially acceptable.

What the Researchers Propose

Danai Korre’s dissertation‑level investigation does not introduce a new algorithmic framework; instead, it offers an empirically grounded perspective on how two distinct agent archetypes perform in a shared task environment. The experiment pits:

• Alex the Instructor: A highly human‑like, spoken ECA with a 3‑D avatar, facial animations, and voice synthesis. Alex’s role is to teach players the rules of pre‑decimal currency and guide them through the game’s tutorial.
• Shopkeeper/Collaborator: A low‑human‑like, text‑only chatbot presented as a speech bubble. This agent acts as a transactional partner, helping players purchase items using the learned currency.

Both agents support voice input (speech‑to‑text) and mouse interaction, allowing a direct comparison of embodiment, modality, and role while keeping the underlying game mechanics constant.

How It Works in Practice

The mobile game, built in Unity, simulates a Victorian‑era marketplace where players must identify and exchange pre‑decimal coins. The workflow unfolds in three phases:

1. Instruction Phase: Alex appears on screen, delivers spoken explanations, and answers spoken questions. The agent’s avatar gestures to reinforce concepts, creating a multimodal teaching experience.
2. Practice Phase: Players interact with the Shopkeeper via a text bubble, typing or speaking their purchase requests. The chatbot validates the transaction, provides corrective feedback, and logs any dialogue breakdowns.
3. Assessment Phase: After a series of purchases, the system records task completion time, error rates, and subjective usability scores.

Key differentiators of this approach include:

• Mixed‑initiative design: Both agents can initiate dialogue (e.g., Alex prompting a quiz, Shopkeeper offering a discount), enabling the study of spontaneous breakdowns.
• Within‑subjects methodology: Each of the 90 participants experiences both agent conditions, eliminating inter‑participant variability.
• Multi‑modal data capture: The experiment collects quantitative metrics (CCIR MINERVA usability questionnaire, Agent Persona Instrument) and qualitative observations (exit interviews, video recordings).

Evaluation & Results

The researchers employed paired t‑tests, repeated‑measures ANOVA, and multiple linear regression to uncover relationships between agent persona traits and perceived usability. Highlights of the findings:

• Statistically significant preference: Participants rated the embodied, spoken Alex 1.8 points higher on the CCIR MINERVA scale (p < .001) with a large effect size (Cohen’s d ≈ 0.9).
• Persona‑usability correlation: Higher scores on the Agent Persona Instrument’s “Warmth” and “Competence” dimensions predicted better usability outcomes, especially for the ECA condition.
• Breakdown frequency: Mixed‑initiative dialogue breakdowns occurred 27% less often with Alex, suggesting that embodiment and voice cues aid error recovery.
• Qualitative insights: Exit interviews revealed that users perceived Alex as a “real teacher” and expected more proactive guidance, whereas the text‑only shopkeeper felt “functional but impersonal.”

Collectively, these results indicate that human‑like embodiment and spoken interaction not only improve subjective satisfaction but also reduce friction in collaborative tasks.

Why This Matters for AI Systems and Agents

For practitioners building AI‑driven assistants, the study offers concrete evidence that design choices around embodiment and role definition have measurable impact on user experience. The implications cascade across several domains:

• Agent design pipelines: When constructing training modules for sales or onboarding bots, incorporating a visual avatar and voice synthesis can boost perceived competence and warmth, leading to higher adoption rates.
• Evaluation frameworks: The combined use of CCIR MINERVA and the Agent Persona Instrument provides a replicable template for assessing human‑agent synergy beyond simple task success metrics.
• Orchestration of multi‑agent systems: Understanding how users differentiate between an “Instructor” and a “Collaborator” informs the allocation of responsibilities in complex workflows, such as a virtual classroom paired with a tutoring chatbot.
• Integration with existing platforms: Developers can leverage the OpenAI ChatGPT integration to power the language core, while adding the ElevenLabs AI voice integration for spoken output, and the Telegram integration on UBOS for cross‑platform deployment.

These takeaways align with the broader trend of “human‑centric AI,” where the goal is not merely to automate tasks but to create agents that users feel comfortable collaborating with.

What Comes Next

While the study delivers robust evidence for the benefits of embodied agents, several limitations point to fertile ground for future research:

• Scalability of embodiment: Rendering high‑fidelity avatars on low‑end mobile devices can strain resources. Exploring lightweight 2‑D avatars or adaptive fidelity could broaden accessibility.
• Domain generalization: The experiment focused on a niche educational game about historical currency. Replicating the design in domains such as healthcare training, corporate compliance, or e‑commerce will test the universality of the findings.
• Long‑term interaction effects: The study measured immediate usability; longitudinal studies could reveal how trust and reliance evolve over weeks or months of usage.
• Dynamic role switching: Future agents might fluidly transition between instructor and collaborator based on context. Implementing such role elasticity would require sophisticated intent detection and policy management.

Developers interested in prototyping these next steps can start with the UBOS platform overview to spin up a mobile app, then use the Workflow automation studio to orchestrate role‑based dialogue flows. For startups seeking rapid iteration, the UBOS templates for quick start provide pre‑built conversational scaffolds that can be customized with voice and avatar assets.

Finally, the broader research community is invited to build on this dataset, share replication studies, and explore how mixed‑initiative breakdown repair mechanisms can be automated using reinforcement learning or meta‑learning techniques.

References

From Instructor to Collaborator: What a 90-Participant Study Reveals about Human-Agent Collaboration in a Mobile Serious Game – arXiv preprint, May 2026.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.