- Updated: July 16, 2026
- 2 min read
Large‑Language‑Models‑as‑a‑Judge in Theory‑Agnostic Adaptive Metric‑Alignment for Prototypical Networks in Personality Recognition
Large‑Language‑Models‑as‑a‑Judge in Theory‑Agnostic Adaptive Metric‑Alignment for Prototypical Networks in Personality Recognition
Authors: Jing Jie Tan, Ban‑Hoe Kwan, Danny Wee‑Kiat Ng, Yan‑Chai Hum, Shih‑Yu Lo, Po‑An Chen, Noriyuki Kawarazaki, Kosuke Takano, Anissa Mokraoui
Published: July 11, 2026
Abstract
Personality recognition has traditionally relied on theory‑dependent models that are forced to fit predefined psychological taxonomies. This limits generalisation because personality is fundamentally theory‑invariant, while existing annotations capture only partial, sometimes inconsistent, views of the underlying latent traits. We introduce JAM (Judge for Adaptive Metric‑Alignment), a theory‑agnostic framework that discovers unified latent pseudo‑facets representing shared psychological structure. JAM learns generalisable representations and can infer an individual’s latent psychological profile directly from text, without requiring theory‑specific labels.
Key Contributions
- Novel Attention‑Pooled Graph Prototypical Network that clusters embeddings to form structured latent representations.
- Cross‑Theory Harmonisation (CTH) that unifies heterogeneous datasets via Human‑Guided Linkage and Machine‑Induced Consensus.
- LLM‑as‑a‑Judge mechanism in two configurations (LLM‑before‑the‑loop and LLM‑in‑the‑loop) to identify ambiguous samples and guide adaptive metric learning.
- Extensive experiments demonstrating improved cross‑framework generalisation and state‑of‑the‑art performance on low‑resource personality tasks.
Methodology Overview
The JAM pipeline first encodes textual samples using a large language model. These embeddings are fed into an attention‑pooled graph where nodes represent prototype clusters. Metric‑alignment is performed adaptively, guided by the LLM‑as‑a‑Judge which flags uncertain instances for re‑training. The CTH layer aligns heterogeneous label spaces, enabling the model to learn from multiple personality theory datasets simultaneously.
Results
On benchmark datasets spanning Big Five, HEXACO, and domain‑specific personality inventories, JAM achieves up to 12 % relative improvement in F1‑score over the strongest baselines. The LLM‑in‑the‑loop configuration yields the most robust performance, especially in low‑resource settings.
Code and Resources
The full codebase, model weights, and reproducible artifacts are available at the project repository: https://research.jingjietan.com/JAM.
Further Reading on ubos.tech
Explore related AI research and implementation guides on our platform: ubos.tech/ai.
For inquiries or collaborations, please contact the lead author.
Andrii Bidochko
CTO UBOS
Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.