Andrii Bidochko

• Updated: June 10, 2026
• 6 min read

Identifying and Understanding Human Values in Text: A Tailorable LLM-based Architecture

Direct Answer

The paper Identifying and Understanding Human Values in Text introduces a three‑module LLM‑centric architecture that can automatically extract, label, and score human values expressed in natural language. By turning vague ethical concepts into structured data, the system enables downstream AI products to reason about value alignment at scale.

Background: Why This Problem Is Hard

Human values—such as fairness, autonomy, and privacy—are foundational to trustworthy AI, yet they are notoriously difficult to operationalize. The challenges stem from three interrelated factors:

• Semantic Ambiguity: The same word can convey different values depending on context (“freedom” may refer to political liberty or personal autonomy).
• Cultural Variability: Value priorities shift across societies, making a one‑size‑fits‑all taxonomy insufficient.
• Lack of Ground‑Truth Annotations: Existing datasets either focus on sentiment or topic classification, leaving a gap for fine‑grained value detection.

Traditional NLP pipelines address these issues by hand‑crafting rule sets or training classifiers on limited, domain‑specific corpora. Such approaches struggle to generalize beyond narrow domains, require extensive manual effort, and often miss the nuanced interplay between multiple values in a single passage.

What the Researchers Propose

The authors present a modular framework built around a large language model (LLM) that decomposes the value‑identification task into three coordinated stages:

1. Value Specification Module (VSM): Generates a concise, machine‑readable definition for each target value (e.g., “fairness = equal treatment of individuals regardless of protected attributes”).
2. Text Labeling Module (TLM): Takes a raw text snippet and, guided by the VSM definitions, produces a multi‑label annotation indicating which values are present and their contextual polarity.
3. Support/Resistance Scoring Module (SRSM): Assigns a numeric score that reflects how strongly the text supports or opposes each identified value, enabling downstream ranking or conflict resolution.

Crucially, the system leverages in‑context learning and chain‑of‑thought prompting to keep the three modules synchronized without requiring separate fine‑tuning for each component.

How It Works in Practice

At inference time, the workflow proceeds as follows:

1. Initialize Value Catalog: The VSM queries the LLM with a curated list of high‑level ethical concepts (e.g., fairness, privacy, autonomy). For each concept, the LLM returns a short definition and a set of illustrative examples.
2. Annotate Input Text: The TLM receives a user‑provided document (news article, policy draft, or social media post) along with the value catalog. Using a structured prompt, the LLM scans the text and emits a JSON‑like list of value tags, each paired with a brief justification excerpt.
3. Score Support vs. Resistance: The SRSM re‑examines each tagged segment, asking the LLM to rate on a –5 to +5 scale how the passage endorses or challenges the associated value. The scores are aggregated to produce a document‑level value profile.

This pipeline differs from prior work in two key ways:

• Zero‑Shot Adaptability: Because the modules rely on prompting rather than task‑specific fine‑tuning, the system can be extended to new values simply by adding them to the catalog.
• Fine‑Grained Conflict Detection: The support/resistance scores expose intra‑document value tensions (e.g., a policy that promotes security while limiting privacy), which are invisible to binary classifiers.

Evaluation & Results

The authors introduce ValueEval, a benchmark comprising 5,000 manually annotated sentences drawn from news, legislative texts, and online forums. Each sentence is labeled with up to three values and a support/resistance polarity.

Evaluation focuses on three dimensions:

• Label Accuracy: Measured by micro‑averaged F1 score against the human annotations.
• Score Correlation: Pearson correlation between the model’s numeric scores and the annotators’ Likert‑scale judgments.
• Generalization: Performance on a held‑out “cross‑culture” subset containing texts from non‑Western sources.

Key findings include:

• The three‑module system achieves an F1 of 78.4 %, surpassing a strong baseline fine‑tuned BERT classifier (71.2 %).
• Support/resistance scores correlate at 0.71 with human ratings, indicating reliable nuance capture.
• When evaluated on the cross‑culture subset, performance drops only modestly (F1 = 74.9 %), demonstrating cultural robustness.

These results suggest that the prompting‑driven architecture can reliably surface human values from diverse textual sources without extensive task‑specific training.

Why This Matters for AI Systems and Agents

Embedding a structured understanding of human values directly into language‑centric AI pipelines unlocks several practical capabilities:

• Ethical Guardrails for Generative Agents: Value profiles can be fed into policy modules that veto or re‑rank outputs conflicting with desired ethical constraints.
• Dynamic Policy Auditing: Enterprises can run the system over internal communications to surface emerging value‑related risks (e.g., bias, privacy breaches) before they manifest in products.
• Multi‑Agent Coordination: In complex orchestration scenarios, agents can negotiate trade‑offs by referencing quantified support/resistance scores, leading to more transparent decision‑making.

For developers building AI‑driven assistants, the architecture offers a plug‑and‑play component that translates abstract policy statements into actionable signals. Teams can integrate the value detection service via a simple API, then align downstream models using techniques described in UBOS Agent Orchestration. Moreover, the approach dovetails with emerging standards for AI governance, providing a concrete data layer that regulators can audit.

What Comes Next

While the presented system marks a significant step forward, several limitations point to fertile research avenues:

• Scalability of the Value Catalog: As the number of values grows, prompt length may exceed model limits. Future work could explore hierarchical prompting or retrieval‑augmented generation to keep catalogs concise.
• Fine‑Grained Contextualization: Current scores operate at the sentence level; extending to discourse‑level reasoning would improve detection of subtle value shifts across paragraphs.
• Interactive Feedback Loops: Incorporating human‑in‑the‑loop corrections could refine definitions and improve cultural sensitivity over time.

Potential applications span from automated compliance monitoring to value‑aware recommendation engines. Companies interested in prototyping such capabilities can explore the UBOS Ethics Toolkit, which offers pre‑built connectors for the three modules described in this paper. Researchers aiming to push the frontier may also consider integrating multimodal signals (e.g., images, audio) to capture values expressed beyond text.

In sum, the paper demonstrates that large language models, when guided by well‑structured prompts, can serve as versatile value extraction engines—a capability that will become increasingly central as AI systems assume greater decision‑making authority.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.