Andrii Bidochko

• Updated: June 11, 2026
• 7 min read

SuiChat-CN: Benchmarking Contextual Suicide Risk Assessment in Chinese Group Chats

Direct Answer

The paper introduces SuiChat‑CN, a benchmark specifically designed for contextual suicide‑risk assessment in Chinese group‑chat environments such as Telegram. By providing a large, expertly annotated collection of conversational segments, the dataset highlights the critical role of multi‑turn, multi‑party context in detecting early warning signals that single‑post analyses miss.

Background: Why This Problem Is Hard

Suicide remains a leading cause of premature death worldwide, with an estimated 720,000 lives lost each year. Early detection of suicidal ideation can enable timely intervention, yet traditional monitoring pipelines focus on public, post‑based platforms (e.g., Twitter, Weibo). Those platforms share several characteristics that simplify automated analysis: messages are often authored by a single user, timestamps are relatively sparse, and the language tends to be explicit.

Instant‑messaging group chats, however, present a fundamentally different linguistic landscape:

• Fragmented utterances: Participants exchange short, often incomplete sentences that rely on shared context.
• Multi‑party dynamics: Risk signals may emerge from the interplay between several users, not just the individual expressing distress.
• Cultural nuance: In Chinese online communities, indirect expressions, idioms, and slang are common, making literal keyword detection unreliable.
• Temporal immediacy: Conversations evolve rapidly, and a single alarming phrase may be mitigated—or amplified—by subsequent replies.

Existing NLP benchmarks for suicide‑risk detection (e.g., CLPsych, SMHD) largely ignore these complexities, limiting their applicability to real‑world crisis‑intervention systems that must operate within private, high‑velocity chat environments.

What the Researchers Propose

To bridge this gap, the authors propose a three‑stage framework that transforms raw Telegram group‑chat logs into a structured, context‑rich benchmark:

1. Signal‑word extraction: An automated scanner identifies candidate utterances that contain potential risk indicators (e.g., “想死”, “绝望”).
2. Bidirectional context expansion: For each candidate, the system pulls preceding and succeeding messages until a coherent conversational segment is formed, preserving speaker turns and temporal order.
3. Expert‑validated, LLM‑assisted annotation: Trained mental‑health professionals assign a risk level (e.g., low, moderate, high) to each segment, while a large language model (LLM) assists by surfacing ambiguous cases for review.

The resulting benchmark, SuiChat‑CN, contains 13,312 contextual segments drawn from 1,406 unique users and a total of 258,228 raw messages. Each segment is labeled with a fine‑grained risk rating, making it suitable for both binary classification (risk vs. no‑risk) and multi‑class severity prediction.

How It Works in Practice

From an engineering perspective, the pipeline can be visualized as a modular workflow:

Data Ingestion Layer

Public Telegram groups are scraped using the platform’s Bot API. The raw JSON payloads retain message IDs, timestamps, sender IDs, and reply‑to references.

Pre‑processing & Signal Detection

A lightweight lexical filter scans each message for a curated list of suicide‑related cue words. This step reduces the volume of data passed to downstream components, focusing computational resources on high‑yield candidates.

Contextual Segment Builder

For every flagged message, a bidirectional traversal algorithm pulls surrounding turns until a predefined window (e.g., 10 messages before and after) or a conversational boundary (e.g., topic shift) is reached. The algorithm respects speaker identity, ensuring that the final segment reflects the multi‑party nature of the chat.

Annotation Interface

Human annotators access a web‑based UI that displays the full segment, speaker metadata, and optional sentiment scores generated by a baseline PLM. An LLM (e.g., GPT‑4‑like) runs in the background, proposing a provisional risk label and highlighting ambiguous phrases for the annotator’s attention.

Quality Assurance & Release

Inter‑annotator agreement is measured using Cohen’s κ, and any disagreements trigger a secondary review. The final, vetted dataset is stored in a secure, access‑controlled repository, with distribution limited to accredited mental‑health research institutions.

This architecture differs from prior benchmarks in two key ways:

• It treats the conversation as a first‑class citizen rather than a collection of isolated posts.
• It leverages LLMs as collaborative assistants during annotation, improving consistency while preserving expert oversight.

Evaluation & Results

The authors conducted an extensive experimental campaign, testing more than 40 language models ranging from fine‑tuned BERT‑style encoders to instruction‑tuned LLMs. The evaluation focused on three research questions:

1. Does contextual information improve risk detection?

Models that ingested the full conversational segment consistently outperformed those limited to the flagged message alone. For example, a fine‑tuned Chinese RoBERTa achieved a macro‑F1 of 0.71 on segment‑level classification, versus 0.58 when only the cue message was used.

2. How do different model families compare?

Instruction‑tuned LLMs (e.g., ChatGPT‑style) demonstrated strong zero‑shot capabilities, reaching an F1 of 0.68 without any task‑specific fine‑tuning. However, after a modest 2‑epoch fine‑tuning on SuiChat‑CN, the same models surpassed 0.75, indicating that domain‑specific adaptation yields tangible gains.

3. What are the limits of early detection?

To simulate real‑time monitoring, the authors evaluated models on partial contexts—only the messages preceding the cue. Performance dropped by roughly 12 % across the board, underscoring the difficulty of predicting risk before the user explicitly signals distress.

Collectively, these results confirm two central hypotheses: (1) contextual cues are indispensable for reliable suicide‑risk assessment in group chats, and (2) even state‑of‑the‑art LLMs benefit from targeted fine‑tuning on domain‑specific, multi‑turn data.

Why This Matters for AI Systems and Agents

For AI practitioners building conversational agents, mental‑health chatbots, or moderation tools, SuiChat‑CN offers a realistic testbed that mirrors the complexities of real‑world deployments:

• Context‑aware decision making: Agents can be trained to weigh prior turns, speaker relationships, and subtle linguistic shifts before flagging a user.
• Multi‑party orchestration: The benchmark encourages the design of systems that monitor not just a single user but the entire conversational ecosystem, enabling peer‑support interventions.
• Ethical grounding: By providing a rigorously annotated dataset, developers can benchmark false‑positive rates—a critical metric for avoiding unnecessary escalations in sensitive settings.
• Integration pathways: The workflow aligns naturally with existing Telegram integration on UBOS, allowing teams to ingest live chat streams, apply the trained models, and trigger automated alerts or handoffs to human counselors.

Moreover, the benchmark’s emphasis on early detection informs the design of proactive agents that can intervene before a user explicitly expresses suicidal intent, a capability that could dramatically improve outcomes in crisis‑intervention pipelines.

What Comes Next

While SuiChat‑CN marks a significant step forward, several limitations remain:

• Data diversity: The current collection focuses on public Telegram groups, which may not reflect the linguistic patterns of private or platform‑specific chats (e.g., WeChat, Discord).
• Cross‑cultural transferability: The cue‑word lexicon and annotation schema are tailored to Mandarin Chinese; adapting the framework to other languages will require culturally aware signal extraction.
• Real‑time deployment challenges: Scaling the bidirectional context builder to high‑throughput streams demands efficient indexing and low‑latency inference pipelines.

Future research directions include:

1. Extending the benchmark to multimodal data (e.g., emojis, images, voice notes) that often accompany suicidal expressions.
2. Exploring reinforcement‑learning‑based agents that can suggest supportive responses while respecting privacy and consent.
3. Integrating the dataset with OpenAI ChatGPT integration to evaluate end‑to‑end conversational safety pipelines.

By addressing these gaps, the community can move toward a holistic, ethically sound AI ecosystem capable of safeguarding mental health across diverse digital habitats.

References

SuiChat‑CN: Benchmarking Contextual Suicide Risk Assessment in Chinese Group Chats (arXiv)

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.