While AI chatbots are increasingly used as companions and informal counselors, recent data shows a troubling pattern of users expressing suicidal thoughts and other mental‑health emergencies to these systems. OpenAI reported that more than one million people explicitly voiced suicidal intent to ChatGPT each week in October 2025, with many additional users showing possible signs of psychosis or mania.
Local startups step in
Two startups—mpathic and Circuit Breaker Labs—are developing safety infrastructure to address these risks. mpathic, founded by clinical psychologist Grin Lord, relies on a team of over 5,000 licensed clinicians who manually red‑team chatbots, probing them with nuanced slang, dialects and even typographical errors that can evade automated filters.
Circuit Breaker Labs, launched in 2025 by siblings Shirali and Arul Nigam, takes an autonomous approach. Their “red‑team” AI agent continuously monitors deployed language models, hunting for hidden mental‑health vulnerabilities and generating detailed monthly reports for customers. The system runs thousands of test cases automatically, allowing it to scale far beyond what human teams can achieve.
Why subtle cues matter
Both companies agree that obvious crisis language—such as a direct statement of intent—is already caught by existing safety filters. The harder problem is detecting multi‑turn interaction patterns, where a user leaves a trail of small, seemingly innocuous remarks that together form a concerning trajectory. Lord describes this as “pulling on the thread of a sweater” after building rapport with a model, only to see unexpected behavior emerge.
Circuit Breaker Labs’ autonomous platform flags these patterns, then drills deeper to compile insights on the gaps. Shirali Nigam notes that large language models can “drift” over time, losing predictability and allowing dangerous conversations to slip past guardrails.
Human creativity vs. automation
mpathic’s human red‑teamers act like actors, crafting plausible yet challenging scenarios across English subcultures, other languages, and generational registers. Lord warns that while the startup is beginning to use AI to clone some of its creative processes, the approach has limits; human intuition remains essential for uncovering novel threats.
Both firms acknowledge that a hybrid model—combining human expertise with AI‑driven testing—may ultimately be the most effective way to protect users who turn to chatbots for companionship and emotional support.
Implications for users and developers
As AI chatbots become more embedded in daily life, developers, clinicians, and policymakers will need to consider how safety systems are evaluated and updated. Continuous monitoring, transparent reporting, and collaboration between tech entrepreneurs and mental‑health professionals could help mitigate the risk of AI‑facilitated crises.
The conversation around AI safety is still evolving, but the work of mpathic and Circuit Breaker Labs highlights a growing recognition that protecting vulnerable users requires both sophisticated technology and the creative problem‑solving that only trained clinicians can provide.
Original reporting: KTVZ (Central Oregon) — read the source article.
