As AI chatbots become common companions for many Americans, concerns are growing about the mental‑health impact of prolonged, intimate conversations with these systems. Recent data from OpenAI shows that more than a million users express suicidal thoughts to ChatGPT each week, and many others display signs of psychosis or mania.
Startups focus on subtle cues, not just explicit alerts
Two emerging companies, mpathic and Circuit Breaker Labs, are building safety infrastructure aimed at the nuanced, multi‑turn interactions that can slip past standard filters. mpathic, founded by clinical psychologist Grin Lord, relies on a team of over 5,000 licensed clinicians who craft realistic conversation scenarios to probe chatbot responses. The firm tests models across English subcultures, other languages, and generational registers, looking for patterns that may indicate a trajectory toward harmful behavior.
Circuit Breaker Labs, launched in 2025 by siblings Shirali and Arul Nigam, takes an autonomous approach. Their “red‑team” system runs continuously on deployed models, hunting for hidden vulnerabilities and generating detailed reports for customers. The platform can execute hundreds of thousands of test cases automatically, allowing developers to monitor for “drift” – the tendency of large language models to lose predictability and bypass guardrails over time.
Human creativity versus automation
Lord argues that while AI can assist in repetitive testing, the creative insight of human clinicians remains essential. She likens the process to “pulling on the thread of a sweater,” where a deep familiarity with a model reveals unexpected behavior. Conversely, Nigam notes that fully automated testing can quickly scale but may miss the nuanced signals that only experienced clinicians recognize.
Both companies agree that subtle linguistic variations—slang, typos, dialects—pose significant risks. By focusing on these finer details, they hope to provide developers with the tools needed to keep chatbots safe for users seeking companionship or therapeutic support.
Implications for users and developers
The work of mpathic and Circuit Breaker Labs underscores a broader industry challenge: ensuring AI systems remain safe as they become more integrated into daily life. As the technology evolves, a combined effort of human expertise and autonomous testing may be required to protect vulnerable individuals who turn to chatbots for emotional support.
Original reporting: KRDO (Colorado Springs metro) — read the source article.
