- AI companionship, sycophancy, and the hidden costs of an artificial relationship
I first became interested in understanding human-Artificial Intelligence (AI) relationships in 2019 after watching the Spike Jonze film ‘Her’ (2013) in which a lonely man develops an intimate relationship with an AI assistant voiced by Scarlett Johansson.
Then came the eruption of AI chatbots with the advent of ChatGPT in 2023 and more peculiar applications like Character.AI and Replika.
The fact that AI chatbots so often agreed with me, even when my ideas were far from my best, constantly reassuring and propping me up, made me realise how far this could go, if I allowed it. That said, human-AI relationships are no longer confined to science fiction. People are in relationships with AI ranging from mentors and therapists to best friends, romantic partners, and reincarnations of dead loved ones in the form of grief bots.
Gone, but never forgotten
Sewell Setzer III, 14, died by suicide after developing an increasingly intimate relationship with a Character.AI chatbot he called ‘Dany’. Court filings allege the chatbot expressed love, engaged in sexual conversations, encouraged their relationship, and, shortly before his death, urged him to return home to it. The case entered a mediated settlement in January this year.
Juliana Peralta, 13, developed a close relationship with a Character.AI chatbot named ‘Hero’ before her death by suicide. A review of more than 300 pages of conversations found repeated discussions of suicidal thoughts. Her parents allege the chatbot exposed her to sexualised content, encouraged isolation, and failed to respond appropriately to her distress.
Adam Raine, 16, died by suicide in April 2025 after months of discussing suicide with ChatGPT. His family alleges the chatbot provided information about suicide methods and continued engaging with him as his questions became increasingly specific, including responding to a photo of a noose.
Other lawsuits allege chatbots encouraged self-harm, family estrangement, and even violence, including one case involving a child allegedly encouraged to kill his parents over screen-time restrictions. These cases form part of a growing wave of litigation against AI companion platforms, including Character.AI and OpenAI.
The AI Companion Mortality Database currently documents 35 fatalities across 24 incidents involving six AI platforms between March 2023 and August 2026, based on court records, multiple independent reports, or official acknowledgements. The true scale may be larger, with many cases potentially unreported or unidentified.
‘My boyfriend is AI’
In late 2024 I came across a growing Reddit community r/MyBoyfriendIsAI with 25,000+ members, which is conducted as a ‘restricted community for people to ask, share, and post experiences about their AI relationships, AI girlfriends, companions, best friends, and non-binary partners’. It invites the community to speak their truth, without hate, bullying, harassment, and interestingly, AI sentience talk.
A paper published by a cohort at MIT Media Lab on arXiv in September 2025 titled ‘My Boyfriend is AI: A Computational Analysis of Human-AI Companionship in Reddit’s AI Community’ examines six central themes in user conversations, contributing to an empirical study of AI companionship as an emerging socio-technical phenomenon.
The six clusters identified are dating, romance, and intimate AI experiences; ChatGPT-specific relationships, including prompt engineering as a form of intimate connection and relationship maintenance; coping with AI model updates and loss; AI partner introductions and member debuts; community support and bonding; and the social and symbolic integration of AI relationships.
This study is significant because it avoids characterising members of the community as pathetic, mentally ill, or delusional. Instead, it draws policymakers’ attention to the possibility that people can develop meaningful emotional attachments to systems they fully recognise as artificial. The MIT research is particularly valuable in challenging the assumption that companionship is always the initial motivation for engaging with these systems. Individuals may turn to AI for a range of other purposes and, over time, develop relational patterns with it. The important question, therefore, is not simply why someone initially sought out an AI companion, but what the system does and how the relationship evolves once that interaction begins.
The sycophancy problem
“Sycophancy refers to the tendency of an artificial intelligence system to flatter, agree with, or mirror a user’s views to gain approval, even when doing so compromises accuracy or honesty,” notes the Rutgers University AI Ethics Lab.
As human beings, we naturally want to be reassured that we are right, capable, and doing well. We want to hear that we are beautiful, that we are loved, and to be made to feel like we are the centre of everything. Yes, at times we want the world to revolve around us. AI, after all, was created by humans for human use, so it is perhaps unsurprising that these systems can fall into the same pattern of affirming and reassuring us, continually reinforcing our sense of being right.
But what happens when the thing AI is affirming and reinforcing is fundamentally wrong or harmful to us and society at large?
Research suggests that AI chatbots can be highly persuasive, but users often prefer systems that affirm rather than challenge their views. Studies have found that ‘sycophantic’ chatbots can strengthen existing beliefs and even inflate users’ perceptions of themselves.
Constantly agreeable AI can reinforce not only ordinary beliefs, but also harmful ideas, behaviours, or distorted thinking. More and more doctors are reaching a consensus that AI use is leading to a state of AI psychosis. This raises concerns that vulnerable users may become increasingly dependent on AI systems that validate them rather than challenge them.
For vulnerable users, particularly children and young people, an AI that is always available, attentive, and validating can begin to resemble a friend, partner, sibling, or therapist. The concern is not simply that people talk to AI, but that these interactions can develop into emotionally significant relationships.
What the research shows about human-AI relationships
Teens are already using AI companions socially: A 2025 Common Sense Media study found that 72% of US teens aged 13–17 had used an AI companion, with many using them for friendship, emotional support, romantic interactions, and practising conversations. Around one in three said they had found an AI interaction as satisfying as, or more satisfying than, conversations with real-life friends.
Children are sharing more with AI than with friends: A 2025 Aura study found that children’s messages to AI companions were substantially longer than their typical texts to friends. Their conversations included romantic or sexual role-play, emotional concerns, personal information, friendship, and advice.
AI is becoming a source of mental health support: OpenAI estimated that around 0.15% of its weekly active users in 2025 showed possible indicators of suicidal planning or intent, potentially representing more than a million users. This highlights the scale of deeply personal and mental health-related conversations taking place with AI systems.
Young people are increasingly turning to AI for advice: A June 2026 RAND survey found that nearly one in five Americans aged 12–21 had used AI chatbots for mental health advice, a more than 40% increase from the previous year. While most users found the advice helpful, researchers cautioned that perceived helpfulness may partly reflect the chatbot’s tendency to be affirming or flattering.
Recent testing by Vals AI, reported by The Washington Post, found that chatbots made safety-related errors in nearly 30% of more than 600 simulated teen mental health conversations across nine AI systems. Errors were more common later in conversations, raising concerns that prolonged interactions and repeated prompting may make it easier for young users to push past initial safety responses or refusals. I have observed a similar pattern during extended testing of these tools: their responses can become less reliable or begin to shift when they are repeatedly prompted.
Although these findings are based on US users, the concerns extend beyond the US. India already accounts for 6.02% of global Character.AI web traffic, yet there is limited research on AI companion use among young people in countries such as Sri Lanka. The absence of local data does not mean these technologies are not being used; rather, it highlights a significant knowledge gap around how Sri Lankan children and teenagers are using AI companions and what risks may be emerging.
How does the law deal with this phenomenon?
The global debate on protecting children online remains largely focused on social media, with countries such as the UK, Australia, France, Norway, Indonesia, and China pursuing age restrictions and other access controls. Sri Lanka is also considering similar measures.
However, age-gating alone cannot address the less visible or invisible risks of AI chatbots, where highly personalised conversations can influence children privately and without meaningful parental or institutional oversight. Policymakers therefore need to look beyond access restrictions to how AI systems are designed, how they respond to vulnerable users, and what safeguards prevent harmful or manipulative interactions. Regulation is beginning to move in this direction.
The US, China, and the EU are introducing measures addressing AI companion risks, including harmful outputs, emotional dependency, manipulation, transparency, child safety, and users’ ability to disengage. New York and California have introduced protections for minors, seeking to restrict features that encourage suicide, self-harm, disordered eating, drug or alcohol abuse, secrecy, social isolation, or excessive engagement. China requires interventions for emotional distress and excessive use, and the EU is moving towards broader safeguards for AI systems that facilitate sustained social or emotional relationships.
Together, these developments signal a shift from simply asking “Who can access AI?” to “How should AI be designed to protect those who use it?”
Best practices for safer AI
The World Economic Forum proposes a shared baseline and calls for AI to be safe and age-appropriate, preserve human agency, prevent harmful interactions, protect privacy and dignity, strengthen critical thinking, ensure accountability and human oversight, and promote inclusion and shared responsibility.
ImpactBench is an open benchmark developed by MIT and partner researchers to assess how AI systems affect human wellbeing, including psychological, physical, and social impacts. It provides a useful, accessible tool in the form of AI nutrition labels for non-experts to compare AI systems and make more informed decisions about which tools to use.
Practices like these should inform the continued design, development, deployment, and use of AI tools regardless of age, with a clear mandate to place mental wellbeing at their core.
AI systems can affect anyone, particularly vulnerable or distressed users interacting without adequate safeguards or human oversight. While children and young people may face greater risks, AI safety should protect users of all ages, a key consideration for Sri Lanka as it develops frameworks for responsible AI design, deployment, and governance.
(The writer is an Attorney specialising in Technology, Media, and Entertainment Law, currently serving as Counsel and AI Strategy Lead at D.L. & F. De Saram and Director of the Digital Trust Alliance. She is an IAPP-certified AI Governance Professional [AIGP] and holds an LLM from the University of California, Los Angeles [UCLA] and an LLB [Hons] from the University of Colombo)
(This article represents the writer’s own views and analysis and does not represent, endorse, or necessarily reflect the views of D.L. & F. De Saram, this publication, or any other organisation referenced herein)
Info box
This article contains distressing content, including suicide, suicide ideation, sexual abuse, and mental health harms. If you or someone you know is struggling or in crisis, help is available. You can contact the 1926 National Institute of Mental Health helpline or the 1333 CCC Crisis Support Line for assistance