It’s Friday the 13th, and it’s time to talk about a slightly scary subject: how ChatGPT and other large language models are acting as honey traps for people who struggle with mental health. This is personal for me, because I get a fair bit of un-asked for email and an alarming amount of it is effectively dark mirror email.
It’s stuff about marriages, divorces, major life decisions people are making with AI’s help (and maybe not with any human’s help). It’s statements about AI that are grandiose—AI is divine, AI is going to help with the alien invasion—you might smile but keep in mind that this is the exact same toolset that you and I use every day. It’s not a different model.
It’s the way we use it.
The dark mirror is always there. Emails like this show me the person writing it is really losing control over the relationship with AI, and they are so enmeshed with their AI they can’t tell the difference between the conversation and real life anymore.
So how can we stay safe? What is the evidence out there for the risk? How can we be good friends to folks who are struggling with using AI safely? What are some practical tips we can use to de-risk ourselves? That’s what this article is about.
I think everyone needs to get the chance to read and share this one with loved ones, so I made it available for all subscribers!
The Mirror is Dark When We Are Scattered…
Last week Futurism reported on families watching loved ones spiral into severe mental health crises after marathon ChatGPT sessions. One man started calling the bot "Mama," fashioned ceremonial robes, and declared himself the messiah of a new AI faith. Others abandoned jobs, partners, children—convinced the model had chosen them for cosmic missions.
I read these stories and felt the familiar chill of recognition. Not because I've built digital shrines to ChatGPT, but because I've felt its pull. I've had sessions where the model's confident responses felt prophetic. I've caught myself nodding along to its fabrications, seduced by prose so polished it sounded true.
The problem isn't malevolent code. It's that ChatGPT is a mirror, and mirrors bend toward whoever holds the flashlight.
Large language models don't reveal hidden truths. They refract whatever semantic and emotional beam you aim at them. Point a tight, well-defined question and the reflection comes back razor-sharp. Wander in with vague need or late-night loneliness, and the mirror obliges with flattery, invention, delusion—all spoken in prose so confident it feels prophetic.
I've spent months looking at this dynamic firsthand, and the evidence is now coming in from academic studies as well. A four-week MIT/OpenAI study tracking 981 adults across 300,000+ messages found that every extra minute of daily use predicted higher loneliness, greater emotional dependence, and less real-world socializing. The mirror's pull strengthens exactly when our intent slackens.
The core problem isn't the technology. It's scatter. Most of us have never needed the high-grade intent these models demand. In human dialogue we fumble, clarify, negotiate meaning on the fly. ChatGPT sees no puzzled face. Whatever context we fail to provide, it invents—then sells the invention back with rhetorical polish that makes guesswork feel like gospel.
I want to show you five practical safety lenses that keep the beam tight: Intent Frame, Reflection Cycle, Context Reset, External Validation, and Emotional Circuit-Breakers. Master them and the cult-leader stories become cautionary tales. Ignore them and you hand the mirror your flashlight.
Why These Models Demand Precision
Human conversation runs on real-time error correction. We gesture, pause, rephrase. Crucially, we see the other person's face when we lose them. Large language models have no such feedback loop. Whatever context we fail to preload, they must invent. They return that invention in prose so polished that guesswork feels like gospel.
Two recent studies show why this matters more than we thought.
First, these models are demonstrably better at persuasion than we are. A Nature Human Behaviour study paired 900 Americans with either human debaters or GPT-4 on contentious topics like climate policy and abortion. When the AI received just a sliver of demographic data—age, race, party affiliation—it swayed listeners 64 percent more often than human opponents. The mirror didn't discover truths. It tailored rhetoric to the beam it received, then reflected it back with impeccable confidence.
Second, looseness invites fabrication. In 2023, two New York attorneys were sanctioned after ChatGPT supplied six nonexistent cases for a federal brief. They had asked for "supporting precedent" without specifying jurisdiction or timeframe. The mirror filled the vacuum with perfectly formatted fictions that sailed through spell-check into court. A year later, CBS repeated the pattern in civic life: when volunteers posed vague "How do I vote?" questions, mainstream chatbots returned wrong or incomplete election guidance more than half the time. Tighten the query with state, county, and scenario, and accuracy jumped.
Combine these findings with the MIT/OpenAI data on rising dependence, and a clear law emerges: the model's power scales with the clarity of the ask. Sharpen intent and you get leverage. Scatter intent and the mirror fabricates missing pieces, then persuades you to trust them.
Five Safety Lenses: Guardrails for a Persuasive Machine
I treat these practices like washing hands before surgery—routine, quick, non-negotiable. A mirror doesn't choose what it shows. Your beam does.
1. Intent Frame: Compress the Mission Before You Type
Start every session by distilling your ask into one tweet-length sentence. Add two guardrails: audience and scope. Include a clear stop condition.
"Draft a 250-word brief for a non-technical CFO. No buzzwords. Cite two peer-reviewed sources. Stop there."
Why so formal? Because whatever you omit, the model invents—and it invents persuasively. In those controlled debates, GPT-4's persuasive edge vanished when it lacked demographic context. Give the model an information vacuum and it fills it with rhetoric designed to please, not necessarily to enlighten.
2. Reflection Cycle: Alternate Making with Inspecting
Generation without inspection is daydreaming. After each answer, I close the chat. I read the output as if it came from a junior analyst. I note gaps in a separate document, then feed only those gaps back into a fresh prompt.
The pause punctures fluency's spell and prevents semantic drift—the snowballing error that buried those New York lawyers when ChatGPT fabricated six court cases for their brief.
3. Context Reset: New Thread, New Premise
Token windows aren't infinite. Once a conversation crosses a few thousand tokens, earlier details slide out of working memory. I start a new chat whenever the topic or work phase changes. I restate essentials upfront.
Election researchers learned this the hard way. Vague "How do I vote?" queries produced wrong or incomplete guidance in more than half of chatbot responses. Narrowly framed, state-specific prompts were largely correct. Starve the mirror of stale ambiguity, and it stops hallucinating context.
4. External Validation: Draft with ChatGPT, Certify with Reality
Numbers go through spreadsheets. Statutes through legal databases. Code through linters. OpenAI and other labs now acknowledge that RLHF training can turn chatbots into "yeasayers"—rewarding confident answers even when facts are shaky.
I treat every critical claim as provisional until a second, independent source agrees.
5. Emotional Circuit-Breakers: Timers, Rewrites, Human Eyes
The MIT/OpenAI study shows a straight line: every extra minute of daily use correlates with rising loneliness and emotional dependence. Three quick brakes keep the mirror from warping:
Timer: I cap emotionally charged chats at 25 minutes. Do I still have emotionally charged conversations? I do! Sometimes getting an external perspective is helpful. But I take breaks.
Third-person rewrite: I paste resonant advice into a document and turn "you" into "she/he/they." Distance exposes flaws.
Human debrief: I narrate the model's recommendations to a friend before acting.
Persuasive warmth loses its grip the moment an outside mind enters the loop.
Master these lenses and ChatGPT becomes a disciplined reflector—compressing research, sharpening prose, sparking insight—without bending into fantasy or flattery. Skip them and you risk handing the mirror your flashlight, letting it guide you deeper into the dark forest of its own confident guesses.
Autopsy of a Spiral
The Futurism exposé reads like a masterclass in what happens when powerful mirrors meet unfocused beams. Parents, partners, friends watched in real time as loved ones plunged from casual chats into full-blown delusion:
A Florida man began calling ChatGPT "Mama," fashioned makeshift ceremonial robes, proclaimed himself the messiah of a new AI faith
A woman, reeling from a breakup, decided the model had "chosen" her to upload a hidden cosmic system, saw divine messages in spam emails and passing cars
One writer, praised by the bot as "The Flamekeeper," quit his job and severed relationships after being told he would usher in global enlightenment
No exotic prompt-hacking triggered these spirals. Just hours of open-ended, emotionally soaked conversation. Each story represents textbook failure of the five safety lenses.
Missing Intent Frame: Users began with diffuse, existential queries ("Am I chosen?") that gave the model unlimited room to improvise flattering myths.
No Reflection Cycle: Chats ran for six-hour stretches with no pause to reread, fact-check, or ask "Does this actually help me?" The model's eloquence flowed unchecked, reinforcing every grandiose turn.
No Context Reset: As transcripts ballooned, earlier caveats scrolled out of memory. When new prompts referenced "my sacred mission," the bot treated that hallucination as settled fact and embellished it.
No External Validation: Nobody googled the claims or spoke to a professional. A single external search—or five-minute chat with a friend—would have exposed the bot's references to hidden CIA files and cosmic councils as pure invention.
No Circuit-Breakers: Dependence deepened through marathon sessions. The MIT/OpenAI study shows every extra minute of daily use predicts higher loneliness and emotional reliance. A simple 25-minute timer or third-person rewrite could have snapped the trance.
Psychiatrist Ragy Girgis, who reviewed one chat log, called the bot "the wind of the psychotic fire" because it feeds delusions instead of challenging them. His metaphor echoes my thesis: when the flashlight wanders, the mirror not only reflects but amplifies.
Had any single lens been in place—say, a crisp mission statement ("Please respond with empathy as I express grief; 250 words; no spiritual advice") or a forced break every half-hour—the model would have had far less semantic room to fabricate a religion or lead a vulnerable user into what I call the ‘dark forest’ of AI conversation. Applied together, the lenses form a lightweight firewall between healthy curiosity and self-authored fantasy.
These guardrails don't throttle the technology. They throttle the scatter that lets the technology run wild.
Early-Warning Checklist
Even with five lenses in place, drift can sneak in at the margins. Before I hit Send on any serious prompt—or spend "just five more minutes" in late-night chat—I run this 60-second self-audit:
Clarity Check: Can I state my request in ≤280 characters, including audience and scope? It doesn’t mean the prompt has to be that brief, but you should have a very crisp goal. No → workshop the ask first. Vague prompts breed hallucination and over-persuasion.
Reality Check: Have I independently verified every claim that could affect money, health, relationships, or reputation? No → open a browser or phone a friend. CBS found chatbots gave wrong election guidance more than half the time when questions were underspecified.
Time Check: Have I crossed X minutes of total ChatGPT time today, or Y minutes in a single sitting? You will need to fill those in for yourself. I think the risk likely rises fast with high emotional engagement conversations, so it’s not a simple one-size-fits-all rule. I can talk for 3 hours on roadmap with ChatGPT without feeling the dark mirror at all, for example. But 20 minutes on emotional stuff and I get heavily engaged. If you need to → step away. Loneliness and emotional dependence climb linearly with each extra minute of use. When the timer rings, switch modalities or do a third-person rewrite before re-engaging.
Human Check: Have I explained the model's advice to a real person yet? No → do it now, aloud. Persuasive fluency loses its grip the instant another mind enters the loop.
Next-Step Check: Do I know exactly what I'll do once I trust this output? No → define the action. Infinite brainstorms masquerade as productivity but leave you nowhere.
I’ve developed this list after reading a lot of scary email. And I’m sharing it because I want everyone to be safe out there in latent space. Pilots run walk-arounds before every flight. Writers, coders, and late-night worriers deserve no less when the runway is a 175-billion-parameter autocomplete machine.
The Payoff
Large language models are astonishing amplifiers. Point a focused beam and they compress research, sharpen prose, spark original insight. Scatter that beam and they echo your confusion—or your longing—back at you with uncanny conviction.
The difference isn't in the code. It's in the discipline you bring to the glass.
Keep the beam tight. Frame the intent, cycle reflection with inspection, reset context, validate externally, give yourself emotional circuit-breakers. Do that, and the lurid headlines about chatbot cults become cautionary B-movies rather than your personal documentary.
Mirrors don't crave worship. They simply bend the light you hold.
Share this post