Artificial intelligence has become almost impossible to avoid, and for many, it’s slowly replacing Google and other search engines. But while ChatGPT and its competitors can be useful, it’s important to remember they aren’t always equipped with factual information. These systems often deliver responses that are biased, outdated or entirely fabricated. Thus, they should never be completely trusted, no matter how confident their responses may seem.
If you’re using AI to generate a silly poem for the holidays or asking for some potential gift ideas this year, an AI hallucination is relatively harmless. Writing is meant to be creative and gifts can be returned. However, relying on it for guidance regarding your finances, medical health or legal matters is a recipe for disaster. With those questions, a single incorrect answer can lead to serious real-world consequences.
To help you navigate the limits of the technology, here are 11 things you should never rely on ChatGPT for.
(Disclosure: Ziff Davis, the parent company of CNET, in April filed a lawsuit against ChatGPT maker OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)
Don’t miss any of our unbiased tech content and lab-based reviews. Add CNET as a preferred Google source.
1. Diagnosing physical health issues
I’ve definitely fed ChatGPT my symptoms out of curiosity, but the answers that come back can read like your worst nightmare. As you pore over potential diagnoses, you could swing from dehydration and the flu to some type of cancer. I have a lump on my chest and entered that information into ChatGPT. Lo and behold, it told me I may have cancer. In fact, I have a lipoma, which is not cancerous and occurs in one in every 1,000 people. My licensed doctor told me that.
I’m not saying there are no good uses of ChatGPT for health: It can help you draft questions for your next appointment, translate medical jargon and organize a symptom timeline so you can walk in better prepared. And that could help make doctor visits less overwhelming. However, AI can’t order labs or examine you, and it definitely doesn’t carry malpractice insurance. Know its limits.
Read more: ChatGPT Health: What the New Dedicated Tab Adds to the AI Chatbot
2. Taking care of your mental health
ChatGPT can offer grounding techniques, sure, but it can’t pick up the phone when you’re in real trouble with your mental health. I know some people use ChatGPT as a substitute therapist. CNET’s Corin Cesaric found it mildly helpful for working through grief, as long as she kept its limits front of mind. But as someone who has a very real, very human therapist, I can tell you that ChatGPT is still really only a pale imitation at best, and incredibly risky at worst.
ChatpGPT doesn’t have lived experience, can’t read your body language or tone, and has zero capacity for genuine empathy. It can only simulate it. A licensed therapist operates under legal mandates and professional codes that protect you from harm. ChatGPT doesn’t. Its advice can misfire, overlook red flags or unintentionally reinforce biases baked into its training data. Leave the deeper work — the hard, messy, human work — to an actual human who is trained to properly handle it. If you or someone you love is in crisis, please dial 988 in the US, or your local hotline.
3. Making immediate safety decisions
If your carbon-monoxide alarm starts chirping, please don’t open ChatGPT and ask it if you’re in real danger. I’d go outside first and ask questions later. Large language models can’t smell gas, detect smoke or dispatch an emergency crew. In a crisis, every second you spend typing is a second you’re not evacuating or dialing 911. ChatGPT can only work with the scraps of info you feed it, and in an emergency, it may be too little and too late. So treat your chatbot as a post-incident explainer, never a first responder.
4. Getting personalized financial or tax planning
ChatGPT can explain what an ETF is, but it doesn’t know your debt-to-income ratio, state tax bracket, filing status, deductions, retirement goals or risk appetite. Because its training data may stop short of the current tax year and of the latest rate hikes, its guidance may well be stale when you hit enter.
I have friends who dump their 1099 totals into ChatGPT for a DIY return. The chatbot simply can’t replace a CPA who can catch a hidden deduction worth a few hundred dollars or flag a mistake that could cost you thousands. When real money, filing deadlines, and IRS penalties are on the line, call a professional, not AI. Also, be aware that anything you share with an AI chatbot will probably become part of its training data, and that includes your income, your Social Security number and your bank routing information.
5. Dealing with confidential or regulated data
As a tech journalist, I see embargoes land in my inbox every day, but I’ve never thought about tossing any of these press releases into ChatGPT to get a summary or further explanation. That’s because if I did, that text would leave my control and land on a third-party server outside the guardrails of my nondisclosure agreement.
The same risk applies to client contracts, medical charts or anything covered by the California Consumer Privacy Act, HIPAA, the GDPR or plain old trade-secret law. It applies to your income taxes, birth certificate, driver’s license and passport. Once sensitive information is in the prompt window, you can’t guarantee where it’s stored, who can review it internally or whether it may be used to train future models. ChatGPT also isn’t immune to hackers and security threats. If you wouldn’t paste it into a public Slack channel, don’t paste it into ChatGPT.
6. Doing anything illegal
This one is self-explanatory.
7. Cheating on schoolwork
I’d be lying if I said I never cheated on my exams. In high school, I used my first-generation iPod Touch to sneak a peek at a few cumbersome equations I had difficulty memorizing in AP calculus, a stunt I’m not particularly proud of. But with AI, the scale of modern cheating makes that look remarkably tame.
Turnitin and similar detectors are getting better at spotting AI-generated prose every semester, and professors can already hear “ChatGPT voice” a mile away (thanks for ruining my beloved em dash). Suspension, expulsion and getting your license revoked are real risks. It’s best to use ChatGPT as a study buddy, not a ghostwriter. You’re also just cheating yourself out of an education if you have ChatGPT do the work for you.
8. Monitoring information and breaking news
Since OpenAI rolled out ChatGPT Search in late 2024 (and opened it to everyone in February 2025), the chatbot can fetch fresh web pages, stock quotes, gas prices, sports scores and other real-time numbers the moment you ask, complete with clickable citations so you can verify the source. However, it won’t stream continual updates on its own. Every refresh needs a new prompt, so when speed is critical, live data feeds, official press releases, news sites, push alerts and streaming coverage are still your best bet.
9. Gambling
I’ve actually had luck with ChatGPT and hitting a three-way parlay during the NCAA men’s basketball championship, but I would never recommend it to anyone. I’ve seen ChatGPT hallucinate and provide incorrect information on player statistics, or misreport injuries and win-loss records. I only cashed out because I double-checked every claim against real-time odds, and even then, I got lucky. ChatGPT can’t predict tomorrow’s box score, so don’t rely on it solely to secure a win.
10. Drafting a will or other legally binding contract
ChatGPT is great for breaking down basic concepts. If you want to know more about a revocable living trust, ask away. However, the moment you ask it to draft actual legal text, you’re rolling the dice. Estate and family-law rules vary by state, and sometimes even by county, so skipping a witness signature or omitting the notarization clause can get your whole document tossed. Rather, let ChatGPT help you build a checklist of questions for your lawyer, then pay that lawyer to turn that checklist into a document that stands up in court.
11. Making art
This isn’t an objective truth, just my own opinion, but I don’t believe AI should be used to create art. I’m not anti-artificial intelligence by any means. I use ChatGPT for brainstorming new ideas and help with my headlines, but that’s supplementation, not substitution. By all means, use ChatGPT, but please don’t use it to make art that you then pass off as your own. It’s kind of gross.
