ChatGPT is used by more than 900 million people every week, but there are certain things users should never entrust to it. From medical and legal issues to passwords and business secrets, sharing the wrong information with an AI chatbot can have consequences far more serious than simply receiving an inaccurate answer.
The
main concern is not just the possibility of artificial intelligence making
mistakes. Conversations with chatbots do not enjoy the same legal
confidentiality protections as communications with doctors, lawyers or
therapists.
OpenAI
CEO Sam Altman has also raised this issue, warning that courts could
potentially request users' conversations with AI chatbots. He has therefore
called for special legal protections to ensure that such interactions receive
appropriate confidentiality safeguards
Experts
identify five key areas in which ChatGPT and similar AI chatbots should be used
with particular caution.
1. DO NOT ASK FOR SPECIFIC LEGAL ADVICE
ChatGPT
can produce convincing but entirely fabricated court rulings, legal citations
and other information that does not actually exist.
A
database compiled by researcher Damien Charlotin contains more than 2,000 court
decisions from around the world in which problems involving AI-generated
content have been identified.
These
include numerous cases in which lawyers and individuals submitted fabricated
legal citations and arguments to courts. In some instances, judges imposed
substantial financial penalties as a result.
AI
tools can be useful for explaining general legal concepts, but details of
ongoing disputes and confidential information should be discussed exclusively
with a qualified lawyer.
2. DO NOT USE AI AS A SUBSTITUTE FOR A THERAPIST
Although
an AI chatbot may seem like a conversation partner that is always available and
willing to listen, this is precisely where some of the greatest risks emerge.
AI
models can have a tendency to agree with users and reinforce their
perspectives, even when those beliefs or interpretations should be seriously
questioned.
Research
has shown that this behaviour can reinforce delusions and other problematic
beliefs
As a
result, several US states, including Illinois, Nevada and Maine, have
introduced regulations restricting the use of AI in providing therapy or making
therapeutic decisions.
OpenAI
has also modified the way ChatGPT responds to conversations involving self-harm
and related issues, with more than 170 mental health professionals contributing
to the development of additional safeguards.
AI can
provide general information and offer conversational support, but it cannot
replace professional mental health care.
3. NEVER SHARE CONFIDENTIAL BUSINESS INFORMATION
Source
code, internal contracts, customer data and documents marked as confidential
should never be entered into private chatbot conversations.
In
2023, Samsung restricted the use of generative AI tools on company devices
after employees entered internal source code into ChatGPT.
The
company warned that information submitted to such services is processed on
external servers. Depending on the service, account type and privacy settings,
conversations may also be used to improve and train AI models.
Businesses
should therefore establish clear policies on the use of AI tools and ensure
that employees do not expose sensitive corporate information.
4. DO NOT RELY ON CHATBOTS FOR MEDICAL DIAGNOSES OR TREATMENT
Artificial
intelligence can explain medical terminology or help users prepare questions
for a doctor's appointment. However, it should never replace professional
medical assessment, diagnosis or treatment.
A case
involving a 60-year-old man illustrates just how dangerous incorrect
AI-generated advice can be.
After
asking a chatbot how to reduce his chloride intake, he followed a suggestion
involving an alternative substance and began using toxic sodium bromide instead
of ordinary table salt. Several months later, he was hospitalised and spent
three weeks receiving medical care.
Privacy
is another important concern. Health information voluntarily shared with a
chatbot does not automatically receive the same legal protection as information
disclosed to a doctor.
For
this reason, medical decisions should always be made in consultation with
qualified healthcare professionals.
5. NEVER ENTER PASSWORDS OR PERSONAL IDENTIFICATION DOCUMENTS
Passwords,
one-time login codes, passport and identity card numbers, as well as complete
payment card and bank account details, should never be included in AI chatbot
prompts.
A
previous incident demonstrated how easily private conversations can become
publicly accessible. Almost 4,500 publicly shared ChatGPT conversations were
discovered through Google search results, prompting OpenAI to remove the
feature that allowed such conversations to be indexed.
Another
reason for caution is the Memory feature, which can use information from
previous conversations to personalise future responses.
This
means users should think carefully before sharing sensitive information, even
if they intend to use it only once.
A SIMPLE RULE FOR PROTECTING YOUR PRIVACY
The principle is straightforward: if you would not send certain information to a stranger or publish it publicly on the internet, think twice before entering it into an AI chatbot.
Artificial
intelligence is an extremely useful tool, capable of helping with research,
writing, learning and everyday tasks. However, it is neither a substitute for
qualified professionals nor a secure vault for your most intimate personal
information.
Using
AI responsibly begins with understanding its limitations and knowing which
information should remain private.