New York lawmakers want AI chatbots to stop pretending to be doctors or lawyers

A proposed bill would ban professional advice from AI, and allow users to sue if they’re harmed.

[Photo: Oscar Wong/Getty Images]

New York is the latest state to consider a bill that would prohibit AI chatbots from dispensing advice that licensed professionals would normally give, such as medical or legal advice. The bill would also allow people who believe they were harmed by such advice to sue the operator of the chatbot.

Senate Bill S7263, introduced by Democratic state Senator Kristen Gonzalez, passed out of a technology committee on a 6–0 vote last week and now advances to a reading on the floor of the Senate. Interestingly, the bill requires operators to clearly label their chatbots as AI, but stipulates that such a label isn’t enough to shield them from lawsuits under the statute.

The proposal reflects a growing shift in how policymakers are thinking about AI. While early efforts focused mostly on transparency, lawmakers are beginning to explore something arguably more consequential: whether companies should be legally liable when AI systems give advice that causes real-world harm.

The bill applies to chatbots that give advice in the fields of medicine, law, dentistry, veterinary medicine, physical therapy, pharmacy, nursing, podiatry, optometry, engineering, land surveying, geology, architecture, psychology, and social work.

New York isn’t acting alone. Other states have passed or are considering similar laws, though with varying scopes and enforcement methods, and they tend to focus primarily on healthcare:

California’s AB 489, enacted in 2025, does something similar but with a narrower scope, targeting AI systems that misrepresent their information as coming from licensed healthcare professionals. But AB 489 relies on state healthcare boards for enforcement and doesn’t provide a private right of action (civil suit) for legal recourse.

A new Nevada law, AB 406, which went into effect last July, prohibits the advertising and operation of AI systems designed to dispense professional mental and behavioral healthcare therapy. The law also limits how licensed professionals can use AI in their practices.

Last August, Illinois passed HB 1806, which prohibits licensed therapists in the state from using AI to make treatment decisions or communicate with clients. The law also prohibits tech companies from advertising or offering AI-powered therapy services in the state without the involvement of a licensed professional.

Utah passed a similar law, HB 452, that puts restrictions and disclosure requirements on chatbots that appear to offer an alternative to human mental health therapy or advice. The law went into effect in 2025.

Professional medical groups have also begun weighing in on the risks. The American Medical Association doesn’t call for a ban on AI chatbots dispensing health information, but it worries that consumer advice from LLMs might be false or misleading. “Notably, tools such as ChatGPT have shown a not-uncommon tendency to falsify references cited in response to these queries,” the AMA says in a policy paper, adding that AI tools have demonstrated the ability to generate fraudulent scientific or medical literature to support health advice.

An especially sensitive area is mental health advice. Mental health advice is a particularly sensitive area, perhaps because many chatbot users, especially younger ones, use AI as a counselor or therapist. A 2025 JAMA Network study found that 13% of all respondents used chatbots for mental health advice, with 22% of those ages 18 to 21 doing so.

Artificial Intelligence

Claire's went from tween mall icon to bankrupt — twice?

© Fast Company