Artificial intelligence-backed chatbots may be just as effective as medical professionals when it comes to providing patients with medical information, according to a recent study.
Researchers from the University of California, San Diego, John Hopkins, and other universities asked OpenAI’s ChatGPT 195 medical questions and compared the quality and compassion of the chatbot’s answers to those provided by physicians on Reddit.
A group of healthcare professionals, including internal medicine specialists, pediatricians, oncologists, and infectious disease specialists, evaluated both the chatbot and human responses on a five-point scale, rating the “quality of information” and “compassion or bedside manner” provided. In 78.6% of the 585 scenarios evaluated, the clinicians preferred the chatbot’s answers over those provided by the physicians. The chatbot’s answers were rated 3.6 times higher in quality and 9.8 times higher in empathy than those of the doctors.
One reason the chatbot outperformed physicians is that its responses to queries were more detailed and personalized than the doctors’ time-saving solutions. For example, when asked whether it’s possible to go blind after getting bleach in the eye, ChatGPT responded with empathy, “I’m sorry to hear that you got bleach splashed in your eye,” and offered four other sentences of description with clear instructions on how to wash it. The doctor, on the other hand, simply said, “Sounds like you will be fine” and instructed the patient to “flush the eye” or call poison control.
However, the study did note that ChatGPT is not capable of diagnosing on its own, and it is still not a physician. The chatbot may be useful for providing patients with basic medical information and advice, but it is not a substitute for a doctor’s diagnosis or medical care.
Despite the limitations, this study is a promising sign for the future of AI in healthcare. Chatbots like ChatGPT could help ease the burden on medical professionals and provide patients with quick and accurate information, especially in situations where doctors are not readily available. As AI technology continues to advance, it will be interesting to see how it can further benefit the medical industry and improve patient care.
