HQ Team
November 19, 2024: The emergence of AI technology as a diagnostic tool is slowly changing how health professionals view patient care. Integrating AI technologies, such as machine learning (ML) and deep learning (DL), into diagnostic processes enhances accuracy, efficiency, and the potential for personalized medicine.
Research is ongoing into the perceived and actual advantages of AI diagnostic tools in arriving at a consensus about diagnosis and what route patient care should take. A latest study adds to the debate.
A randomized clinical trial was conducted to compare the diagnostic reasoning performance of physicians using a commercial LLM AI chatbot (ChatGPT Plus [GPT-4]; OpenAI) compared with conventional diagnostic resources (eg, UpToDate, Google).
Fifty physicians with training in a general medical speciality (internal medicine, family medicine, or emergency medicine) at Stanford University, Beth Israel Deaconess Medical Center, and the University of Virginia were recruited for the study for a month last year.
Of these, 39 (78%) participated in virtual encounters and 11 (22%) were in-person. Median years in practice was 3 (IQR, 2-8).
Study results
The study results were unconventional. Doctors who were given ChatGPT-4 along with conventional resources did only slightly better than doctors who did not have access to the bot. And ChatGPT alone outperformed the doctors.
The chatbot scored an average of 90 percent when diagnosing a medical condition from a case report and explaining its reasoning. Doctors randomly assigned to use the chatbot got an average score of 76 percent. Those randomly assigned not to use it had an average score of 74 percent.
The methodology involved giving the participants six case histories and testing them on their diagnostic abilities to explain why they favored or ruled out a certain diagnosis.
The graders were medical experts who saw only the participants’ answers, not knowing whether they were from a doctor with ChatGPT, a doctor without it, or from ChatGPT alone.
The results unveiled doctors’ cognitive biases and belief in their intuitive or experiential diagnosis even when a chatbot potentially suggested a better one. It also exposed the unpreparedness of physicians to take advantage of the A.I. systems’ ability to solve complex diagnostic problems and offer explanations for their diagnoses.
Although the study was small and researchers suggest a deeper dive with a large study base to arrive at any final conclusions, but the future leans towards integrating ML and AI with clinical practices.
AI’s beneficial role in diagnostics
AI applications span numerous diseases, with specific models achieving significant diagnostic performance. For example, a recurrent neural network (RNN) achieved a 97.59% accuracy rate in diagnosing liver diseases. Furthermore, deep learning models have been shown to detect pneumonia with a sensitivity of 96%, outperforming radiologists.
Studies have demonstrated that AI models can achieve high accuracy rates in diagnosing skin diseases using real-world data. Techniques such as Convolutional Neural Networks (CNNs) and support vector machines (SVMs) have been pivotal in these advancements.
Enhanced Accuracy: AI algorithms can outperform traditional diagnostic methods by reducing human error and improving sensitivity and specificity in detecting diseases. For example, a study indicated that AI could diagnose breast cancer with a sensitivity of 90%, compared to 78% for human radiologists. This consistent accuracy is crucial in critical areas like oncology and emergency medicine.
Early Detection: By identifying early signs of diseases like cancer or diabetic retinopathy, AI facilitates timely interventions. It provides speed and efficiency by being able to process vast amounts of datasets and facilitating decision-making
Personalized Medicine: AI can tailor treatment plans based on individual patient data, optimizing outcomes.
Challenges and limitations
Despite the promising advancements, several challenges remain in the integration of AI into clinical practice
Inaccurate or biased datasets: The effectiveness of AI models is highly dependent on the quality and comprehensiveness of training datasets. Inadequate or biased data can lead to inaccurate diagnoses.
The rapid evolution of AI technologies poses challenges for regulatory frameworks, which must adapt to ensure safety and efficacy.
Integrating and incorporating AI tools into existing healthcare systems along with proper training of clinical practitioners in the right usage requires significant changes in workflow and training.
