Objective: Artificial intelligence models human thinking and problem-solving abilities, allowing computers to make autonomous decisions. There is a lack of studies demonstrating the clinical utility of GPT and Gemin in the field of toxicology, which means their level of competence is not well understood. This study compares the responses given by GPT-3.5 and Gemin to those provided by emergency medicine residents.
Methods: This prospective study was focused on toxicology and utilized the widely recognized educational resource ‘Tintinalli Emergency Medicine: A Comprehensive Study Guide’ for the field of Emergency Medicine. A set of twenty questions, each with five options, was devised to test knowledge of toxicological data as defined in the book. These questions were then used to train ChatGPT GPT-3.5 (Generative Pre-trained Transformer 3.5) by OpenAI and Gemini by Google AI in the clinic. The resulting answers were then meticulously analyzed.
Results: 28 physicians, 35.7% of whom were women, were included in our study. A comparison was made between the physician and AI scores. While a significant difference was found in the comparison (F=2.368 and p<0.001), no significant difference was found between the two groups in the post-hoc Tukey test. GPT-3.5 mean score is 9.9±0.71, Gemini mean score is 11.30±1.17 and, physicians’ mean score is 9.82±3.70 (Figure 1).
Conclusions: It is clear that GPT-3.5 and Gemini respond similarly to topics in toxicology, just as resident physicians do.
This work is licensed under a Creative Commons Attribution 4.0 International License.