Background Artificial intelligence (AI) is a rapidly developing field with the potential to transform various aspects of healthcare and public health including medical training. During the ‘Hygiene and Public Health’ course for fifth-year medical students, a practical training session was conducted on vaccination using AI as an educational supportive tool. Before receiving specific training on vaccination, the students were given an online questionnaire on vaccination. After completing the questionnaire, a correction of each question was performed using AI chatbots. Objectives The main aim was to identify whether AI chatbots can be considered as educational support tool for training in public health. Secondary objective was to assess performance of 6 different AI chatbot models. Results The questionnaire consisted of 15 items selected from the Italian National Medical Residency Test (years 2015-2022) on the topic of vaccination. 36 medical students participated in the training, achieving a mean score of 8.22/15 on the quiz (range:3-15,median:8). 6 different AI chatbot models (3 modes of BingChat, ChatGPT, ChatSonic, and YouChat) were asked the same questions and achieved a mean score of 12.17/15 (range:10-15,median:11.5), ranking in the top 25% of medical student scores. During the class, the answers provided by the AI chatbots were evaluated with the students, paying particular attention to the completeness of the information, reliability of the sources cited, and use of technical language. Conclusions AI chatbots have shown to be efficient tools for retrieving bibliographic references from the web and answering questions related to specific topics such as vaccination. However, it is important to note that they cannot be considered more intelligent than students, as they may not perform as well when faced with situational questions. The responsible and critical use of AI should be integrated into medical education to ensure its effective utilization.
Artificial Intelligence Chatbots for medical training in Public Health: a tool to explore
Baglivo, F;De Angelis, L;Casigliani, V;Arzilli, G;Privitera, G P;Rizzo, C
2023-01-01
Abstract
Background Artificial intelligence (AI) is a rapidly developing field with the potential to transform various aspects of healthcare and public health including medical training. During the ‘Hygiene and Public Health’ course for fifth-year medical students, a practical training session was conducted on vaccination using AI as an educational supportive tool. Before receiving specific training on vaccination, the students were given an online questionnaire on vaccination. After completing the questionnaire, a correction of each question was performed using AI chatbots. Objectives The main aim was to identify whether AI chatbots can be considered as educational support tool for training in public health. Secondary objective was to assess performance of 6 different AI chatbot models. Results The questionnaire consisted of 15 items selected from the Italian National Medical Residency Test (years 2015-2022) on the topic of vaccination. 36 medical students participated in the training, achieving a mean score of 8.22/15 on the quiz (range:3-15,median:8). 6 different AI chatbot models (3 modes of BingChat, ChatGPT, ChatSonic, and YouChat) were asked the same questions and achieved a mean score of 12.17/15 (range:10-15,median:11.5), ranking in the top 25% of medical student scores. During the class, the answers provided by the AI chatbots were evaluated with the students, paying particular attention to the completeness of the information, reliability of the sources cited, and use of technical language. Conclusions AI chatbots have shown to be efficient tools for retrieving bibliographic references from the web and answering questions related to specific topics such as vaccination. However, it is important to note that they cannot be considered more intelligent than students, as they may not perform as well when faced with situational questions. The responsible and critical use of AI should be integrated into medical education to ensure its effective utilization.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
