An investigation of physicians' evaluations towads artificial intelligence chatbot's support for health literacy

The purpose of this study is to evaluate the medical reliability, potential usefulness, and potential risk to patient safety of software responses to health-related questions posed to an artificial intelligence-based chatbot. The study aims first to determine the performance of the answers in terms of these criteria, and then to formulate recommendations for the safe, beneficial, and non-hazardous use of artificial intelligence. The study is designed as a mixed design phenomenological study. Three questions posed to an AI-enabled chatbot and the answers received from it were evaluated by 88 physicians from different specialties in terms of three criteria (reliability, utility and added value, potential danger). Quantitative analysis of the ratings was performed using descriptive statistics and one-way analysis of variance. The physicians who participated in the survey were also asked to provide written justifications for their ratings of the criteria. These written justifications were divided into subgroups and analyzed to provide a more detailed rationale for the quantitative analysis results. At the end of the study, the results of the quantitative and qualitative analysis were used to relate concepts and emphases and to make recommendations on the use of artificial intelligence chatbots and health literacy in light of physician ratings. Among the recommendations is the creation of a user guide to reduce the risks and increase the benefits of artificial intelligence-based chatbots, which are expected to increase in the coming period, such as threatening patient safety, damaging patient-doctor communication, and negatively affecting patient compliance with treatment. It would be beneficial to conduct research on chatbot usage skills with samples of different demographic and socioeconomic categories, who will be using chatbots for health purposes.