We would like to discuss on the publication “Exploring the potential of artificial intelligence in traumatology: Conversational answers to specific questions”.1 When it came to answering medical queries, ChatGPT had the best accuracy (72.81%), followed by Perplexity (67.54%) and BARD (60.53%) in a study that compared the three chatbot models. Although BARD offered the most accessible and thorough answers, in 14% of the questions all three models failed at the same time. Conversational bots’ inability to effectively handle medical queries was demonstrated by the identification of errors in information and logical reasoning in the responses.
The study found that one of the chatbot models’ weaknesses was its dependence on accuracy as the main performance indicator. In assessing the efficacy of the bots, readability, logical reasoning, and the utilization of outside data should all be taken into account in addition to accuracy. Furthermore, the evaluation's breadth may have been restricted by the technique employed to evaluate the chatbots, which was limited to responding to particular medical queries rather than having a more comprehensive dialogue or offering context-based answers.
Further study in this field may focus on creating better chatbot models that give precedence to external information retrieval and logical reasoning in their responses. Furthermore, investigating methods to incorporate human supervision and input into the chatbot exchanges may assist reduce errors and guarantee the accuracy of the data returned. In order to evaluate the continued advancement and efficacy of conversational bots in the healthcare industry, longitudinal studies may also be carried out, incorporating user feedback and fine-tuning the models according to actual usage scenarios.
Level of evidenceLevel of evidence V.
Ethics of approval statementNot applicable.
Funding statementThere is no funding.
Authors’ contributionsHP 50% ideas, writing, analyzing, approval.
VW 50% ideas, supervision, approval.
Patient consent statementNot applicable.
Permission to reproduce material from other sourcesNot applicable.
Clinical trial registrationNot applicable.
Conflict of interestThe authors declare no conflict of interest.
Data availability statementThere is no new data generated.
