Evaluation of Artificial Intelligence and Large Language Models (LLMs) in Clinical Practice: Precision, Safety, and Applicability in Middle-Income Countries
Authors
Byron Hernán Mena Acosta (Author)
Key Information
Journal
Published in Vol. 1 No. 2 (2026) of Loja Medical Journal.
About the JournalAbstract
Introduction: Large language models (LLMs) are being investigated to support diagnosis, communication, and clinical decision-making. Performance depends on the task, model, and evaluation setting, limiting extrapolation to middle-income countries. Objective: To analyze evidence on the performance and clinical safety of LLMs and other artificial intelligence systems and discuss their applicability in middle-income countries, with an emphasis on Ecuador. Methods: A narrative review with a structured search of PubMed and Google Scholar conducted in May 2026, targeting publications from 2022 through that date. One author selected sources and extracted data. Twelve publications addressing performance or implementation were synthesized, distinguishing LLM evaluations, other AI systems, and documents without comparative clinical results. No meta-analysis or formal risk-of-bias assessment was performed. Results: In obstetrics and gynecology scenarios, the evaluated LLMs as a group achieved an overall accuracy of 73.75%, compared with 65.35% among residents. Conversely, an evaluation of abdominal pain cases found performance inferior to physicians. Ratings of potential harm in Med-PaLM 2 answers did not establish equivalent safety in patient care. Studies of communication, prognosis, and automated decision support measured different outcomes and could not support a pooled estimate. Connectivity, interoperability, local validation, and costs influence applicability in Ecuador. Conclusions: LLMs show potential as supervised support for specific tasks, but the reviewed evidence does not establish general clinical superiority or safety equivalent to specialists. Implementation requires prospective local evaluation, data protection, training, and performance monitoring.
References
1. World Health Organization. Noncommunicable diseases [Internet]. Geneva: WHO; 2025 [citado 18 may 2026]. Disponible en: https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases
2. World Health Organization. Health workforce [Internet]. Geneva: WHO [citado 18 may 2026]. Disponible en: https://www.who.int/health-topics/health-workforce
3. Teles AS, Abd-Alrazaq A, Heston TF, Damseh R, Ruback L. Editorial: Large Language Models for medical applications. Front Med (Lausanne). 2025;12:1625293. doi:10.3389/fmed.2025.1625293. DOI: https://doi.org/10.3389/fmed.2025.1625293
4. Ríos-Hoyo A, Shan NL, Li A, Pearson AT, Pusztai L, Howard FM. Evaluation of large language models as a diagnostic aid for complex medical cases. Front Med (Lausanne). 2024;11:1380148. doi:10.3389/fmed.2024.1380148. DOI: https://doi.org/10.3389/fmed.2024.1380148
5. Artsi Y, Sorin V, Glicksberg BS, Korfiatis P, Freeman R, Nadkarni GN, et al. Challenges of Implementing LLMs in Clinical Practice: Perspectives. J Clin Med. 2025;14(17):6169. doi:10.3390/jcm14176169. DOI: https://doi.org/10.3390/jcm14176169
6. Gao Y, Feng Y, Jahng SG. Implicit bias in digital health: systematic biases in large language models’ representation of global public health attitudes and challenges to health equity. Front Public Health. 2025;13:1705082. doi:10.3389/fpubh.2025.1705082. DOI: https://doi.org/10.3389/fpubh.2025.1705082
7. Santorum Gaibor M, Carrión Toro M. ¿Conectados pero no incluidos? Brecha digital e inclusión en el Ecuador (2013-2025) [Internet]. Quito: Escuela Politécnica Nacional, ÁPICE y LudoLab; 2025. Disponible en: https://apice.epn.edu.ec/images/apice/noticias/boletin/Conectados%20pero%20no%20incluidos%20vf.pdf
8. Ministerio de Salud Pública del Ecuador. Política Nacional de Transformación Digital del Sector Salud 2024-2034 [Internet]. Quito: Ministerio de Salud Pública; 2024. Disponible en: https://www.salud.gob.ec/wp-content/uploads/2025/07/polItica_nacional_de_transformaciOn_digital_del_sector_salud_14nov2024___.pdf
9. PRISMA. PRISMA 2020 flow diagram [Internet]. PRISMA Statement [citado 22 oct 2026]. Disponible en: https://www.prisma-statement.org/prisma-2020-flow-diagram
10. Martinelli C, Giordano A, Carnevale V, Burk SR, Porto L, Vizzielli G, et al. The PERFORM study: Artificial intelligence versus human residents in cross-sectional obstetrics-gynecology scenarios across languages and time constraints. Mayo Clin Proc Digit Health. 2025;3(2):100206. doi:10.1016/j.mcpdig.2025.100206. DOI: https://doi.org/10.1016/j.mcpdig.2025.100206
11. Hager P, Jungmann F, Holland R, Bhagat K, Hubrecht I, Knauer M, et al. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nat Med. 2024;30(9):2613-2622. doi:10.1038/s41591-024-03097-1. DOI: https://doi.org/10.1038/s41591-024-03097-1
12. Rotem R, Simon C, Rottenstreich M, O’Reilly BA, Weintraub AY, O’Sullivan OE. ChatGPT in urogynecology: Comparing large language model responses to human experts. Acta Obstet Gynecol Scand. 2026;105(8):1629-1635. Publicado en línea el 23 de octubre de 2025. doi:10.1111/aogs.70085. DOI: https://doi.org/10.1111/aogs.70085
13. Sarraju A, Bruemmer D, Van Iterson E, Cho L, Rodriguez F, Laffin L. Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model. JAMA. 2023;329(10):842-844. doi:10.1001/jama.2023.1044. DOI: https://doi.org/10.1001/jama.2023.1044
14. Pedro T, Sousa JM, Fonseca L, Gama MG, Moreira G, Pintalhão M, et al. Exploring the use of ChatGPT in predicting anterior circulation stroke functional outcomes after mechanical thrombectomy: a pilot study. J Neurointerv Surg. 2025;17(3):261-265. doi:10.1136/jnis-2024-021556. DOI: https://doi.org/10.1136/jnis-2024-021556
15. Singhal K, Tu T, Gottweis J, Sayres R, Wulczyn E, Amin M, et al. Toward expert-level medical question answering with large language models. Nat Med. 2025;31(3):943-950. doi:10.1038/s41591-024-03423-7. DOI: https://doi.org/10.1038/s41591-024-03423-7
16. Sandmann S, Hegselmann S, Fujarski M, Bickmann L, Wild B, Eils R, et al. Benchmark evaluation of DeepSeek large language models in clinical decision-making. Nat Med. 2025;31(8):2546-2549. doi:10.1038/s41591-025-03727-2. DOI: https://doi.org/10.1038/s41591-025-03727-2
17. Chen B, Li Y, Sun Y, Sun H, Wang Y, Lyu J, et al. A 3D and explainable artificial intelligence model for evaluation of chronic otitis media based on temporal bone computed tomography: Model development, validation, and clinical application. J Med Internet Res. 2024;26:e51706. doi:10.2196/51706. DOI: https://doi.org/10.2196/51706
18. Huang X, Zhou S, Ma X, Jiang S, Xu Y, You Y, et al. Effectiveness of an artificial intelligence clinical assistant decision support system to improve the incidence of hospital-associated venous thromboembolism: a prospective, randomised controlled study. BMJ Open Qual. 2023;12(4):e002267. doi:10.1136/bmjoq-2023-002267. DOI: https://doi.org/10.1136/bmjoq-2023-002267
19. Wu Y, Min H, Li M, Shi Y, Ma A, Han Y, et al. Effect of Artificial Intelligence-based Health Education Accurately Linking System (AI-HEALS) for Type 2 diabetes self-management: protocol for a mixed-methods study. BMC Public Health. 2023;23:1325. doi:10.1186/s12889-023-16066-z. DOI: https://doi.org/10.1186/s12889-023-16066-z
20. de Marques DG, Summer H, Barreto GLL, de Miranda AFS, da Silva MRB, de Vasconcelos G, et al. Cuida-AI: Virtual assistant for optimizing oncology care in public hospitals in Recife Brazil. In: Proceedings of 20th Iberian Conference on Information Systems and Technologies (CISTI 2025). Lecture Notes in Networks and Systems. Vol. 1716. Cham: Springer; 2026. p. 73-85. doi:10.1007/978-3-032-10929-3_7. DOI: https://doi.org/10.1007/978-3-032-10929-3_7
21. Amores Zurita D. Sistema de Salud Pública: una crisis que cobra vidas [Internet]. INREDH; 23 abr 2026 [citado 24 may 2026]. Disponible en: https://inredh.org/sistema-de-salud-publica-una-crisis-que-cobra-vidas/
Details
License
Copyright (c) 2026 Loja Medical Journal
This work is licensed under a Creative Commons Attribution 4.0 International License.
This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided that the original work is properly cited.
