Artificial Intelligence Chatbots as Sources of Information on Oral Cancer: Assessment of Readability and Discursive Contente
DOI:
https://doi.org/10.32635/2176-9745.RBC.2026v72n3.5731Keywords:
Generative Artificial Intelligence, Comprehension, Health Communication, Mouth NeoplasmsAbstract
Introduction: The growing use of artificial intelligence (AI) chatbots as sources of health information requires a critical evaluation of the readability and communicative adequacy of their responses, particularly regarding sensitive topics such as oral cancer. Objective: To evaluate and compare the readability of responses provided by different AI chatbots to basic questions about oral cancer and to describe the discursive content generated by these platforms. Method: This observational and descriptive study evaluated ten AI chatbots that answered seven standardized questions. Textual readability was assessed using an automated tool by calculating a global mean score. Additionally, the Discourse of the Collective Subject technique was applied to synthesize the convergent content of the responses for each question. Results: The global mean readability score was 12.9, a borderline value classified as high readability, indicating a required educational level equivalent to completion of secondary education, with variations across platforms and question types. Convergence was observed around a technically adequate biomedical core, associated with recurrent expansion of content beyond the scope of the questions. Conclusion: Although AI chatbots show potential to expand access to information on oral cancer, persistent limitations related to readability, content organization, and communicative adequacy remain, indicating the need for human curation to ensure complementary and equitable use in health contexts.
Downloads
References
Ferlay J, Ervik M, Lam M, et al. Global Cancer Observatory: cancer today [Internet]. Lyon: International Agency for Research on Cancer; 2024 [acesso 2025 nov 10]. Disponível em: https://gco.iarc.who.int/today
Instituto Nacional de Câncer. Estimativa 2026: incidência de câncer no Brasil [Internet]. Rio de Janeiro: INCA; 2026. [acesso 2026 maio 9]. Disponível em: https://ninho.inca.gov.br/jspui/bitstream/123456789/17914/1/Estima2026_completo%20%281%29.pdf
Coletta RD, Yeudall WA, Salo T. Current trends on prevalence, risk factors and prevention of oral cancer. Front Oral Health. 2024;5:1505833. doi: https://doi.org/10.3389/froh.2024.1505833 DOI: https://doi.org/10.3389/froh.2024.1505833
Casotti E, Almeida PFD, Silva AND. Trajetórias assistenciais de usuários com câncer de boca na busca por cuidados na Rede de Atenção à Saúde. Physis. 2025;35(2):e350217. doi: https://doi.org/10.1590/S0103-73312025350217 DOI: https://doi.org/10.1590/s0103-73312025350217pt
Silva CHF, Queiroz NB, Benedito FCS, et al. Conhecimento de usuários da Estratégia Saúde da Família sobre o câncer de boca. Arq Odontol. 2018;54. doi: https://doi.org/10.7308/aodontol/2018.54.e07 DOI: https://doi.org/10.7308/aodontol/2018.54.e07
Prado NS, Bonan RF, Leonel AC, et al. Awareness on oral cancer among patients attending dental school clinics in Brazil. Med Oral Patol Oral Cir Bucal. 2020;25(1):e89-e95. doi: https://doi.org/10.4317/medoral.23207 DOI: https://doi.org/10.4317/medoral.23207
Powell J, Inglis N, Ronnie J, et al. The characteristics and motivations of online health information seekers. J Med Internet Res. 2011;13(1):e20. doi: https://doi.org/10.2196/jmir.1600 DOI: https://doi.org/10.2196/jmir.1600
Gkinko L, Elbanna A. AI chatbots sociotechnical research: an overview and future directions. In: Bednar P, Islind AS, Vallo Hult H, et al., editores. Proceedings of the 8th International Workshop on Socio-Technical Perspective in Information Systems Development (STPIS 2022) [Internet]; 2022 ago 19-20 [acesso 2025 nov 10]; Reykjavik, Iceland. CEUR Workshop Proceedings. 2022;3239:173-183. Disponível em: https://ceur-ws.org/Vol-3239/paper17.pdf
Aggarwal A, Tam CC, Wu D, et al. Artificial intelligence–based chatbots for promoting health behavioral changes: systematic review. J Med Internet Res. 2023;25:e40789. doi: https://doi.org/10.2196/40789 DOI: https://doi.org/10.2196/40789
Conway DI, Petticrew M, Marlborough H, et al. Socioeconomic inequalities and oral cancer risk: a systematic review and meta-analysis of case-control studies. Int J Cancer. 2008;122(12):2811-9. doi: https://doi.org/10.1002/ijc.23430 DOI: https://doi.org/10.1002/ijc.23430
Okuhara T, Furukawa E, Okada H, et al. Readability of written information for patients across 30 years. Patient Educ Couns. 2025;135:108656. doi: https://doi.org/10.1016/j.pec.2025.108656 DOI: https://doi.org/10.1016/j.pec.2025.108656
Singh S, Jamal A, Qureshi F. Readability metrics in patient education: where do we innovate? Clin Pract. 2024;14(6):2341-9. doi: https://doi.org/10.3390/clinpract14060183 DOI: https://doi.org/10.3390/clinpract14060183
Moreno GCDL, Souza MPM, Hein N, et al. ALT: um software para análise de legibilidade de textos em língua portuguesa. Policromias. 2023;8(1):91-128. doi: https://doi.org/10.61358/policromias.v8i1.54352 DOI: https://doi.org/10.61358/policromias.v8i1.54352
Figueiredo MZA, Chiari BM, Goulart BNG. Discurso do Sujeito Coletivo: uma breve introdução à ferramenta de pesquisa qualiquantitativa. Distúrb comum [Internet]. 2013[acesso 2026 jan 11];25(1):129-36. Disponível em: https://revistas.pucsp.br/index.php/dic/article/view/14931/11139
Genaro LE, Marconato JV, Pinotti FE, et al. Discurso do sujeito coletivo: bases conceituais, enfoques e aplicações em pesquisas qualitativas. Rev Pesq Qual. 2024;12(32):553-69. doi: https://doi.org/10.33361/RPQ.2024.v.12.n.32.739 DOI: https://doi.org/10.33361/RPQ.2024.v.12.n.32.739
NVivo [Internet]. Versão 14.24.3 [Denver]: Lumivero; [sem data]. [acesso 2025 dez 1]. Disponível em: https://lumivero.com/products/nvivo/
Lefevre F, Lefevre AMC, Marques MCDC. Discurso do sujeito coletivo, complexidade e auto-organização. Ciênc. saúde colet. 2009;14(4):1193-204. doi: https://doi.org/10.1590/S1413-81232009000400025 DOI: https://doi.org/10.1590/S1413-81232009000400025
Conselho Nacional de Saúde (BR). Resolução n° 510, de 7 de abril de 2016. Dispõe sobre as normas aplicáveis a pesquisas em Ciências Humanas e Sociais cujos procedimentos metodológicos envolvam a utilização de dados diretamente obtidos com os participantes ou de informações identificáveis ou que possam acarretar riscos maiores do que os existentes na vida cotidiana, na forma definida nesta Resolução [Internet]. Diário Oficial da União, Brasília, DF. 2016 maio 24 [acesso 2026 abr 7]; Edição 98; Seção 1:44. Disponível em: http://bvsms.saude.gov.br/bvs/saudelegis/cns/2016/res0510_07_04_2016.html
Walsh TM, Volsko TA. Readability assessment of internet-based consumer health information. Respir Care. 2008;53(10):1310-5. doi: https://doi.org/10.4187/respcare.08531310 DOI: https://doi.org/10.4187/respcare.08531310
Erdat EC, Köksoy EB, Utkan G. Artificial intelligence chatbot as a companion for cancer patients about most common questions: analysis of readability and quality. Acta Oncol Turc. 2025;58(1):21-7. doi: https://doi.org/10.4274/ahot.galenos.2024.2024-8-2 DOI: https://doi.org/10.4274/ahot.galenos.2024.2024-8-2
Friedman DB, Hoffman-Goetz L, Arocha JF. Readability of cancer information on the internet. J Cancer Educ. 2004;19(2):117-22. doi: https://doi.org/10.1207/s15430154jce1902_13 DOI: https://doi.org/10.1207/s15430154jce1902_13
Çitir M. ChatGPT and oral cancer: a study on informational reliability. BMC Oral Health. 2025;25(1):86. doi: https://doi.org/10.1186/s12903-025-05479-4 DOI: https://doi.org/10.1186/s12903-025-05479-4
Grassini E, Buzzi M, Leporini B, et al. A systematic review of chatbots in inclusive healthcare: insights from the last 5 years. Univ Access Inf Soc. 2025;24(1):195-203. doi: https://doi.org/10.1007/s10209-024-01118-x DOI: https://doi.org/10.1007/s10209-024-01118-x
Diniz-Freitas M, López-Pintor RM, Santos-Silva AR, et al. Assessing the accuracy and readability of ChatGPT-4 and Gemini in answering oral cancer queries — an exploratory study. Explor Digit Health Technol. 2024:334-45. doi: https://doi.org/10.37349/edht.2024.00032 DOI: https://doi.org/10.37349/edht.2024.00032
Davis RJ, Ayo-Ajibola O, Lin ME, et al. Evaluation of oropharyngeal cancer information from revolutionary artificial intelligence chatbot. Laryngoscope. 2024;134(5):2252-7. doi: https://doi.org/10.1002/lary.31191 DOI: https://doi.org/10.1002/lary.31191
Boscolo-Rizzo P, Marcuzzo AV, Lazzarin C, et al. Quality of information provided by AI chatbots in head and neck cancer reconstruction. Clin Otolaryngol. 2025;50(2):330-5. doi: https://doi.org/10.1111/coa.14127 DOI: https://doi.org/10.1111/coa.14261
Pan A, Musheyev D, Bockelman D, et al. Assessment of artificial intelligence chatbot responses to top searched queries about cancer. JAMA Oncol. 2023;9(10):1437-40. doi: https://doi.org/10.1001/jamaoncol.2023.2947 DOI: https://doi.org/10.1001/jamaoncol.2023.2947
Presidência da República (BR). Lei nº 15.263, de 14 de novembro de 2025. Institui a Política Nacional de Linguagem Simples nos órgãos e entidades da administração pública direta e indireta de todos os Poderes da União, dos Estados, do Distrito Federal e dos Municípios. Diário Oficial da União, Brasília, DF. 2025 nov 17; Seção 1.
Presidência da República (BR). Lei nº 14.129, de 29 de março de 2021. Dispõe sobre princípios, regras e instrumentos para o Governo Digital e para o aumento da eficiência pública e altera a Lei nº 7.116, de 29 de agosto de 1983, a Lei nº 12.527, de 18 de novembro de 2011 (Lei de Acesso à Informação), a Lei nº 12.682, de 9 de julho de 2012, e a Lei nº 13.460, de 26 de junho de 2017. Diário Oficial da União, Brasília, DF. 2021 mar 30; Seção 1.
Musheyev D, Pan A, Gross P, et al. Readability and information quality in cancer information from free vs paid chatbot. JAMA Netw Open. 2024;7(7):e2422275. doi: https://doi.org/10.1001/jamanetworkopen.2024.22275 DOI: https://doi.org/10.1001/jamanetworkopen.2024.22275
Dantas TS, Silva PGB, Sousa EF, et al. Influence of educational level, stage, and histological type on survival of oral cancer in a brazilian population: a retrospective study of 10 years observation. Medicine (Baltimore). 2016;95(3):e2314. doi: https://doi.org/10.1097/MD.0000000000002314 DOI: https://doi.org/10.1097/MD.0000000000002314
Annunziata MA, Muzzatti B, Bidoli E. Psychological distress and needs of cancer patients: a prospective comparison between the diagnostic and the therapeutic phase. Support Care Cancer. 2010;19(2):291-5. doi: https://doi.org/10.1007/s00520-010-0818-9 DOI: https://doi.org/10.1007/s00520-010-0818-9
Sumanu SA. The crucial role of communication in managing the long-term health impact of cancer. Qual Quant Methods Libr [Internet]. 2024[acesso 2026 abr 25]13(4):513-536 Disponível em: https://www.qqml-journal.net/index.php/qqml/article/view/891/764
Abreu AA, Murimwa GZ, Farah E, et al. Enhancing readability of online patient-facing content: the role of AI chatbots in improving cancer information accessibility. J Natl Compr Canc Netw. 2024;22(2D):e237334. doi: https://www.doi.org/10.6004/jnccn.2023.7334 DOI: https://doi.org/10.6004/jnccn.2023.7334
Al-Hilli Z, Noss R, Dickard J, et al. A randomized trial comparing the effectiveness of pre-test genetic counseling using an artificial intelligence automated chatbot and traditional in-person genetic counseling in women newly diagnosed with breast cancer. Ann Surg Oncol. 2023;30(10):5990-6. doi: https://www.doi.org/10.1245/s10434-023-13888-4 DOI: https://doi.org/10.1245/s10434-023-13888-4
Altamimi I, Altamimi A, Alhumimidi AS, et al. Artificial intelligence chatbots in medicine: a supplement, not a substitute. Cureus. 2023;15(6):e40917. doi: https://www.doi.org/10.7759/cureus.40922 DOI: https://doi.org/10.7759/cureus.40922
Sohal KS, Okechi U. Evaluating the ability of artificial intelligence chatbots to respond to oral cancer questions: a descriptive study. Int Health Trends & Persp. 2025;5(1):GR13-17. doi: https://www.doi.org/10.32920/ihtp.v5i1.2308 DOI: https://doi.org/10.32920/ihtp.v5i1.2308
Cunha AR, Bigoni A, Antunes JLF, et al. The impact of inequalities and health expenditure on mortality due to oral and oropharyngeal cancer in Brazil. Sci Rep. 2021;11(1):12845. doi: https://www.doi.org/10.1038/s41598-021-92207-x DOI: https://doi.org/10.1038/s41598-021-92207-x
Klerings I, Weinhandl AS, Thaler KJ. Information overload in healthcare: too much of a good thing? Z Evid Fortbild Qual Gesundhwes. 2015;109(4-5):285-90. doi: https://www.doi.org/10.1016/j.zefq.2015.06.005 DOI: https://doi.org/10.1016/j.zefq.2015.06.005
Published
How to Cite
Issue
Section
License
Os direitos morais e intelectuais dos artigos pertencem aos respectivos autores, que concedem à RBC o direito de publicação.
This work is licensed under a Creative Commons Attribution 4.0 International License.
