Large language model-based triage to identify antiretroviral therapy adherence barriers and risk levels in patient messages

Yuanchao Ma, Sofiane Achiche, David Lessard, Kim Engler, Serge Vicente, Gavin Tu, Benoît Lemire, Lina Del Balso, Nathalie Paisible, Bertrand Lebouché

Article de revue (2026)

Accéder à ce document
Disponible
Libre accès au texte intégral dans PolyPublie et chez l'éditeur officiel
Version officielle de l'éditeur
Texte Texte • 2MB •
Matériel supplémentaire
Vidéo Vidéo • 25MB •
Matériel supplémentaire
Texte Texte • 274kB •
Matériel supplémentaire
Texte Texte • 853kB •

Résumé

Objective: This study aimed to develop large language models (LLMs) to automatically identify antiretroviral therapy (ART) adherence barriers and stratify nonadherence risk levels from patient-generated messages. Materials and Methods: With a co-construction committee of people with HIV and providers, 15 480 sentences were annotated for barrier and risk levels. General-domain LLMs (eg, Flan-T5) and clinical foundation models (eg, Clinical-T5) were fine-tuned for multiclass classification and evaluated using Macro-F1. Best-performing models were compared with GPT family and other open-source LLMs. Model fairness, error patterns, and environmental footprints were also assessed. Results: Flan-T5-xl achieved the best barrier detection (Macro-F1 = 0.83 test/0.71 external), and Flan-T5-large excelled in risk stratification (0.79/0.57). Fine-tuned general-domain LLMs significantly outperformed clinical foundation models (P < .001) on the test split. Error analysis identified both annotation challenges (ambiguous inputs, cross-category overlap, and inconsistencies) and inference limitations (semantic overlap, implicit meanings, and contextual misinterpretation). On external datasets, Flan-T5 models outperformed other LLMs (P < .001), and were more robust to demographic descriptors, while GPT-4.1 and Gemma3 often misclassified inputs as “None.” Additionally, GPT-4.1’s inference consumed ∼30× more energy than that of Flan-T5-xl. Discussion: Fine-tuned Flan-T5 models demonstrated strong classification performance, greater robustness to demographic attributes, and lower energy consumption, though challenges remained for subjective and underrepresented categories, reflecting both data imbalance and model limitations in implicit reasoning. Conclusion: LLM-based approaches show promise for real-time ART adherence monitoring, offering a scalable solution to individualized HIV care. Beyond performance, our findings highlight the importance of fairness and environmental sustainability in clinical AI development, with next steps focused on real-world validation through deployment in patient-facing digital tools.

Matériel d'accompagnement:
Organismes subventionnaires:
FRQS, OBVIA, NSERC, FRQNT
Numéro de subvention:
268938, 311200
Adresse URL de PolyPublie:
Titre de la revue :
JAMIA Open ( vol. 9 , no 5 )
Maison d'édition:
University of Oxford
OAI:
oai:publications.polymtl.ca:82827
ORCID
Date du dépôt:
28 sept. 2026 09:48
Dernière modification:
03 nov. 2026 06:03
Citer en APA 7:
Ma, Y., Achiche, S., Lessard, D., Engler, K., Vicente, S., Tu, G., Lemire, B., Del Balso, L., Paisible, N., & Lebouché, B. (2026). Large language model-based triage to identify antiretroviral therapy adherence barriers and risk levels in patient messages. JAMIA Open, 9(5), ooag169 (20 pages). https://doi.org/10.1093/jamiaopen/ooag169

Statistiques

Total des téléchargements à partir de PolyPublie

Téléchargements par année

Provenance des téléchargements

Dimensions

Actions réservées au personnel

Afficher document
Afficher document