Share
Title
Measuring the performance of artificial intelligence to interpret images of HIV self-testing results
Presenter
Obinna Ekwunife
Authors
O. Ekwunife1, S. Roche1, R. Mendonca2, B. Kwach3, P. Ongwen4, S. Zhang5, D. Hattery2, S. Smedinghoff2, S. Morris2, V. Omollo3, D. Were6, D. Rech2, E. Bukusi7,8,9, K. Ortblad1
Institutions
1Fred Hutchinson Cancer Center, Cancer Prevention Program, Seattle, United States, 2Audere, Seattle, United States, 3Kenya Medical Research Institute, Center for Microbiology Research, Kisumu, Kenya, 4Kenya Medical Research Institute, Centre for Microbiology Research, Kisumu, Kenya, 5University of Washington, Department of Epidemiology, Seattle, United States, 6Jhpiego, Nairobi, Kenya, 7University of Washington, Department of Global Health, Seattle, United States, 8University of Washington, Department of Obstetrics and Gynecology, Seattle, United States, 9Kenya Medical Research Institute, Center for Microbiology Research, Nairobi, Kenya

BACKGROUND: HIV self-testing (HIVST) is a promising intervention for supporting community-based HIV service delivery; however, it has yet to be fully embraced by policymakers, in part, due to concerns about result misinterpretation and subsequent incorrect treatment decisions. Identifying tools that can support correct HIVST interpretation will likely be an important prerequisite to any large-scale incorporation of HIVST into national HIV service delivery programs. We sought to understand how well a cost-effective artificial intelligence (AI) algorithm could correctly interpret a common brand of blood-based HIVST kits.
METHODS: At 20 private pharmacies in Kisumu, Kenya, we offered free blood-based HIVST to clients =18 years purchasing products indicative of sexual activity (e.g., condoms). Trained pharmacy providers assisted with testing, as needed. In real-time, each test was independently interpreted by (1) the client, (2) the pharmacy provider, and (3) a certified HIV testing service (HTS) counselor who then photographed the result. Each test image was subsequently interpreted by (4) an AI algorithm and (5) a panel of three expert HIV rapid diagnostic test readers. Using expert panel determination as the gold standard, we calculated the sensitivity and specificity of each group’s interpretation.
RESULTS: From March-June 2022, we screened 1691 pharmacy clients, enrolled 1500, and collected 855 test images. Among clients with test images, 63% (540/855) were female, the median age was 26 years (IQR 22–31), and 39% (335/855) reported casual sex partners. The AI algorithm correctly interpreted all positive tests as positive (100% sensitivity) and slightly outperformed HTS counselors and pharmacy providers (each 98% sensitivity; 95% CI 97%-99%) as well as clients (93% sensitivity; 95% CI 91%-94%). The AI algorithm correctly interpreted nearly all negative tests as negative (99% specificity; 95% CI 98%-99%), similar to the aforementioned comparison groups, which all had 100% specificity.
CONCLUSIONS: AI algorithms are capable of correctly interpreting HIVSTs and may perform just as well, if not better, than HTS counselors, pharmacy providers, and clients. As differentiated HIV service delivery continues to gain momentum, AI algorithms could provide additional quality control, validation, and disease surveillance to policymakers and HIVST end-users, including new providers to whom HIV service delivery is being task-shifted.

Download the e-Poster (PDF)