health · digital · family: the model was never trained on this
perfect on competition day, blind in the clinic
AI diabetic retinopathy screening achieves competition-grade accuracy but fails in real clinical deployment
Problem statement
A deep learning algorithm for diabetic retinopathy (DR) screening that had been shown to have specialist-level accuracy (>90% sensitivity and specificity) ran into trouble once it was deployed in clinics, and public competitions (Kaggle 2015, APTOS 2019) rank such models only by how well they agree with a clinician's grade. Google's deployment in Thai clinics (Beede et al.) found that 393 of the 1,838 images put through the system in its first six months (21%) were rejected as ungradable; low-quality images came from rooms that could not be darkened, clinics that did not use dilation drops, and a camera that needed repair; internet connectivity was unreliable; and at one clinic half of patients opted out of the study because of the distance and inconvenience of the referral hospital. Beede et al. attribute the trouble to socio-environmental factors — above all the tension between the system's image-quality threshold and the images that an imperfect, resource-constrained clinic produces.
Why this matters
Diabetic retinopathy is estimated to affect over 93 million people (Kaggle 2015 competition page) and is the leading cause of blindness among working-age adults. Specialists are too few to screen everyone: Thailand has about 1,500 ophthalmologists for roughly 4.5 million patients with diabetes, a ratio of about 1:3,000, about double the ratio in the United States. AI screening promised to close this gap, but in the Thai deployment nurses somewhat questioned the system when it rejected images they felt they could read themselves. The pattern plausibly extends beyond DR to other imaging-based screening applications (cervical cancer, skin cancer, tuberculosis).
What’s been tried and why it hasn’t worked
Competition images are not pristine — both the Kaggle 2015 and APTOS 2019 pages warn that images "may contain artifacts, be out of focus, underexposed, or overexposed" — but the leaderboard scores only agreement with a clinician's grade (quadratic weighted kappa); it does not measure what happens when a deployed system refuses an image while a patient waits. In the Thai clinics Beede et al. studied, images were taken by nurses or camera technicians with the clinic's existing fundus camera, often in rooms that were not darkened, in Pathum Thani without dilation drops, and in one case with a camera that needed repair; age-related cataract can also leave the ocular media too cloudy to image. Image quality filtering creates a tradeoff: the deployed system was designed for patient safety to assess only the highest-quality images and rejected any with a bit of blur or a dark area, even where it could have made a strong prediction; the paper leaves open what threshold ensures accuracy and safety and what the risk is of deferring a prediction on imperfect data. The Beede et al. study found that ungradable images had the largest effect on model performance and the patient experience, that slow or lost internet connections delayed patients, that concern about far-away referrals led some nurses to discourage patients from joining the study, and that nurses worked around the study protocol rather than refer every patient whose image was ungradable.
What would unlock progress
Three complementary approaches: (1) camera-agnostic model architectures that explicitly handle image quality variation as an input feature rather than a rejection criterion; (2) offline-capable deployment systems that don't depend on cloud connectivity for inference; (3) co-design of the screening workflow with actual clinical staff in target settings before, not after, algorithm development. The deeper lesson is that AI medical device development must integrate human factors engineering from the start rather than optimizing accuracy on clean data and hoping deployment works.
Entry points for student teams
A team can reproduce the deployment gap at a desk instead of buying cameras: take a public DR-graded fundus set and apply a simulated acquisition-degradation pipeline — defocus and motion blur, uneven or reduced illumination, JPEG compression, and cropped field of view — then measure how a model trained on clean images loses sensitivity at each degradation level and how many images its quality filter rejects along the way. The DDR dataset (https://github.com/nkicsl/DDR-dataset, dataset licensed CC BY-NC-SA 4.0 with code under MIT, direct download) and the 45-image CC-BY High-Resolution Fundus set (https://www5.cs.fau.de/research/data/fundus-images/) need no registration; the EyeQ re-annotation supplies Good/Usable/Reject quality labels for 28,792 EyePACS images (https://github.com/HzFu/EyeQ) so the simulated degradation can be calibrated against real clinical quality grades, though those images must be pulled from the Kaggle EyePACS competition (https://www.kaggle.com/c/diabetic-retinopathy-detection, free account plus rules acceptance) and Messidor-2 requires a registration form with personal information (https://www.adcis.net/en/third-party/messidor2/). Alternatively, a team could design a screening workflow prototype that addresses the specific failure modes identified in the Beede study — capture-time guidance on lighting and dilation, offline operation, and a referral rule that does not send every ungradable image to a hospital an hour away — and test it in a mock clinic with role-played patient flow rather than in a live one. Skills: machine learning, human-computer interaction, clinical workflow design, mobile development.
Genes
Structural cousins: stuck the same way, in other fields
Sources
Kaggle Diabetic Retinopathy Detection competition (2015) and APTOS 2019 competition post-mortems; Beede et al., "A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy," CHI 2020, Google Health Thailand deployment reports go to source ↗
Working notes
Notes made while writing and checking this brief, shown as written. They use working shorthand.
Tier 3 pilot brief sourced from Kaggle competition post-mortem analyses. The Kaggle DR Detection (2015) and APTOS (2019) competitions are among the most-discussed examples of the competition-to-deployment gap in ML. The Beede et al. CHI 2020 paper documenting Google Health's Thai deployment failures is a landmark study in human-centered AI evaluation. Cross-references: health-aravind-telemedicine-retinal-screening-dropout (same clinical domain, different failure mode — patient follow-through rather than algorithm deployment), digital-algorithmic-fairness-measurement-gap (ML model fairness challenges).
Reconciliation 2026-08-21: The C37 entry-point triage flag was upheld — the first door asked students to "collect fundus images from multiple camera types (including smartphone attachments)," which means imaging human eyes with several $10–30k fundus cameras under IRB, not a semester's work. It is replaced by a simulated acquisition-degradation study on public DR-graded images, which asks the same research question (how does performance fall as capture quality falls) without a camera or a patient. Resources verified by fetch this session: DDR (https://github.com/nkicsl/DDR-dataset — MIT license, direct Google Drive download, no registration), the High-Resolution Fundus database (https://www5.cs.fau.de/research/data/fundus-images/ — 45 images across healthy/DR/glaucoma, CC BY 4.0, direct download, no registration), EyeQ (https://github.com/HzFu/EyeQ — 28,792 EyePACS images re-annotated Good/Usable/Reject, labels shipped as CSVs in the repo), the Kaggle EyePACS competition (https://www.kaggle.com/c/diabetic-retinopathy-detection — live, free account plus rules acceptance, labeled as such), and Messidor-2 (https://www.adcis.net/en/third-party/messidor2/ — free for research but its own page states "a form with personal information needs to be completed to download the databases," so it is labeled registration-gated rather than public). Whole-section check under the per-door rule: the second door (workflow prototype from the Beede failure modes) was unflagged and is genuinely facility-free, so the brief was never at zero doors; it was tightened only by saying the test happens in a mock clinic, since "prototype a clinical workflow" can quietly read as clinic access. Genome Tags untouched.
Reconciliation 2026-09-30: The whole write-up was read against its sources, and a second reader went over the repairs on the same day. Beede et al. (CHI 2020) was read in full text from the Internet Archive copy of the ACM full-text page (web.archive.org/web/2id_/dl.acm.org/doi/fullHtml/10.1145/3313831.3376718; the publisher page itself returns 403 to scripts); its citation record at https://api.crossref.org/works/10.1145/3313831.3376718 matches the Source line. The Kaggle 2015 and APTOS 2019 competition pages were read from Internet Archive copies (2018-12-20 and 2020-08-07), since the live Kaggle pages serve a reCAPTCHA wall to scripts. The DDR GitHub page with its README and commit history, the EyeQ GitHub page, the FAU High-Resolution Fundus page and the ADCIS Messidor-2 page were read live. "Google Health Thailand deployment reports" and "competition post-mortems" name no document and could not be opened. (1) The problem statement said models "achieve >95% sensitivity and >90% specificity on curated competition datasets." Neither competition page reports sensitivity or specificity; both score quadratic weighted kappa against a clinician's rating. Beede et al. say their algorithm "has been shown to have specialist-level accuracy (>90% sensitivity and specificity)." The text now gives that figure for that one algorithm and says only how the competitions score. "Real-world deployments consistently underperform" became a statement about the one deployment the paper documents. "Google Health" became "Google," the affiliation the paper gives. (2) "21% of images were rejected as ungradable (vs. <5% in competitions)": the paper gives 393 of 1,838 images (21%) in the first six months; no source gives a competition rejection rate, so the comparison is removed. (3) "Nurses struggled with the camera equipment" and "patients left before receiving results" are not in the paper. It attributes low-quality images to non-darkened rooms, clinics not using dilation drops and a camera that needed repair, and reports that 50% of patients at one clinic opted out of the study given the distance and inconvenience of the referral hospital; the text now says that. (4) "The gap is not algorithmic" was replaced by the paper's own framing: socio-environmental factors, and the tension between the image-quality threshold and the images a resource-constrained clinic produces. (5) "~100 million people globally" is now the figure a named source gives: the Kaggle 2015 page says the disease "is estimated to affect over 93 million people." "Preventable blindness" and "ophthalmologist-to-patient ratios can exceed 1:500,000" are in no source. The APTOS page calls the disease "the leading cause of blindness among working aged adults"; Beede et al. give Thailand's 1,500 ophthalmologists for about 4.5 million patients with diabetes, "a ratio of about 1:3000, about double of what it is in the United States." The text now uses those figures. (6) "Slowed adoption by years and eroded clinical trust in AI diagnostics more broadly" is in no source; the paper says nurses "somewhat questioned the power of the deep learning system" when it rejected images they felt were human-readable. The text now says that, and the extension to other screening applications, which no source makes, is marked as plausible rather than stated. (7) The claim that competition models are trained on "high-quality fundus photographs taken by skilled technicians with standardized cameras in controlled lighting" is contradicted by both competition pages, which warn that images "may contain artifacts, be out of focus, underexposed, or overexposed" (APTOS adds that they came from "multiple clinics using a variety of cameras"). The paragraph now quotes the competition pages and describes the Thai clinic conditions as the paper does; smartphone attachments and handheld devices, "minimally trained staff," "patient movement," "domain adaptation" and the transfer-learning sentence, none of which appear in a source, are removed. The filter tradeoff is now stated as the paper states it: a system built to assess only the highest-quality images, and an open question about the risk of deferring a prediction on imperfect data. (8) "Nurse unfamiliarity" and "even when the algorithm performed well" are not in the paper; the sentence now lists what it reports: ungradability as the largest factor, slow and lost internet connections, nurses discouraging participation over far-away referrals, and nurses working around the study protocol. (9) DDR was described as "MIT-licensed." Its README, updated 2026-09-28 (the 2026-08-21 note above was accurate when written), now licenses the dataset CC BY-NC-SA 4.0 and only the repository code under MIT; the entry point now says so. (10) Messidor-2 "requires a signed registration form": the ADCIS page says "A form with personal information needs to be completed," not signed; wording changed. (11) The workflow-prototype door listed "image-quality feedback at the moment of capture" and "nurse-friendly interface" as Beede failure modes; the paper says immediate gradability feedback already existed and was itself a source of frustration, and reports no interface complaint. The door now names failure modes the paper reports: lighting and dilation at capture, connectivity, and referral of every ungradable image to a distant hospital. Confirmed in the source and left as written: 21% ungradable; unreliable internet (60-90 second uploads, a two-hour outage that halved a day's screening); the Source line citation; HRF's 45 images under a Creative Commons 4.0 Attribution licence with direct download; EyeQ's 28,792 EyePACS images graded Good/Usable/Reject with labels as CSVs in the repository; Messidor-2 free for research. Not checked: the Kaggle competition page as it stands today (reCAPTCHA wall; the archived copy was used, so today's account and rules-acceptance terms were not seen); the unnamed "deployment reports" and "post-mortems." Left as it was and not checked against any source: the "What Would Unlock Progress" section, which is the write-up's own proposal, and the Source Notes sentence that calls the deployment "Google Health's." Titles unchanged. Genome Tags untouched.