When Algorithms Drift: The Hidden Biases in Medical Imaging AI
When COVID-19 first overwhelmed hospitals in early 2020, the patients filling intensive care units shared a grim commonality: no prior immunity, acute symptoms, and chest X-rays that told a consistent story of aggressive disease. Train an algorithm on those images, and it learned to recognize a specific kind of infection at a specific moment in time.
But the virus kept evolving. Vaccines arrived. Immunity accumulated. By 2023, a COVID patient's chest X-ray might look fundamentally different—and an algorithm trained on 2020 data could quietly start making mistakes nobody had caught.
The Drift Problem
This phenomenon is what Maryellen Giger, a professor of radiology at the University of Chicago, calls temporal bias. It represents one of roughly 30 distinct ways that bias can infiltrate a medical imaging algorithm.
Most of these biases are far harder to spot than the problem of training on homogeneous patient populations. An algorithm developed with images from unvaccinated patients in 2020 might misinterpret the chest X-rays of vaccinated patients in 2023—not because the algorithm broke, but because the underlying disease presentation had shifted.
A Repository Born from Necessity
Giger leads the Medical Imaging and Data Resource Center (MIDRC), a data repository funded by the National Institute of Biomedical Imaging and Bioengineering (NIBIB). Born from the pandemic but designed to outlast it, MIDRC addresses something the field desperately lacked: a curated, diverse, and rigorously organized collection of medical images.
The Crisis Response
The need became obvious almost immediately after COVID struck. Researchers desperate to help began pulling imaging data from wherever they could find it. Datasets accumulated duplicates, low-quality scans, and skewed patient demographics.
The classic computing adage—garbage in, garbage out—applied with particular force to medical AI. An algorithm that claims to predict COVID severity from chest X-rays cannot be trusted if it was built on flawed inputs, tested on the same patients used for training, or deployed in populations that look nothing like the training set.
By the Numbers
Since August 2020, MIDRC has built an impressive foundation for medical imaging research:
309,000+
imaging studies collected—primarily chest X-rays and CT scans
55,000
patients represented across the United States
27
algorithms produced to detect, diagnose, monitor, or predict COVID-19 outcomes
The 80/20 Innovation
The key innovation is a split that guards against one of the most persistent problems in machine learning. MIDRC keeps 80 percent of its data freely available to researchers worldwide for algorithm development. The remaining 20 percent stays locked away in a sequestered dataset that will never be used for training.
That reserve exists for one purpose: independent testing. A company or research team can bring an algorithm to MIDRC, and the center can evaluate its performance against data it has never seen before. The sequestered dataset is now large enough to begin those assessments, and MIDRC has started working with companies to help expedite FDA regulatory approval.
Why This Architecture Matters
When researchers develop and test an algorithm using the same patient population, they produce results that look impressive but often fail to generalize. A model might perform beautifully on the data it was trained to recognize, only to falter when confronted with patients from a different demographic, images captured on different machines, or disease presentations that have shifted over time.
Mapping the Bias Landscape
In a recent paper published in the Journal of Medical Imaging, Giger and her colleagues outlined potential sources of bias across five stages of algorithm development:
Stage 1: Data Collection
Bias enters through skewed patient populations and incomplete sampling.
Stage 2: Preparation and Annotation
Human labelers bring their own inconsistencies and blind spots.
Stage 3: Model Development
Architectural choices encode assumptions about what matters.
Stage 4: Evaluation
Testing on familiar data creates an illusion of performance.
Stage 5: Deployment
The real world introduces variables no training set could anticipate.
Some failures are obvious in retrospect—an algorithm trained predominantly on white patients may underperform in a hospital serving a predominantly Hispanic population. Others require deeper scrutiny, like temporal bias that creeps in when the world changes faster than the training data.
To help researchers navigate these traps, MIDRC has published an online bias awareness tool that catalogs potential sources of bias and offers guidance for mitigating them. The goal is to make fairness a deliberate design choice rather than an afterthought.
Beyond the Pandemic
The pandemic was MIDRC's catalyst, but not its ceiling. The repository is now expanding in several directions:
- Collecting brain and heart images through MRI and ultrasound
- Gathering data from patients with Long COVID
- Extending interoperability with other data repositories
- Positioning itself to support medical imaging AI across a widening range of clinical problems
The infrastructure amounts to something more durable than a COVID response. Before MIDRC, researchers lacked a large-scale, curated, and diverse medical imaging dataset with paired patient outcomes—a foundation required to build trustworthy algorithms.
NIBIB had been searching for a use case to prove the concept even before the pandemic, contemplating projects like improving breast cancer diagnosis from mammograms. COVID accelerated the timeline and provided an urgent test bed, but the underlying need always extended beyond a single disease.
The Quiet Insurance Policy
For now, MIDRC's data commons remains freely accessible to registered researchers, its bias awareness tool is publicly available online, and its sequestered dataset continues to grow. It stands as a safeguard for the moment when an algorithm trained in one context gets deployed in another.
The infrastructure was built for a pandemic. It may prove more valuable for everything that comes after.
Based on: How to Build an Unbiased Medical Imaging Dataset; Maryellen Giger, University of Chicago; National Institutes of Health/NIBIB, 2024.