Deep learning models of artificial intelligence can be trained to predict self-reported race from imaging results, which raises concerns about worsening health inequalities, Published in the study The Lancet Digital Health.
Researchers have discovered that models can detect races from a variety of book imaging results, including X-rays, CT scans and mammograms. Capacity could not be found from the distribution of the disease, where a condition is more prevalent in certain groups, or physiological features.
The study further found that the deep learning model could predict race even after using low-quality images, whereas a model trained in high-pass filtered images could perform when human radiologists could not determine whether the image was an X-ray. Not at all
“In conclusion, our research shows that medical AI systems can easily learn to recognize self-reported ethnic identities from medical images, and that this ability is extremely difficult to isolate. And can be generalized to multiple imaging methods, “the study authors wrote.
“We strongly recommend that all developers, regulators and users involved in medical image analysis consider the use of deep learning models very carefully as such information may be misused to perpetuate or even worsen the well-documented racial discrimination that exists in medical practice.”
Why it matters
The researchers wrote that perseverance in the power of the model shows that behavior can be difficult to control when needed and that the problem needs to be further studied. Since human radiologists cannot usually determine the race from imaging results, they will not be able to provide supervision for the models and will not be able to alleviate any potential problems.
“The results of our research emphasize that the ability of AI deep learning models to predict self-reporting races is not important in itself. Clinical experts do not, creating a huge risk for the deployment of all models of medical imaging, “the researchers wrote.
As AI has expanded into more areas of healthcare and life sciences, so have experts Ethnic health inequalities have raised concerns about the possibility of permanent and worsening.
According to a The study was published last week Journal of the American Medical Informatics AssociationFinding bias in AI and machine learning requires a holistic approach that requires multiple perspectives to address, as models that perform well for one group may fail for another.