Background
AI research is both theoretical and computational. It is important to have a deep understanding of different AI models’ error bounds and associated factors. One of our recent studies is to analyze the impact of data distribution on fairness guarantees in equitable deep learning, which is elaborated as follows.
Machine learning models have achieved remarkable performance across various applications, but concerns about fairness and bias in these systems have become increasingly prominent, especially in high-stakes domains like healthcare. Recent studies have shown that deep learning models can exhibit significant performance disparities across different demographic groups – for example, being more accurate for one racial group compared to another, or working differently for men versus women. These biased predictions in healthcare applications can have severe consequences, affecting patient outcomes and perpetuating systemic inequalities.
The challenge of achieving fairness stems from multiple factors, particularly how data is distributed across different demographic groups. Some groups might have more examples to learn from than others, or their examples might have different characteristics. Therefore, it is critically important to have a deep theoretical understanding of how data distribution affects an AI model’s performance fairness.
What We Do
While previous fairness studies have focused primarily on empirical evaluations, we aim to understand the mathematical foundations behind these disparities. Several datasets have been developed to study fairness, particularly in medical imaging where accuracy is crucial for patient care. We analyze datasets like FairVision (ophthalmology), CheXpert (chest X-rays), and HAM10000 (dermatology) to evaluate fairness across different demographic attributes such as age, gender, and race. However, our main contribution is developing a comprehensive theoretical framework that explains how differences in class prevalence and feature distributions across demographic groups affect fairness guarantees in machine learning systems.
Through this work, we seek to move beyond just observing performance differences between groups and instead provide a mathematical understanding of why these differences occur and how to systematically address them. This theoretical foundation is essential for developing more reliable and equitable machine learning systems, particularly for critical applications in healthcare where fairness is paramount. Check out our open-source code repositories on our Harvard AI Robotics Lab GitHub account.
Selected Publications
- Luo, Y., Wen, C., Huang, H., Li, M., Shi, M., Fang, Y. and Wang, M., Impact of data distribution on Fairness Guarantees in Equitable Deep Learning. arXiv preprint arXiv:2412.20377.