Health Apps Encode Bias via Pre-Code Assumptions and Narrow Datasets

The MedicalXpress synthesis of evidence shows developers and funders set priorities before any code exists, embedding assumptions about normal bodies, valued knowledge, and profitable problems. Datasets drawn mainly from light-skinned and high-income populations produce skin-diagnosis tools that misclassify lesions on darker skin and risk-stratification models that tie need to spending rather than clinical severity. These choices create feedback loops where under-served groups generate less data, further skewing subsequent models.

Obermeyer et al. (Science 2019) documented the hospital algorithm that required Black patients to be substantially sicker than white patients to reach the same risk threshold, illustrating how expenditure proxies reproduce systemic disparities. The new review extends this pattern to period trackers, mental-health quizzes, and wearables whose training cohorts exclude Indigenous and low-resource populations by design rather than accident.

Community consultation occurs after core decisions on data ownership, outcome metrics, and whether digital tools are even appropriate, violating principles of data sovereignty. Power asymmetries between technology firms and marginalized groups remain unaddressed in current development pipelines.

Next steps require prospective equity audits before funding and regulatory review of training-data demographics with transparent thresholds for inclusion.