A/HRC/57/70 fairness and accuracy of complex algorithms. This regulatory gap allows biased AI systems to proliferate unchecked, further entrenching existing social and economic disparities. 14 C. Housing, health and nutrition 32. Although racial bias and discrimination in housing has been well documented for decades, it is not yet clear how digitalization and AI are impacting access to housing and related basic services such as water, sanitation and electricity. For example, in the United States, racial bias in housing has been manifested in the lower valuation of housing in predominantly Black neighbourhoods by as much as 21% to 23% in non-Black neighbourhoods and homes appraised below the contract price 1.9 times more than in majority-White neighbourhoods.15 This pattern of residential segregation can be observed in other countries, such as Brazil. 33. AI is revolutionizing healthcare, offering the potential to enhance diagnostic accuracy, streamline patient care, and improve health outcomes. Initially conceived to revolutionize clinical decision-making and patient care, AI relies heavily on vast datasets comprising diverse sources: patient histories, genetic profiles, lifestyle data, and more. If training data predominantly represents majority groups, the resulting models are inherently biased, producing recommendations and predictions that favour those populations. Also, the extraction and utilization of this data often occur without robust oversight or clear consent frameworks, raising significant concerns about fairness and equity. Such practices can inadvertently embed biases into AI models, perpetuating disparities in healthcare outcomes, particularly affecting people of African descent. However, the ethical principles guiding this extraction and implementation are often left to the discretion of the developers due to a significant lack of regulation. This regulatory gap means that embedding ethical considerations from the outset is crucial to ensure that AI technologies benefit patients rather than cause harm. This bias is compounded by decisions in feature engineering and hyperparameter tuning, which may overlook factors critical to understanding and addressing health disparities among people of African descent. 34. The opacity of these algorithms — often referred to as the "Black Box Problem" — obscures how decisions are made, making it challenging to identify and rectify biases that disadvantage African descent patients. Research indicates that health data is predominantly skewed towards white and male populations, reflecting historical social practices and individual programmer biases that shape AI systems. An important study scrutinized a widely adopted AI algorithm in US healthcare, revealing a troubling bias favouring white patients over equally ill Black patients. The algorithm’s reliance on historical cost data disadvantaged African-descent patients due to lower previous healthcare expenditures influenced by socioeconomic factors. Consequently, Black patients received fewer critical medical interventions, exacerbating health disparities and revealing systemic anti-African biases within AI systems. The algorithm relied on health care spending to predict future health needs. But with less access to care historically, patients of African descent often spent less. As a result, they had to be much sicker to be recommended for extra care under the algorithm. 16 35. The COVID-19 pandemic was a reminder of both the promise of AI and also the urgent need of striking a balance between protecting the collective interest and individual rights. The crisis brought to light issues regarding data access, sharing, liability, data and 14 15 16 Submission by Motse Ntloedibe-Kuswani, the American University of Paris. Jonathan Rothwell, Andre M. Perry and Mike Andrews, “The Black innovators who elevated the United States: Reassessing the Golden Age of Invention”, The Brookings Institution, 23 November 2020, available at https://www.brookings.edu/articles/the-black-innovators-who-elevated-the-unitedstates-reassessing-the-golden-age-of-invention/. Science, “Dissecting racial bias in an algorithm used to manage the health of populations”, 25 October 2019, available at https://www.science.org/doi/10.1126/science.aax2342. 9