Behind the Shimmery Veil of Artificial Intelligence — Exploring the Ways Algorithms Contribute to Discrimination

“One cannot determine what an algorithm will do by reading the underlying code” [1]

In a continuously technologically-oriented world, algorithms are at the center of everything. They are an integral component of computational processes that contribute to automation and, put simply, hacking the world toward efficiency and simplicity [2]. Despite being almost everywhere — from healthcare and the legal system, to design and education — there is an assumption of algorithmic neutrality. This is to say that, it is presumed that technologies like artificial intelligence (AI) algorithms, “do not have, have embedded in them, or contain values” — making them (morally) neutral [3]. However, that is not necessarily an accurate reflection of AI and related technologies.

Algorithmic bias

Although the algorithm itself might be, in a sense, neutral per the intentions of its design, this is not a reflection of the realities of AI. Much of the training data used to teach algorithms to make proper decisions (e.g. in hiring and in the justice system) can also perpetuate biases, whether explicitly or implicitly [1]. These are biases that are integrated into how and why this historical data was collected, analyzed, and interpreted before being used to teach algorithms. For example, if training datasets used in hiring settings historically excluded women, then the algorithm will continue the pattern of discriminating against women by excluding them from future output regarding hiring decisions. In this way, AI exhibits a bias that exacerbates existing disparities, which damages the perspective that AI is impenetrably objective [1;4]. It also diminishes its perceived efficiency — by perpetuating harm, it is also making the wrong (non-moral) decisions (e.g. minimizing the hiring pool and excluding potentially qualified candidates). This algorithmic bias occurs when algorithms deliver “systematically biased results” that are the result of “erroneous assumptions” in the training process and tend to reflect human biases [4;5].

More Data? An Issue and A Solution

These biases do not end with the algorithm itself. Once output is generated by the AI, it is interpreted by humans who might use it to guide their own decision making (e.g. for predicting the likelihood of re-offence in criminal justice contexts) [6]. These interpretations further contribute to a flawed cycle wherein data is precariously collected and inputted into a system with an incomplete picture of the world, thus generating results that continue to misrepresent people and the world. All of this leads to conditions that perpetuate the systemic inequality which AI objectivity claims to solve. In response to these concerns, some have suggested offering the algorithm more, “better data” to overcome the issue of algorithmic bias [1;9]. Others have argued that this reduces a fundamental issue of algorithmic AI to a mere “glitch” that can be repaired with a quick-fix addition of more data. In reality, the new data will just perpetuate the existing flaws in the algorithm, further extending the issues without giving a real solution.

“Algorithmic biases rely on proxy attributes” [4]

Proxy Discrimination

Not all discrimination or bias is directly attributable to its justification. This is to say that, sometimes, there can be non-causal or extra factors that are used as a substitution or proxy for the causal factor because of their explainability or measuring convenience. Such factors or characteristics can be used because of their “facially neutral” nature — meaning they are not explicitly related, only correlated to the factor that is being measured, and are likely not protected classes (e.g. race, gender, etc.) [7]. Discrimination of this sort is called proxy discrimination, and can be either intentional or unintentional, but is nonetheless harmful in its nature [10]. An example of this might be an algorithm relying on an individual’s zip code to act as a proxy for their socioeconomic class, and might lead to discrimination based on their race or ethnicity, such as in recidivism prediction algorithms like COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) [8]. This raises a slew of issues with AI and continues to call into question the legitimacy of algorithmic fairness and objectivity, as well as the transparency and accountability of AI output and systems [1].

Disparate Impact and Treatment

Proxy discrimination and algorithmic bias are not the only ways that AI can perpetuate harm. Although mainly present in the American context, there is room for both disparate impact and disparate treatment in other contexts. The former, disparate impact refers to a form of unintentional discrimination that results from “some requirement or practice” that seems superficially neutral, but disproportionately targets or harms a specific protected group [1]. Disparate impact occurs in algorithmic AI when these systems, while designed without explicit biases encoded into them, lead to discriminatory outcomes for a certain group based on the training data or the features of the model itself. An example might be when a hiring algorithm favours candidates from a certain educational background, where that background is also associated with a specific demographic (e.g. racial, socioeconomic, etc.). That would lead to discrimination towards those who do not belong to this group by unintentionally favouring the group.

On the other hand, the latter, disparate treatment is an intentional differential treatment of others based on their group membership (e.g. race, gender, sex, etc.) [1]. With this kind of discrimination, it does not matter whether it is based on statistical probabilities or “taste” — all that matters is that some prejudicial bias took place. It happens when AI explicitly uses certain traits — such as protected characteristics — to make decisions and predictions. While it is intentional in humans, it can be considered unintentional in AI and simply based on the datasets it was trained on and has access to. A common example of disparate treatment is paying men more than women for performing the same job.

Potential Solutions

This brief overview of the spectrum of bias and harm that can result from algorithmic AI is just one piece of a large, ongoing conversation about a complex yet critical topic. Within this discussion, several solutions have been proposed — here are a few.

• Ensuring that algorithms are trained with representative datasets that reflect the diversity of populations and demographics to avoid reinforcing biases.
• Providing comprehensive ethics and awareness training for engineers and developers (and everyone else who has a hand in training and developing algorithms) to ensure that their own biases are not translated into the systems.
• Develop clear definitions for transparency and explainability and AI, and enhance these principles in the technologies to encourage clear justificatory practices for decisions.
• Gather feedback from users, and encourage the participation of diverse stakeholders (i.e. consumers and users of AI) in discussions around its development, testing, and improvement of algorithms.

While not a complete list, these solutions might be a step in the right direction toward amending biases in algorithms.

Author: Kawthar Fedjki

Sources

[1] J. Kleinberg, J. Ludwig, S. Mullainathany, and C. R. Sunsteinz, “Discrimination in the Age of Algorithms,” Journal of Legal Analysis, 10, pp. 113–174. https://doi.org/10.1093/jla/laz001
[2] I. H. Sarker, “AI-Based Modeling: Techniques, Applications and Research Issues Towards Automation, Intelligent and Smart Systems,” SN Computer Science, 3(2), 158, 2022. doi: 10.1007/s42979–022–01043-x
[3] J. C. Pitt, “Guns Don’t Kill, People Kill; Values in and/or Around Technologies,” The Moral Status of Technical Artefacts, P. Kroes and P. Verbeek (Eds.) Spring Dordrecht, 2014, pp. 89–101.
[4] G. M. Johnson, “Algorithmic bias: on the implicit biases of social technology,” Synthese, 198, pp. 9941–9961, 2021. https://doi.org/10.1007/s11229-020-02696-y
[5] Z. Larkin, “AI Bias — What Is It and How to Avoid It?” Levity, Nov. 16. https://levity.ai/blog/ai-bias-how-to-avoid
[6] D. Pessach and E. Schmueli, “Algorithmic Fairness,” ArXiv, abs/2001.09784, 2001.
[7] A. E. R. Prince and D. Schwarcz. “Proxy Discrimination in the Age of Artificial Intelligence and Big Data,” Iowa Law Review, 105(3), pp. 1257–1318, 2020.
[8] C. Rudin, C. Wang, and B. Coker. “The Age of Secrecy and Unfairness in Recidivism Prediction,” Harvard Data Science Review, 2(1), pp. 1–46, 2020. https://doi.org/10.1162/99608f92.6ed64b30
[9] J. Toh, “More data will not solve bias in algorithmic systems: it’s a systemic issue, not a ‘glitch’,” Racism and Technology Center, Apr. 14. https://racismandtechnology.center/2023/04/14/more-data-will-not-solve-bias-in-algorithmic-systems-its-a-systemic-issue-not-a-glitch/
[10] M. C. Tschantz, “What is Proxy Discrimination?” 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022. https://doi.org/10.1145/3531146.3533242