Artificial intelligence has fast become part of our everyday lives, at home and at work. As an organisational tool, AI has the potential to improve operations and efficiency across every sector, as well as deliver a better experience for customers or service users.
Of course, several concerns have been raised about the validity and ethics of AI technology. One that’s proved particularly vexing for business leaders and data scientists is bias, and the extent to which human biases are reflected in, and perpetuated by, AI systems.
Following on from our webinar, Understanding AI bias: Your essential guide to a fairer AI landscape, we’ll explain the various ways bias can enter AI models and the risks of operating with biased data or algorithms. Finally, we’ll touch on how organisations can reap the cost-saving and productivity benefits of AI while minimising the harm arising from unwanted bias.
First, it’s helpful to look at what artificial intelligence is. When we talk about AI in this context, what we’re really talking about is machine learning – a process whereby data goes in, rules are applied, and we’re given an outcome. The model development itself is a process whereby the rules are created or ‘trained’ using machine learning power and even with relatively small models, this still represents billions of rules. These rules are described through machine learning – hence AI, but ultimately it is still a process, based on rules and data – critically with data being at the heart of this process.
Bias occurs when the outcomes produced unfairly penalise an individual or group based on certain characteristics. In one high-profile example, Amazon was forced to scrap an AI recruitment tool that was found to exhibit a preference for male candidates.
In the early days of AI, people were excited at the possibility of complete objectivity – a ‘way of overcoming human subjectivity, bias, and prejudice’. Instead, in the words of Michael Sandel, political philosopher at Harvard, we’ve found that many algorithms merely “replicate and embed the biases that already exist in our society.”
Bias can enter AI systems in three ways:
AI models are built on a data sample. Bias creeps in when the data sample used isn’t representative of the whole of the target population. ASR (automatic speech recognition) is a great example. Models are trained using audio scraped from the internet, with speech that’s clearer and more articulate than that we encounter in real life, as well as generally free from strong accents. Subsequently, models will struggle to process audio from the whole population, resulting in bias against those with regional accents or different dialects.
This occurs when the data gathered is misrepresented at source. An example could be an advisor or customer support agent using spoken accounts to record a case. If the person they’re talking to doesn’t speak the same language, the challenge that person is experiencing and the details of their case could well be captured, and therefore assessed, inaccurately. The advisor will inevitably, if unintentionally, be influenced by their own experiences and preferences which further exacerbates this bias.
Finally, bias may be present in the algorithm itself, either as a result of the unconscious bias of the developer programming it or by nature of its inherent simplification of a problem or set of problems, which will be repeated or enhanced as it’s use extends. Different tools use different algorithms, and some will be better suited to certain datasets, or groups of people, than others.
The first and most obvious risk of unintentional bias is lack of fairness. If bias goes unaddressed, it will lead to discrimination against particular groups, who won’t be able to access the same benefits, opportunities or help as others. Real-world impacts of AI bias can be seen in the spheres of health and social care, policing, credit, and recruitment, to name but a few.
In our ASR example, these models are increasingly being used to help organisations interact with their customers, and will inform decisions that can have far-reaching impacts on people’s lives. That’s why, as part of our Innovate UK-funded research to reduce bias in ASR models, we’re working to identify and address the biases that occur due to misinterpretation of speech.
A machine-learning model built using biased data will continue to perpetuate historical biases and social inequalities. More worryingly, without intervention, it will become more and more efficient in doing so: ‘when discriminatory data and algorithms are baked into AI models, the models deploy biases at scale and amplify the resulting negative effects’ (IBM).
Further harm can be caused when AI biases are hidden behind misleading metrics. To take an example from our webinar, say you have a machine-learning model that works perfectly for men, but not for women. If that’s used in a male-dominated industry, where 99% of people are men, you could report that the system is 99% accurate. While that sounds great, it’s completely failing half of the population. For this reason, when looking at how well a model performs, it’s important to drill down to sub-group level.
You might hear about two different types of bias: intrinsic and extrinsic. The difference is all about the outcome that the bias produces. An intrinsic bias occurs when there’s an inaccuracy fed into the system, but it doesn’t adversely affect the outcome. In contrast, an extrinsic bias is one that changes the outcome for the user. In the case of group C below, it could mean not getting the help that’s needed.
Group | Transcription | Classification | Bias |
A | I would like to talk about universal credit | Benefits | Nil |
B | I would like to walk about universal credit | Benefits | Intrinsic |
C | I would like to talk about universal debit | Debt | Intrinsic & extrinsic |
While it’s nearly impossible to eliminate all bias in AI, there are ways to reduce it:
Understand where bias exists and listen to end users’ feedback to understand their overall experience and learn how the model can be improved to better meet their needs.
Look at accuracy and performance metrics, not just at a high level, but also for the groups you’ve identified as underrepresented. Search out intrinsic or extrinsic bias and identify the source.
In the case of algorithmic bias, corrective action may mean switching to a different tool. Different algorithms may be better suited to certain groups simply because of the way they interact with data.
Fixing current models often means sourcing more data from underrepresented groups and reprogramming models to make sure that they’re representative. When building your next model, factor in your users’ varying backgrounds and perspectives. It’s also useful to have better representation and multiple perspectives within your own organisation.
At Wyser, our focus is helping UK organisations realise the benefits of AI to improve productivity and increase capacity, reducing backlogs and enhancing data quality. We’re on a mission to offer trustworthy, reliable and ethical AI with the aim of helping more people access fair and inclusive services.
While we specialise in bespoke AI models, there’s still value in off-the-shelf AI products, as long as they can offer transparent and explainable outcomes and decisions. To find out how we can help, get in touch with our team.