Can computer algorithms be racist?
This was a subject of a heated debate in the US recently when Congresswoman Alexandria Ocasio-Cortez claimed that facial-recognition algorithms are biased against people with darker skin.
This was a huge claim that, if proven correct, could have serious societal implications. For example, algorithms that are racially, culturally or gender-biased could prevent women and people belonging to certain races and cultural groups from getting bank loans and being considered for jobs.
They could also force them to pay higher interest and insurance premiums.
Some people reacted to Ocasio-Cortez’s claim by mocking her, saying that algorithms are driven by maths, so can never be biased.
Others came out in support of her.
A number of experts in the field of artificial intelligence chimed in, saying she is indeed right: algorithms can be, and in many cases are, in fact, biased.
But how is this possible?
An algorithm in the most general sense is a step-by-step procedure for solving a mathematical problem. Algorithms may be as simple as calculating the area of a rectangle or as complex as calculating the trajectory of a rocket in space.
These algorithms work on a basis of inputs and outputs: you put data in, you get data out. They are straightforward in the sense that they have a set of inputs and predictable outputs. Clearly, there can be no bias there.
Around the 1950s we saw completely new types of algorithms emerge, known as machine learning (ML) algorithms.
This refers to a branch of artificial intelligence that allows computers to learn and improve their performance over time by analysing existing data.
ML algorithms are highly complex algorithms designed to analyse data, make inferences from the data and then adapt.
Over time these machines learn our search habits and then customise the search results to our habits and preferences.
As a result, two people searching for the same term will probably get completely different results.
For example, search for “Java” will likely yield results relating to coffee for a person who regularly searches for coffee, while the same search term will result in links relating to computer programming for a coder.
YouTube search and recommendations work in a similar way. If you search for a specific topic – Italian recipes, for example – you will continue to see recommendations for videos relevant to Italian recipes long after your search.
As advanced and complex as they are, these algorithms have one thing in common with their simpler counterparts: they need data inputs in order to function.
If that input is garbage, then they will simply return garbage.
In computer science circles, this is described more concisely as “garbage-in, garbage out”, more commonly known by its acronym, “Gigo”.
There have been a number rather shocking examples of Gigo in recent times, such as when Microsoft’s chatbot named “Tay” began to use racial language within a day of its launch.
The chatbot was intended to be an experiment in “conversational understanding” and it was hoped that, like a child, it could learn by listening to, and engaging with people.
Tay began on a positive note, making statements like “human are super cool”, but unfortunately things spiralled out of control very quickly, and it began to make statements like “I just hate everybody” and others that are too disturbing to mention.
By the end of the day, it began to sympathise with the Nazis. Naturally, Microsoft had to pull it down. What went wrong?
Simply put, Tay was innocent when it went live, but got into the wrong company who fed it garbage, resulting in garbage output.
The case with Amazon’s artificially intelligent recruiting tool was less dramatic, but a lot more insidious. The system was meant to automatically screen large numbers of resumes and pick out the best people for the job.
But they found out that it had a serious problem: it didn’t seem to like women.
It was discovered that the system did not screen resumes in a gender-neutral way, and was biased against women. It even penalized applicants who used words like “women’s” and terms like “women’s chess club champion”.
Although it was eventually shelved, the root cause was found to be with people, not the system itself.
It apparently picked up its bias against women by observing the company’s recruitment patterns over the previous 10 years. In other words, the algorithms basically picked up on existing biases and simply automated them.
Can we ultimately say algorithms are biased?
Machine learning algorithms certainly do not start out life with biases but, like children, they will probably pick up those biases along the way, depending on the attitudes of the people who interact with them.
If they are found to have biases, we need not look for the problem in the machines, but in their creators.
Machines will, after all, only detect and automate existing bias.
