At the start of 2021, Twitter discovered that the algorithm that selects preview images on Twitter posts had some deeply disturbing programming biases. Users found that posting numerous images with different people led to black faces and darker tones being cropped out and maligned in favour of lighter skin tones and white faces.
On Twitter, you can post a collection of images (max 4) in a tweet. A cropped curated preview of each of the four images appears in the body of a tweet. Only when you click and expand the tweet will you see the full version of each image absent of any algorithm cropping. The algorithm that selects what to preview to users is called a ‘saliency algorithm’.
“Bug bounties” and flaws in the code
To learn more about how exactly the algorithm operates and what biases it holds, Twitter organised a bounty competition at the start of the year. Social media and online companies have started to practice ‘bug bounties’ a lot recently. This is when a company offers a reward to any researchers around the world who can find flaws in their code. ‘Bug Bounties’ are becoming very common as large user bases are already willing to use and learn more about the platforms they use.
HackerOne and BugCrowd are programs that continuously offer bug bounty programs on behalf of large companies like Google, Intel and Microsoft. These companies want to find security flaws before any potentially malicious hackers can and ‘bug bounties’ are a terrific way to do so. The efficiency and monetary aspects cannot be ignored as this allows a company to use their cybersecurity budget cleverly and in a focused manner.
Twitter trends for the wrong reasons
Back at Twitter, at the beginning of the year, Twitter users had started testing different images and combinations to see what decisions the algorithm would make. This was after a period where the algorithm’s bias was trending daily with more and more troublesome examples being posted daily. When posting multiple images with black and white faces, the algorithm would crop the preview to hide black faces and show white faces instead, even when the images shared the same size and dimensions.
Twitter’s bounty competition
In response, Twitter organised a ‘algorithmic-bias bounty competition’ in July 2021. Three money-based prizes were offered to any researchers that could find any bugs and biases in the algorithm. Below, we breakdown the researchers and what they discovered:
- Bogdan Kulynyc, a Swiss graduate student, won $3,500 (£2,530) from the contest after Twitter had already found many other biases earlier in the year. Kulynyc discovered the ‘saliency’ of a face in an image could be increased – by making the person’s skin lighter and smoother, changing the appearance to be younger and slimmer. A skinny face was more likely to be shown, whilst a fuller face would be cropped. Darker and black skin tones were cropped out, whilst lighter skin tones (whether edited or natural) and white skin were highlighted. Whilst the preference for lighter skin tones was already known, Kulynyc discovered that edited images can also game the algorithm.
- The second prize went to a female-founded University of Toronto start-up called Halt AI which discovered the algorithm perpetuated in marginalising the elderly and disabled people by cropping them out of images. This was a major surprise as the thought was that the algorithm’s bias was skin and colour-focused but this opened up a whole new worry regarding the saliency programme.
- The third prize went to Taraaz Research founder Roya Pakzad. Pakzad found that the algorithm was more likely to crop out Arabic text than English text. Pakzad tested this with stories and memes with English text posted alongside Arabic text. The algorithm would hide Arabic text and push the English text forward instead. This once again highlighted the algorithm’s bias but on a different level to skin and race focus.
Every social media and online company is utilising bug bounties to discover any problems in their code and algorithms. Twitter’s saliency algorithm was the talk of the virtual town when it was discovered to favour lighter faces in place of darker ones. By enlisting the help of researchers around the world, more biases based on language, age and ability were also found.
