LAST MAY, San Francisco became the first city in the world to ban the use of face surveillance technology. Days later, Somerville, Massachusetts, became the first city on the East Coast to do the same. Now—thanks to a movement led by dozens of civil rights organizations nationwide—municipalities and states across the country are debating the government’s use of a technology that poses unprecedented threats to our civil rights and civil liberties.
Face recognition systems use computer algorithms paired with databases to analyze and classify images of human faces in order to identify or track people. The technology is currently entirely unregulated in the U.S., but police departments and other government agencies are nonetheless using it—too often in secret. But it’s not like what you’ve seen on cop shows like CSI; face recognition doesn’t always work. And the inaccuracies are particularly damaging for certain groups of people, notably for black women and for trans and nonbinary people.
For example, a study conducted by MIT researcher Joy Buolamwini found significant racial and gender bias in facial recognition algorithms. While face recognition algorithms almost always get it right when classifying white men’s faces, they misclassified black women’s faces nearly 35 percent of the time. Moreover, this technology often fails to take into account transgender and nonbinary people. Studies show that face recognition products sold by Amazon, IBM, and Microsoft consistently misclassify people in these communities.
A critical shortcoming of this technology is that it has been trained to read people as either male or female—a technological assertion that the gender binary is absolute. Even within the confines of this rigid binary, the technology has a rather retrograde view of what “male” and “female” look like. For example, systems can be trained to recognize short hair as a “male” trait or makeup as a “female” characteristic. These outcomes reflect choices made by computer programmers about which images they will use to train algorithms, as well as how those training data are classified.
One recent study revealed that face recognition technology can accurately guess the gender only of cisgender women and men. On average, the technology got it right for cisgender individuals around 98 percent of the time. But the same system consistently performed worse on transgender individuals, and universally was unable to classify nonbinary faces.
This deficiency is already proving to have harmful effects. A 2018 report stated that some transgender individuals who drove for Uber had their accounts suspended because the company used face recognition software as a built-in security feature, and the software was unable to recognize the faces of individuals who were transitioning.
Face surveillance mistakes can pose serious problems for anyone who does not conform to the traditional æsthetic of binary gender options. When governments use these systems, trans and nonbinary people are at risk of being misgendered or even rendered invisible. That’s the last thing we need for communities that already face serious stigma and high rates of homelessness and incarceration.
Face surveillance technology’s rigid adherence to traditional gender norms threatens to exacerbate these health risks and erase the existence of anyone who does not conform. On a systemic level, the use of these technologies underscores the notion that transgender and gender nonconforming people are not part of “normal” society. We cannot allow technology to be used in areas like government services, policing, or border control if it excludes and alienates or otherwise ignores an entire class of people. Unfortunately, absent regulation, that’s exactly what’s happening.
Today, face recognition systems consistently fail to identify transgender and nonbinary people, meaning millions of human beings are at risk of being rendered invisible in our increasingly digital world. But we won’t return to the 1950s, whether by the Trump administration trying to reverse anti-discrimination protections or through automated enforcement of outdated gender norms. Winning this fight is critical to the welfare of transgender, gender nonconforming, and nonbinary members of the LGBT community.
Tre’Andre Valentine is the executive director of the Massachusetts Transgender Political Coalition. This piece was adapted from a version published inRainbow Times (New England) andThe Advocate (on-line).
