Earlier this year, the independent research organisation of which I am the Director, London-based Ada Lovelace Institute, hosted a panel at the world’s largest AI conference, CogX, called The Ethics Panel to End All Ethics Panels. The title referenced both a tongue-in-cheek effort at self-promotion, and a very real need to put to bed the seemingly endless offering of panels, think-pieces, and government reports preoccupied with ruminating on the abstract ethical questions posed by AI and new data-driven technologies. We had grown impatient with conceptual debates and high-level principles.
And we were not alone. 2020 has seen the emergence of a new wave of ethical AI – one focused on the tough questions of power, equity, and justice that underpin emerging technologies, and directed at bringing about actionable change. It supersedes the two waves that came before it: the first wave, defined by principles and dominated by philosophers, and the second wave, led by computer scientists and geared towards technical fixes. Third-wave ethical AI has seen a Dutch Court shut down an algorithmic fraud detection system, students in the UK take to the streets to protest against algorithmically-decided exam results, and US companies voluntarily restrict their sales of facial recognition technology. It is taking us beyond the principled and the technical, to practical mechanisms for rectifying power imbalances and achieving individual and societal justice.
From philosophers to techies
Between 2016 and 2019, 74 sets of ethical principles or guidelines for AI were published. This was the first wave of ethical AI, in which we had just begun to understand the potential risks and threats of rapidly advancing machine learning and AI capabilities and were casting around for ways to contain them. In 2016, AlphaGo had just beaten Lee Sedol, promoting serious consideration of the likelihood that general AI was within reach. And algorithmically-curated chaos on the world’s duopolistic platforms, Google and Facebook, had surrounded the two major political earthquakes of the year – Brexit, and Trump’s election.
In a panic for how to understand and prevent the harm that was so clearly to follow, policymakers and tech developers turned to philosophers and ethicists to develop codes and standards. These often recycled a subset of the same concepts and rarely moved beyond high-level guidance or contained the specificity of the kind needed to speak to individual use cases and applications.
This first wave of the movement focused on ethics over law, neglected questions related to systemic injustice and control of infrastructures, and was unwilling to deal with what Michael Veale, Lecturer in Digital Rights and Regulation at University College London, calls “the question of problem framing” – early ethical AI debates usually took as a given that AI will be helpful in solving problems. These shortcomings left the movement open to critique that it had been co-opted by the big tech companies as a means of evading greater regulatory intervention. And those who believed big tech companies were controlling the discourse around ethical AI saw the movement as “ethics washing.” The flow of money from big tech into codification initiatives, civil society, and academia advocating for an ethics-based approach only underscored the legitimacy of these critiques.
At the same time, a second wave of ethical AI was emerging. It sought to promote the use of technical interventions to address ethical harms, particularly those related to fairness, bias and non-discrimination. The domain of “fair-ML” was born out of an admirable objective on the part of computer scientists to bake fairness metrics or hard constraints into AI models to moderate their outputs.
This focus on technical mechanisms for addressing questions of fairness, bias, and discrimination addressed the clear concerns about how AI and algorithmic systems were inaccurately and unfairly treating people of color or ethnic minorities. Two specific cases contributed important evidence to this argument. The first was the Gender Shades study, which established that facial recognition software deployed by Microsoft and IBM returned higher rates of false positives and false negatives for the faces of women and people of color. The second was the 2016 ProPublica investigation into the COMPAS sentencing algorithmic tool, which found that Black defendants were far more likely than White defendants to be incorrectly judged to be at a higher risk of recidivism, while White defendants were more likely than Black defendants to be incorrectly flagged as low risk.
Second-wave ethical AI narrowed in on these questions of bias and fairness, and explored technical interventions to solve them. In doing so, however, it may have skewed and narrowed the discourse, moving it away from the root causes of bias and even exacerbating the position of people of color and ethnic minorities. As Julia Powles, Director of the Minderoo Tech and Policy Lab at the University of Western Australia, argued, alleviating the problems with dataset representativeness “merely co-opts designers in perfecting vast instruments of surveillance and classification. When underlying systemic issues remain fundamentally untouched, the bias fighters simply render humans more machine readable, exposing minorities in particular to additional harms.”
Some also saw the fair-ML discourse as a form of co-option of socially conscious computer scientists by big tech companies. By framing ethical problems as narrow issues of fairness and accuracy, companies could equate expanded data collection with investing in “ethical AI.”
The efforts of tech companies to champion fairness-related codes illustrate this point: In January 2018, Microsoft published its “ethical principles” for AI, starting with “fairness;” in May 2018, Facebook announced a tool to “search for bias” called “Fairness Flow;” and in September 2018, IBM announced a tool called “AI Fairness 360,” designed to “check for unwanted bias in datasets and machine learning models.”
What was missing from second-wave ethical AI was an acknowledgement that technical systems are, in fact, sociotechnical systems — they cannot be understood outside of the social context in which they are deployed, and they cannot be optimised for societally beneficial and acceptable outcomes through technical tweaks alone. As Ruha Benjamin, Associate Professor of African American Studies at Princeton University, argued in her seminal text, Race After Technology: Abolitionist Tools for the New Jim Code, “the road to inequity is paved with technical fixes.” The narrow focus on technical fairness is insufficient to help us grapple with all of the complex tradeoffs, opportunities, and risks of an AI-driven future; it confines us to thinking only about whether something works, but doesn’t permit us to ask whether it should work. That is, it supports an approach that asks, “What can we do?” rather than “What should we do?”
Ethical AI for a new decade
On the eve of the new decade, MIT Technology Review’s Karen Hao published an article entitled “In 2020, let’s stop AI ethics-washing and actually do something.” Weeks later, the AI ethics community ushered in 2020 clustered in conference rooms at Barcelona, for the annual ACM Fairness, Accountability and Transparency conference. Among the many papers that had tongues wagging was written by Elettra Bietti, Kennedy Sinclair Scholar Affiliate at the Berkman Klein Center for Internet and Society. It called for a move beyond the “ethics-washing” and “ethics-bashing” that had come to dominate the discipline. Those two pieces heralded a cascade of interventions that saw the community reorienting around a new way of talking about ethical AI, one defined by justice — social justice, racial justice, economic justice, and environmental justice. It has seen some eschew the term “ethical AI” in favor of “just AI.”
As the wild and unpredicted events of 2020 have unfurled, alongside them third-wave ethical AI has begun to take hold, strengthened by the immense reckoning that the Black Lives Matter movement has catalysed. Third-wave ethical AI is less conceptual than first-wave ethical AI, and is interested in understanding applications and use cases. It is much more concerned with power, alive to vested interests, and preoccupied with structural issues, including the importance of decolonising AI. An article published by Pratyusha Kalluri, founder of the Radical AI Network, in Nature in July 2020, has epitomized the approach, arguing that “When the field of AI believes it is neutral, it both fails to notice biased data and builds systems that sanctify the status quo and advance the interests of the powerful. What is needed is a field that exposes and critiques systems that concentrate power, while co-creating new systems with impacted communities: AI by and for the people.”
What has this meant in practice? We have seen courts begin to grapple with, and political and private sector players admit to, the real power and potential of algorithmic systems. In the UK alone, the Court of Appeal found the use by police of facial recognition systems unlawful and called for a new legal framework; a government department ceased its use of AI for visa application sorting; the West Midlands police ethics advisory committee argued for the discontinuation of a violence-prediction tool; and high school students across the country protested after tens of thousands of school leavers had their marks downgraded by an algorithmic system used by the education regulator, Ofqual. New Zealand published an Algorithm Charter and France’s Etalab – a government task force for open data, data policy, and open government – has been working to map the algorithmic systems in use across public sector entities and to provide guidance.
The shift in gaze of ethical AI studies away from the technical towards the socio-technical has brought more issues into view, such as the anti-competitive practices of big tech companies, platform labor practices, parity in negotiating power in public sector procurement of predictive analytics, and the climate impact of training AI models. It has seen the Overton window contract in terms of what is reputationally acceptable from tech companies; after years of campaigning by researchers like Joy Buolamwini and Timnit Gebru, companies such as Amazon and IBM have finally adopted voluntary moratoria on their sales of facial recognition technology.
The COVID crisis has been instrumental, surfacing technical advancements that have helped to fix the power imbalances that exacerbate the risks of AI and algorithmic systems. The availability of the Google/Apple decentralised protocol for enabling exposure notification prevented dozens of governments from launching invasive digital contact tracing apps. At the same time, governments’ response to the pandemic has inevitably catalysed new risks, as public health surveillance has segued into population surveillance, facial recognition systems have been enhanced to work around masks, and the threat of future pandemics is leveraged to justify social media analysis. The UK’s attempt to operationalize a weak Ethics Advisory Board to oversee its failed attempt at launching a centralized contact-tracing app was the death knell for toothless ethical figureheads.
Research institutes, activists, and campaigners united by the third-wave approach to ethical AI continue to work to address these risks, with a focus on practical tools for accountability (we at the Ada Lovelace Institute, and others such as AI Now, are working on developing audit and assessment tools for AI; and the Omidyar Network has published its Ethical Explorer toolkit for developers and product managers), litigation, protest and campaigning for moratoria, and bans.
Researchers are interrogating what justice means in data-driven societies, and institutes such as Data & Society, the Data Justice Lab at Cardiff University, JUST DATA Lab at Princeton, and the Global Data Justice project at the Tilberg Institute for Law, Technology and Society in the Netherlands are churning out some of the most novel thinking. The Mindaroo Foundation has just launched its new “future says” initiative with a $3.5 million grant, with aims to tackle lawlessness, empower workers, and reimagine the tech sector. The initiative will build on the critical contribution of tech workers themselves to the third wave of ethical AI, from AI Now co-founder Meredith Whittaker’s organizing work at Google before her departure last year, to walk outs and strikes performed by Amazon logistic workers and Uber and Lyft drivers.
But the approach of third-wave ethical AI is by no means accepted across the tech sector yet, as evidenced by the recent acrimonious exchange between AI researchers Yann LeCun and Timnit Gebru about whether the harms of AI should be reduced to a focus on bias. Gebru not only reasserted well established arguments against a narrow focus on dataset bias but also made the case for a more inclusive community of AI scholarship.
Mobilized by social pressure, the boundaries of acceptability are shifting fast, and not a moment too soon. But even those of us within the ethical AI community have a long way to go. A case in point: Although we’d programmed diverse speakers across the event, the Ethics Panel to End All Ethics Panels we hosted earlier this year failed to include a person of color, an omission for which we were rightly criticized and hugely regretful. It was a reminder that as long as the domain of AI ethics continues to platform certain types of research approaches, practitioners, and ethical perspectives to the exclusion of others, real change will elude us. “Ethical AI” can not only be defined from the position of European and North American actors; we need to work concertedly to surface other perspectives, other ways of thinking about these issues, if we truly want to find a way to make data and AI work for people and societies across the world.
Carly Kind is a human rights lawyer, a privacy and data protection expert, and Director of the Ada Lovelace Institute.
