Why AI’s diversity crisis matters, and how to tackle it

Artificial intelligence (AI) is facing a diversity crisis. If it isn’t addressed promptly, flaws in the working culture of AI will perpetuate biases that ooze into the resulting technologies, which will exclude and harm entire groups of people. On top of that, the resulting ‘intelligence’ will be flawed, lacking varied social-emotional and cultural knowledge.

In a 2019 report from New York University’s AI Now Institute, researchers noted that more than 80% of AI professors were men. Furthermore, Black individuals made up just 2.5% of Google employees and 4% of those working at Facebook and Microsoft. In addition, the report authors noted that the “overwhelming focus on ‘women in tech’ ” when discussing diversity issues in AI “is too narrow and likely to privilege white women over others”.

Some researchers are fighting for change, but there’s also a culture of resistance to their efforts. “Beneath this veneer of ‘oh, AI is the future, and we have all these sparkly, nice things’, both AI academia and AI industry are fundamentally conservative,” says Sabine Weber, a scientific consultant at VDI/VDE Innovation + Technik, a technology consultancy headquartered in Berlin. AI in both sectors is “dominated by mostly middle-aged white men from affluent backgrounds. They are really attached to the status quo”, says Weber, who is a core organizer of the advocacy group Queer in AI. Nature spoke to five researchers who are spearheading efforts to change the status quo and make the AI ecosystem more equitable.

DELALI AGBENYEGAH: Bolster African AI

Senior data science manager at Shopify in Atlanta, Georgia, and a general chair of the 2023 Deep Learning Indaba conference.

I am originally from Ghana and did my master’s in statistics at the University of Akron in Ohio in 2011. My background is in using machine learning to solve business problems in customer-experience management. I apply my analytics skills to build models that drive customer behaviour, such as customer-targeting recommendation systems, aspects of lead scoring — the ranking of potential customers, prioritizing which ones to contact for different communications — and things of that nature.

This year, I’m also a general chair for the Deep Learning Indaba, a meeting of the African machine-learning and AI community that is held in a different African country every year. Last year, it was held in Tunisia. This year, it is taking place in Ghana in September.

Our organization is built for all of Africa. Last year, 52 countries participated. The goal is to have all 54 African countries represented. Deep Learning Indaba empowers each country to have a network of people driving things locally. We have the flagship event, which is the annual conference, and country-specific IndabaX events (think TED and TEDx talks).

During Ghana’s IndabaX conferences, we train people in how to program and how to deal with different kinds of data. We also do workshops on what is happening in the industry outside of Ghana and how Ghana should be involved. IndabaX provides funding and recommends speakers who are established researchers working for companies such as Deep Mind, Microsoft and Google.

To strengthen machine learning and AI and inclusion in Ghana, we need to build capacity by training young researchers and students to understand the skill sets and preparation they need to excel in this field. The number one challenge we face is resources. Our economic status is such that the focus of the government and most Ghanaians is on people’s daily bread. Most Ghanaians are not even thinking about technological transformation. Many local academics don’t have the expertise to teach the students, to really ground them in AI and machine learning.

Most of the algorithms and systems we use today were created by people outside Africa. Africa’s perspective is missing and, consequently, biases affect Africa. When we are doing image-related AI, there aren’t many African images available. African data points make up no more than 1% of most industry machine-learning data sets.

When it comes to self-driving cars, the US road network is nice and clean, but in Africa, the network is very bumpy, with a lot of holes. There’s no way that a self-driving car trained on US or UK roads could actually work in Africa. We also expect that using AI to help diagnose diseases will transform people’s lives. But this will not help Africa if people are not going there to collect data, and to understand African health care and related social-support systems, sicknesses and the environment people live in.

Today, African students in AI and machine learning must look for scholarships and leave their countries to study. I want to see this change and I hope to see Africans involved in decision-making, pioneering huge breakthroughs in machine learning and AI research.

Researchers outside Africa can support African AI by mentoring and collaborating with existing African efforts. For example, we have Ghana NLP, an initiative focused on building algorithms to translate English into more than three dozen Ghanaian languages. Global researchers volunteering to contribute their skill set to African-specific research will help with efforts like this. Deep Learning Indaba has a portal in which researchers can sign up to be mentors.

MARIA SKOULARIDOU: Dismantle AI’s ableist culture

PhD candidate in biostatistics at the University of Cambridge, UK, and founder and chair of {Dis}Ability in AI.

I founded {Dis}Ability in AI in 2018, because I realized that disabled people weren’t represented at conferences and it didn’t feel right. I wanted to start such a movement so that conferences could be inclusive and accessible, and disabled people such as me could attend them.

That year, at NeurIPS — the annual conference on Neural Information Processing Systems — in Montreal, Canada, at least 4,000 people attended and I couldn’t identify a single person who could be categorized as visibly disabled. Statistically, it doesn’t add up to not have any disabled participants.

I also observed many accessibility issues. For example, I saw posters that were inconsiderate with respect to colour blindness. The place was so crowded that people who use assistive devices such as wheelchairs, white canes or service dogs wouldn’t have had room to navigate the poster session. There were elevators, but for somebody with limited mobility, it would not have been easy to access all the session rooms, given the size of the venue. There were also no sign-language interpreters.

Since 2019, {Dis}Ability in AI has helped facilitate better accessibility at NeurIPS. There were interpreters, and closed captioning for people with hearing problems. There were volunteer escorts for people with impaired mobility or vision who requested help. There were hotline counsellors and silent rooms because large conferences can be overwhelming. The idea was: this is what we can provide now, but please reach out in case we are not considerate with respect to something, because we want to be ethical, fair, equal and honest. Disability is part of society, and it needs to be represented and included.

Many disabled researchers have shared their fears and concerns about the barriers they face in AI. Some have said that they wouldn’t feel safe sharing details about their chronic illness, because if they did so, they might not get promoted, be treated equally, have the same opportunities as their peers, be given the same salary and so on. Other AI researchers who reached out to me had been bullied and felt that if they spoke up about their condition again, they could even lose their jobs.

People from marginalized groups need to be part of all the steps of the AI process. When disabled people are not included, the algorithms are trained without taking our community into account. If a sighted person closes their eyes, that does not make them understand what a blind person must deal with. We need to be part of these efforts.Being kind is one way that non-disabled researchers can make the field more inclusive. Non-disabled people could invite disabled people to give talks or be visiting researchers or collaborators. They need to interact with our community at a fair and equal level.

WILLIAM AGNEW AND SABINE WEBER: Queering AI

William Agnew is a computer science PhD candidate at the University of Washington in Seattle. Sabine Weber is a scientific consultant at VDI/VDE Innovation + Technik in Erfurt, Germany. They are organizers of the advocacy organization Queer in AI.

Agnew: I helped to organize the first Queer in AI workshop for NeurIPS in 2018. Fundamentally, the AI field doesn’t take diversity and inclusion seriously. Every step of the way, efforts in these areas are underfunded and underappreciated. The field often protects harassers.

Most people doing the work in Queer in AI are graduate students, including me. You can ask, “Why isn’t it the senior professor? Why isn’t it the vice-president of whatever?” The lack of senior members limits our operation and what we have the resources to advocate for.

The things we advocate for are happening from the bottom up. We are asking for gender-neutral toilets; putting pronouns on conference registration badges, speaker biographies and in surveys; opportunities to run our queer-AI experiences survey, to collect demographics, experiences of harm and exclusion, and the needs of the queer AI community; and we are opposing extractive data policies. We, as a bunch of queer people who are marginalized by their queerness and who are the most junior people in our field, must advocate from those positions.

In our surveys, queer people consistently name the lack of community, support and peer groups as their biggest issues that might prevent them from continuing a career path in AI. One of our programmes gives scholarships to help people apply to graduate school, to cover the fees for applications, standardized admissions tests, such as the Graduate Record Examination (GRE) and university transcripts. Some people must fly to a different country to take the GRE. It’s a huge barrier, especially for queer people, who are less likely to have financial support from their families and who experience repressive legal environments. For instance, US state legislatures are passing anti-trans and anti-queer laws affecting our membership.

In large part because of my work with Queer in AI, I switched from being a roboticist to being an ethicist. How queer peoples’ data are used, collected and misused is a big concern. Another concern is that machine learning is fundamentally about categorizing items and people and predicting outcomes on the basis of the past. These things are antithetical to the notion of queerness, where identity is fluid and often changes in important and big ways, and frequently throughout life. We push back and try to imagine machine-learning systems that don’t repress queerness.

You might say: “These models don’t represent queerness. We’ll just fix them.” But queer people have long been the targets of different forms of surveillance aimed at outing, controlling or suppressing us, and a model that understands queer people well can also surveil them better. We should avoid building technologies that entrench these harms, and work towards technologies that empower queer communities.

Weber: Previously, I worked as an engineer at a technology company. I said to my boss that I was the only person who was not a cisgender dude in the whole team of 60 or so developers. He replied, “You were the only person who applied for your job who had the qualification. It’s so hard to find qualified people.”

But companies clearly aren’t looking very hard. To them it feels like: “We’re sitting on high. Everybody comes to us and offers themselves.” Instead, companies could recruit people at queer organizations, at feminist organizations. Every university has a women in science, technology, engineering and mathematics (STEM) group or women in computing group that firms could easily go to.

But the thinking, “That’s how we have always done it; don’t rock the boat”, is prevalent. It’s frustrating. Actually, I really want to rock the boat, because the boat is stupid. It’s such a disappointment to run up against these barriers.

LAURA MONTOYA: Evolve to meet Latinx community needs

Executive director of the Accel.AI Institute and LatinX in AI in San Francisco, California.

In 2016, I started the Accel.AI Institute as an education company that helps under-represented or underserved people in AI. Now, it’s a non-profit organization with the mission of driving AI for social impact initiatives. I also co-founded the LatinX in AI programme, a professional body for people of Latin American background in the field. I’m first generation in the United States, because my family emigrated from Colombia.

My background is in biology and physical science. I started my career as a software engineer, but conventional software engineering wasn’t rewarding for me. That’s when I found the world of machine learning, data science and AI. I investigated the best way to learn about AI and machine learning without going to graduate school. I’ve always been an alternative thinker.

I realized there was a need for alternative educational options for people like me, who don’t take the typical route, who identify as women, who identify as people of colour, who want to pursue an alternative path for working with these tools and technologies.

Later on, while attending large AI and machine-learning conferences, I met others like myself, but we made up a small part of the population. I got together with these few friends to brainstorm, “How can we change this?”. That’s how LatinX in AI was born. Since 2018, we’ve launched research workshops at major conferences, and hosted our own call for papers in conjunction with NeurIPS.

We also have a three-month mentorship programme to address the brain drain resulting from researchers leaving Latin America for North America, Europe and Asia. More senior members of our community and even allies who are not LatinX can serve as mentors.

In 2022, we launched our supercomputer programme, because computational power is severely lacking in much of Latin America. For our pilot programme, to provide research access to high-performance computing resources at the Guadalajara campus of the Monterey Institute of Technology in Mexico, the technology company NVIDIA, based in Santa Clara, California, donated a DGX A100 system — essentially a large server computer. The government agency for innovation in the Mexican state of Jalisco will host the system. Local researchers and students can share access to this hardware for research in AI and deep learning. We put out a global call for proposals for teams that include at least 50% Latinx members who want to use this hardware, without having to be enrolled at the institute or even be located in the Guadalajara region.

So far, eight teams have been selected to take part in the first cohort, working on projects that include autonomous driving applications for Latin America and monitoring tools for animal conservation. Each team gets access to one graphics processing unit, or GPU — which is designed to handle complex graphics and visual-data processing tasks in parallel — for the period of time they request. This will be an opportunity for cross-collaboration, for researchers to come together to solve big problems and use the technology for good.