ChatGPT allowed study participants to finish a writing assignment faster than participants who did not have access to the tool.Credit: Gabby Jones/Bloomberg via Getty
ChatGPT can help people perform writing assignments faster and better, a new study1 says. The artificial-intelligence (AI) tool is especially helpful for those who have relatively weak writing skills, boosting their performance to levels closer to those of more-proficient individuals.
The results raise questions about whether ChatGPT and similar tools will displace human labour. But the findings also hint at possible social benefits, the authors say.
“This could potentially mean that, in the longer run, we could see reductions in inequality as performance is equalized across different skill groups,” says Whitney Zhang, a PhD student in economics at the Massachusetts Institute of Technology (MIT) in Cambridge, and a co-author of the study.
Writing assignment
Table of Contents
When ChatGPT was launched in November 2022, there was widespread speculation about how the tool — which generates fluent text when prompted — would affect the labour market, Zhang says. So she and her colleague Shakked Noy, also an economics PhD student at MIT, designed an experiment to understand the chatbot’s effect on workers’ productivity. The results were published today in Science.
The researchers recruited 453 college-educated professionals, including marketers, grant writers and managers, and asked them to write press releases, short reports, analysis plans and other texts.
How Nature readers are using ChatGPT
After completing an initial assignment, about half of the participants were instructed to register for ChatGPT and use it to generate a second piece of text if they found it helpful. The group exposed to the chatbot finished the second task significantly faster: in 16 minutes on average, compared with an average of 27 minutes for the group that did not have access.
The authors had experienced professionals rate the texts for quality on a scale from 1 to 7. The increase in score from the first task to the second was 18% higher, on average, for the group with access to the chatbot compared to the group without access.
The quality boost was bigger for participants who received a low score on their first task. Their score increased by 1–2 points with chatbot access, whereas those who initially received high scores maintained them after ChatGPT was introduced. Time to complete the second task dropped whether participants had high or low scores on the first task.
“It has almost like a democratizing effect,” says economist Robert Seamans at the New York University’s Stern School of Business, in New York City, who was not involved in the research. “The workers who are less experienced are the ones who would benefit more from it.”
Bot-based workforce
That is good news considering the rising inequality in the labour market, says Anton Korinek, an economist at think tank The Brookings Institution, who is based in Charlottesville, Virginia. “But there is also a hint of bad news there,” he adds: ChatGPT could obviate the need for certain skills. “All the white-collar workers who have great writing skills — and also other analytic skills that language models can perform — their skills are suddenly becoming devalued.”
Some of the study’s findings seem to corroborate that point. Participants using ChatGPT did not heavily edit the text it produced, and the small changes they introduced did not contribute to any improvement in their score.
Does that mean that AI tools could completely replace some workers? “Most jobs involve a large variety of tasks, and writing is part of them. That part can be facilitated and augmented with language models, but all the other parts are still there and they will still require humans, at least for now,” Korinek says.
It’s difficult to predict how this boosted productivity will affect wages, Seamans says. “But it really behooves all workers to learn as much as we can about the technology,” he says. “AI is going to be ubiquitous, much in the way that the Internet now is ubiquitous. And people that understand how to work with the technology are those that are going to be compensated well.”