Generative AI at Work presents the findings from a comprehensive study conducted by researchers at Stanford University and MIT.
The study examines the impact of applying an AI-based conversational assistant in a software support environment. This study provides tangible evidence about the potential benefits for generative AI in support.
The study was published in April 2023 by the NATIONAL BUREAU OF ECONOMIC RESEARCH and was authored by Erik Brynjolfsson of the Stanford Digital Economy Laboratory, Lindsey R. Raymond and Danielle Li of the MIT Sloan School of Management.
This is an excellent read for anyone considering the future of AI in service and support. Here are some of the highlights:
Overview:
- Study of a chat assistant using data from 5,000 agents working for a Fortune 500 software firm.
- The tool studied was built on a recent version of the GPT large language model developed by OpenAI.
- The tool is designed to augment agents with real-time suggestions for how to respond.
- Agents remain responsible for the conversation and are free to ignore its suggestions.
Findings:
- Access to the tool increases productivity, as measured by issues resolved per hour by 14 percent on average.
- AI assistance disproportionately increases the performance of less skilled and less experienced workers and helps them to communicate more like high-skill agents.
- Minimal impact on experienced and highly skilled workers.
- AI assistance improves customer sentiment, reduces requests for managerial intervention, and improves employee retention.
- AI assistance improves how customers treat agents, as measured by the sentiments of their chat messages.
The study is well done, comprehensive, and directly relates to the application of generative AI in a support environment. It is well worth the read. You can find the study here Generative AI at Work | NBER.
