When the COVID-19 shutdown began in March throughout the United States, my team at Adobe had to face a stark reality: Business as usual was no longer an option. Suddenly, over just a single weekend, we had to shift our global workforce of over 22,000 people to working remotely. Not surprisingly, our existing processes and workflows weren’t equipped for this abrupt change. Customers, employees, and partners — many also working at home — couldn’t wait days to receive answers to urgent questions.
We realized pretty quickly that the only way to meet their needs was to completely rethink our support infrastructure.
Our first step was to launch an organization-wide open Slack channel that would tie together the IT organization and the entire Adobe employee community. Our 24×7 global IT help desk would front the support on that channel, while the rest of IT was made available for rapid event escalation.
As we began building the framework and interfaces on our Slack Channel, we realized the same, specific questions and issues were coming up frequently. By focusing on the most common and weighty issues, we decided to optimize our support for frequently asked questions and issues. We dubbed this AI and machine-learning-based Slack channel “#wfh-support,” and it had built-in natural language processing (NLP).
The chatbot’s answers could be as simple as directing employees to an existing knowledge base article or FAQ, or walking them through steps to solve a problem, such as setting up a virtual private network. We chose to focus first on the eight most frequently reported topics, and today we’re continuing to add capabilities as we learn what works and what delivers the biggest benefits.
Clear results – satisfied employees
The results have been remarkable. Since the initiative went live on April 14, the automated system has responded to more than 3,000 queries, and we’ve witnessed significant improvements in critical areas. For example, we noticed more employees were seeking IT support through email when we shifted to work from home, and it became important to decrease the turnaround time on email help tickets. With the help of a deep learning and NLP based routing mechanism, 38% of email tickets are now automatically routed to the correct support queue within six minutes. The AI routing bot uses a neural network-based classification technique to sort email tickets into classes, or support queues. Based on the predicted classification, the ticket is automatically assigned to the correct support queue.
This AI enhancements has reduced the average time required to dispatch and route email tickets from about 10 hours to less than 20 minutes. Continuous supervised training on the routing bot has helped us reach approximately 97% accuracy — nearly on par with a human expert. As a result, call volumes for internal support have dropped by 35%.
We improve on the response and resolution rates of our chatbot by continuously reviewing past conversations in the Slack channel and identifying keywords to refine the rule-based engine, labelling data from past conversations to help train the NLP model for better intent matching and reviewing conversations to identify top issues and create new bot responses. We retrain the routing bot’s neural network model every two weeks by adding new data from resolved tickets to the training set. This not only helps to identify new or changed routing patterns but also enables the model to re-learn and avoid past errors in future predictions.
Making conversations count
As we continue to transition additional process functions to AI and chatbots, we’re focused on a few core considerations. First, we examine where a high return on investment results from the technology – taking into account numbers and metrics to point us in the right direction. At the same time, we closely consider how technology impacts customers and employees and where it delivers value.
Once we have identified a path, we allow groups to experiment, testing chatbots and AI for different purposes and in novel ways so we can learn and grow. We have also established a center of excellence that allows us to share knowledge about what we learn internally quickly and widely. For example, we’re leveraging the work done on our Slack “#wfh-support” channel in other conversational chatbots for finance and customer-facing tasks. Another area we’re continuing to look at is robotic process automation (RPA), which refers to business improvements that result through the combination of autonomous software robots (bots) and AI. We’re continuing to experiment with and evaluate new ways to leverage RPA technology to enhance our employees’ experience.
Finally, it’s critical to address change management issues. We view this challenge as even more important than getting the technology exactly right — especially at the beginning of an initiative. People must understand AI and chatbot technology, why it’s being used, how it can help them, and how their roles may change. When introducing a new/unknown technology tool, it’s critical to keep employee experience at the core of the training and integration process – to ensure they feel comfortable and confident with the change.
To ensure a smooth implementation, we’re collaborating with our training partner, Coursera, to roll-out AI training for our workforce via a six-month, technical AI and machine learning training and certification program for our global engineers. The goal is to help all our engineers be AI savvy given the growing role of AI and automation in their day-to-day work.
AI and chatbots have emerged as a new “complementary” workforce at Adobe. The technology enhances what our teams can do and frees them to tackle work more efficiently and strategically. Industry research supports this approach. A 2017 PwC report found that 72% of business executives believe that AI produces business advantage.
Although there’s no easy way to navigate the pandemic and digital transformation, the strategic use of AI automation and chatbots can deliver value to everyone in the employee ecosystem. It’s a technology that’s ready for day-to-day prime time.
Cynthia Stoddard is Senior Vice President and CIO at Adobe.
