Key takeaways from the 2023 RE•WORK Conversational AI Summit
Last week, we were at the RE•WORK Conversational AI Summit as a sponsor of the event. It was fantastic to mingle with other conversational AI experts, as well as explain the power and the possibilities that come with this transformative technology to those who are interested in harnessing it for themselves.
As part of our presence at the event, our Chief Customer Officer, Håvard Dahl-Olsen, joined a group of experts on a panel discussion to explore how organizations can measure the success of conversational AI solutions. Moderated by Stefania-Catalina Baincescu, Team Lead CAI Romania & Technical Lead at E.ON Software Development, the panel also featured Richard Moore, Senior Research Fellow at Sheffield Hallam University and Ben Hazel, Senior Chatbot Conversational Designer at Admiral Group, and explored all aspects of measuring the success of a virtual agent.
Here are our key takeaways from the insightful discussion: -
It’s important to understand the purpose of your virtual agent
When looking at measuring the success of a conversational AI tool, the first place to start is to clearly understand what the purpose is for the virtual agent you have. Do you want to improve the customer experience and shorten average customer wait times? Perhaps you want to improve customer feedback about your services? Or maybe it is about improving your employee’s experience and supporting them with dealing with customer queries? Each of these goals is focused on a different audience and would have different KPIs attached. Work out what you want to achieve with your virtual agent, and that will help you determine how to measure its success.
You need to define what success looks like for your virtual agent
Once you know what your virtual agent will be doing for your business, you can then set out goals for the AI tool which will help to define a successful implementation. For customer-facing virtual agents, you may want to achieve a level of automation, whereby success looks like a certain percentage of customer inquiries being dealt with without any human interaction required. Or perhaps it’s a case of streamlining customers to the right agent and reducing their waiting times.
For an employee-focused AI platform, you may want to measure how quickly employees are able to find information using the virtual agent, or maybe it’s more important to record how quickly employees are able to respond to customer queries. Whatever your goals are for conversational AI, by having these clear in your mind, you can set out specific KPIs for your virtual agent.
It’s vital that you get measurement right
Once you understand your audience and you know the KPIs for your solution, it’s important to start measuring its success straight away. Any conversational AI solution will create a large amount of data, so proper data analysis is required if you plan to understand your virtual agent’s impact on user experiences. Make sure you have the right team and tools in place to fully benefit from analytics, as this can help shed insight on how your conversational AI platform is performing.
Don’t be afraid to look at other available research data as well. By gathering feedback on the user experience, whether that is from customers or employees, you will be able to understand areas where your solution could be enhanced and improved for a better overall experience. Collaboration with research organizations regarding customer relationships can also be helpful in this scenario – so you get real data from in-depth analysis of customer conversations.
Prepare to adapt and adjust as both your business and your conversational AI solution scales
It’s natural that the focus of your solution may change over time as your business grows, and you look to increase the workload for your chatbot. For example, your initial focus could be reducing mail as a channel, as part of a larger virtual agent project. Over time you’ll look at the channel mix and see you’ve achieved the KPIs you’ve initially set out, at which point you’d look to scale up and achieve the same results across phone volumes and voice solutions. There’s always something else you can automate to make customer journeys better. The high-level KPIs would remain same but the specifics and scope would evolve over time.
Similarly, if your business strategy changes suddenly and you switch your roadmap, then the direction and utility of your conversational AI tool may also change and you may need to reassess KPIs accordingly. Moreover, as you continue to scale, you’ll likely have more metrics to keep track of, so it’s imperative that you’re regularly monitoring the KPIs and that they support the ongoing business strategy.
***
Want to improve how you are tracking the success of your virtual agent?
Boost.ai is a leader in conversational AI platforms for enterprise organizations. We help businesses transform their customer and employee experience through the use of virtual agents. If you’re interested in maximizing the potential of your conversational AI tools and want to explore how to measure their success yourself, connect with one of our experts today!