Automation for Customer Excellence: An Internship Journey in Tokopedia Customer Operations

Be curious, ask the right question, look into the data, make improvements

obody expects to be greeted nor replied by robots. I think we can agree to that, don’t we? But, what if actually an automated system for customer service can eventually increase your customer satisfaction?

This is a story based on my experience during the internship as a part of Customer Operations Team at Tokopedia during Jun-Aug 2019. I took the role as a Project Delivery Analyst, which was responsible for handling, reporting, and analyzing customer operations data and turn them into insights to improve the service performance. I was specifically in charge of the live chat channel in New Retail tribe (Mitra Tokopedia).

Me and Customer Operations Team
Me and Live Chat Team

Before introducing the proposed automation, let me tell the story about how we managed to end up with the solution. First things first, look into the data! For the live chat channel, we measured several metrics to maintain performance, such as FRT (first response time), abandon rate, and QM score (quality measures score). During my tenure, the major problem we encountered was an increasing abandon rate week by week, while FRT and QM score metrics were still maintained. Abandon rate was a metric that measured how many tickets that were not responded by agents until the user left the live chat page.

This circumstance led my curiosity to other questions, was the abandon rate increasing due to an increasing number of tickets? If so, was there any intermittent issue that caused users to reach our contact center? Or perhaps the number of users was increasing, then indirectly affected the number of tickets to increase as well?

Through a further investigation, we found that there were both intermittent errors and a significant increase in the number of users, which caused the increasing number of tickets. We might ask for the agents to work overtime to handle tickets due to intermittent issues, but an overtime system wouldn’t work for the long-run. While the number of agents remained the same, this would obviously lead to a higher abandon rate. Then, another scenario popped up in my head. If we needed to add manpower, how much did we need?

To estimate the number, other than the number of abandoned chats, I analyzed other metrics such as handling time and waiting time. I needed to know how long on average did an agent need to solve tickets (avg. handling time) in a day, and how long was a user willing to wait to get his/her ticket to be first responded by an agent (avg. waiting time)? Surprisingly, I found that on days with a high abandon rate, both the average handling time and average waiting time were shorter. Of course, once again, I asked myself, why???

Long story short, after discussing with the team and other stakeholders, we found the potential root causes:

  1. The users were not aware when the agents had already been in full capacity and when they needed to wait for their ticket to be responded. Instead of that, the users were notified that the system was offline. This led users to leave immediately without waiting, which was the main cause of the short waiting time.
  2. If there was still any queue capacity left, the users did not get an offline system notification, but the live chat screen went blank. Some users may be willing to wait, but if it was too long, they might be no longer stand by to reply once an agent was available and greeted them. This did not lead to an abandoned chat, but there was also no conversation going between user and agent. Hence, the ticket only had a short handling time and we did not get any insight about what problem was currently going on with the users.
The misleading offline notification

In determining the optimum number of additional manpower needed, we know that too much manpower can be costly. Implementing an automated system can be the smart solution for increasing the capacity to handle more tickets. Moreover, the system can help us to keep the conversation going with the users so that we can gain more insights.

Service recovery paradox

According to Wirtz & Lovelock¹, there is a phenomenon where customers who experience any issue but receive an excellent service recovery, will likely to feel more satisfied, compared to customers who don’t encounter any issue in the first place, which is known as the service recovery paradox. Furthermore, service failure is inevitable, since service has the characteristic of heterogeneity (variability of quality).

Let me illustrate with the 80/20 principle (Pareto principle): We may need to spend about 80% effort to overcome the excess 20% probability of cause of service failures. Thus, the effort in delivering a perfect service quality can be very costly. Instead of spending too much effort, such as hiring too much manpower (which will mostly end up as idle capacity) to have zero service failure, it is more feasible to develop a strategy for delivering excellent service recovery. For example, an e-commerce platform that manages to solve a complaint from a user and mediates the transaction between the user and seller, will gain trust from the user to make a purchase again from the same e-commerce platform.

The 80/20 principle (Source: 50minutes.com)

However, regarding the service recovery paradox, there are several things that should be noted:

  • The paradox may not apply to the second failure. Once the customer has received a very good recovery, it will be the new standard of expectation from the customer. It will be very expensive for the service provider to work to meet the new service standard.
  • We should pay attention to the severity and recoverability of service failure. Some failures may not be tolerated and can be too much of a loss. For example, late delivery for a package that should be for a surprise party is not recoverable.

Both initial service quality and service recovery are important to boost customer satisfaction, and these two complete each other. Though, when the service failure is inevitable, it is always nice to conduct excellent service recovery and turn it into feedback to avoid the same thing to occur again, as well as improving our initial service quality. Technology is one of the resources that can be utilized to generate feedback.

How we improve

UI/UX Design

The misleading offline notification system was the first thing that we needed to get rid of when the agent capacity was not sufficient to handle the tickets. Moreover, we increased the optimum queue capacity so that we could pool more users, as well as notified them with a friendlier message to give the best experience for users. We improved the queue interface and user experience by telling the users that they were in the queue, providing them with the countdown number, and ensuring that our agents would soon be ready to serve them. The improved queuing system design allowed users to keep updated automatically about their queue numbers, so that hopefully users might still be on stand by when an agent greets them and both parties can continue the conversation.

We care for your queuing experience!

Chatbot

After implementing the UI/UX design for live chat queuing system, we strived to Make It Happen, Make It Better, so then we discovered another potential improvement. Another way to trigger the conversation when the agent capacity was already overload, is by implementing a chatbot system. Instead of asking the users to come back later, the chatbot can ask about the general issue that was encountered by the users while waiting for the queue slot to be available. This might be helpful in categorizing the tickets, such as which issue that was occurring at a certain time, whether it was payment validation, invalid promo, delayed top-up, or anything else. With this information, we can gain insights to follow up on the issue and get back to the users immediately.

Aligned with Tokopedia’s DNA, Focus on Consumer, we care to bring an excellent service for our users. Customer feedback is not something to be neglected. Conversely, it is a valuable input to understand customer’s needs and to be further translated as product improvement. Along with that, data and technology are powerful resources to enable a more advanced, personalized, and rapid improvement to boost customer satisfaction.

Reference:

¹Wirtz, J., & Lovelock, C. (2016). Services Marketing: People, Technology, Strategy (8th ed.). World Scientific.

Data analytics for business, supply chain, and marketing.