In 2019, I organized an audit to investigate an internal oversight in my team's process. Our internal SLA improved by 25%.

Background - Why No One Cared

As a Product Expert (PE), I partnered with our Product, Usability, and Engineering teams to resolve and proactively prevent gaps in the product experience, using feedback raised by both customer-facing teams and customers directly. A primary channel that we monitored and tracked the health of our users was through Support tickets and internal Jiras.

As individual contributors, we were deep in our own trenches fighting fires with Jiras everyday. Because of this heavy volume of work to tackle, a lot of us focused most of our attention on preventing issues in our product. Naturally, what unites and excites our team the most were conversations around monumental feature releases or product overhauls. Our mindset and siloed nature meant that no one was questioning if the our team is adequately addressing existing issues in our Jiras.

Problem - Neglected Jiras

Personally, I had not been as concerned about our overall team, but we were growing rapidly and cracks eventually showed up. One pattern stood out - we were receiving more feedback from the customer-facing teams that they were not getting timely updates on Jiras. Sensing a problem, I looked into our team metrics and found that we were behind our Internal Jira Service-Level Agreement (SLA) by about 30%. 

Through experience alone, I might have attributed the SLA failure to Jiras that needed more time to resolve because of their complexity.

However, based on what we had been hearing at that time, I formed the following hypothesis:

Most Jiras failed the SLA because they had been neglected. 

 

There will be room for the SLA attainment to improve if this hypothesis is true. Success could then be defined as these outcomes:

• Neglected Jiras make up a smaller proportion of Jiras that failed the SLA.
• SLA recovers to a range closer to our target SLA

Approach

1. Goals

• Find out why Jiras took longer than X days to be resolved.
  • Output 1: Create data labels (of the reasons behind the SLA failures)
  • Output 2: A labeled dataset showing the distribution of SLA failures by their causes. As an added benefit, the data labels can be re-used in the future.
    
    

2. Constraints

• Large Dataset: For qualitative data, 500 tickets will take a long time to get through.
• Hard to Sort Insights: A mass of qualitative data is hard to organize.
  
  

3. Affinity Diagram - Synthesize Qualitative Inputs

Based on my goals and constraints, the Affinity Diagram workshop was a natural next step to sort through qualitative data and to collaborate effectively.

The affected Jiras totaled 500 over the past 30 days. I reduced this number by focusing on Jira tickets belonging to APAC customers, which came up to 145 Jira tickets - I thought we should lead by example first before suggesting actions to our global team.

I then enlisted the help of my immediate team in APAC. With the 4 of us, we each had to sort through about 36 tickets for an Affinity Diagram workshop.

4. Improvised Cart Sort

The output of the Affinity Diagram was 15 different categories that can be used as data labels. Judging from experience, having fewer labels to learn will make our lives easier in the latter step. 

I reviewed the sub-categories and noticed a cause-and-effect relationship between some sub-categories. 

This observation led me to improvise the card sorting method to organize sub-categories into statements about the root causes of each issue, which further reduced the sub-categories from 15 to 10. 

Outcome

The Hypothesis was Correct 

Going back to label the dataset with our newly created labels validated my hypothesis. Jiras were getting neglected. No matter whose fault it was, me and my peers were responsible for directing efforts in closing out the gaps in our product experience.

Looking back, these solutions looked obvious but the small team size I started with had very different needs back then. This problem signaled a need to scale the team. We worked with management to share the findings and execute the roll-out, which were based on 2 areas:

• Enable the Team - Guides for adding filters to dashboards in Jira and other reporting tools were created for existing team members and for onboarding training. A follow-up action framework was also shared for cleaning up our queue.
• Proactive Monitoring - Notifications to individual PEs and managers were also created for inactive Jiras across various internal channels. 

After one month, we achieved being 5% within target of our SLA, and held steady ever since.

Appendix

Credits:

• Los angeles icons created by Freepik - Flaticon
• Silo icons created by Freepik - Flaticon