As the first step in improving the UX Debt Process, I led exploratory research to understand how Product Designers work through UX Debt.
Context - A Resurgent Pile of Debt
The UX/Design team in HubSpot has a few main channels of discovering UX debt or usability issues, a significant one being UX Jira. Originally created to combat UX debt that had accumulated over the years, the initiative managed to resolve 80% of all UX debt when the company committed itself to fixing existing issues instead of creating new features for the whole of 2019.
It was expected that UX debt would accumulate again after the one-off clearance. However, 2 years later, the customer-facing teams at HubSpot reported hearing more frustration from their customers. They also noticed that some pain points were affecting a lot of customers.
A preliminary look into the data suggested a resurgence of UX debt that would become unsustainable:
- 83% of existing UX Jiras remain untouched, of which:
- High severity ignored: 4% (18) had more than 6 customer reports (a high number in HubSpot) each. The highest frequency of occurrence was 18 reports.
- Slow lead time: 40% were created more than 90 days ago but were not updated.
- Resolved UX Jiras down 50% YoY - In 2020, 1033 UX Jiras were resolved compared to 569 in 2021.
- Resolution time climbed substantially - Starting January 2021, resolution time exceeded 100 days.
I wrote and shared an internal memo. Together with the the co-workers who were interested, we formed a task force comprising of 2 peer-level individual contributors, and 2 managers acting as leadership sponsor.
Discovery - UX Leadership Priorities
As a task force leading the charge for a Support-UX program, we were keenly aware that we needed the buy-in of the UX team early on. The managers spoke to various UX leaders. While they were open to hearing more about our findings in the future, they would not be able to devote resources to help us investigate further because they did not think the problem warranted attention for now. 2 reasons stood out:
- UX Leadership Priorities - The focus of the UX team was on growing the team and advocating UX within the organization. E.g. They needed to hire enough people to have 1 Product Designer for each product team.
- "Healthy" SLA - On the surface, it looked as if the UX team had near-perfect attainment for SLA. In reality, the SLA will only start counting when an issue has been worked on, meaning the SLA failed as a warning indicator.
Problem - Not Understanding the User
Without someone from UX team working in our task force, we realized we were throwing around assumptions about how product designers work. We could gather that designers did not like working with UX Jiras but could not understand why.
Without a solid foundation of our users, I directed the task force to our true immediate problem:
We cannot improve the UX Jira experience for product designers because we do not understand how they work through UX debt.
Approach
1. Goals
- Understand how product designers work through UX debt
- Understand why product designers do not use UX Jira
2. Constraint - Our Users are UXers
We had to be mindful that our users are professional UX designers who have conducted user research themselves. My worry was that they might play along with usability tests and walk us through steps that they think they ought to take instead of showing us what they usually do.
3. Shortlist Questions
To keep our scope in check, I organized a remote Post-up and a Forced Ranking workshop over Google Sheets:
4. Exploratory Research
Every one of us on the task force had previously spoken with designers about their UX debt process, but these talks did not help us identify the roadblocks associated with using UX Jiras. To get an accurate as-is picture of the UX team's process, I persuaded the task force to conduct user interviews that also included a short session where designers would walk us through their UX debt process.
Following feedback from the task force, I also decided to position this research as an "engagement exercise" with the UX team, in view that designers may not feel inclined to be studied as users.
- Participants: 5 Product Designers, 4 Senior Product Designers/Design Leads (the recruitment of designers were done by everyone on the task force)
- Researchers: 1 Interviewer/Moderator, 1 Scribe
- Format: Video conference over Zoom with screen sharing
- Duration: 45 mins total; Interview (15 mins) + Shadowing Session (20 mins)
- Themes:
- Attitude towards UX debt and prioritization
- Behavior when processing a UX Jira
- Behavior when monitoring UX debt
Surprisingly, our research participants were very candid and gave us a lot more attitudinal and behavioral data than we had previously gathered.
5. Synthesis - Affinity Mapping
Our team made it a point to take notes on a Miro board during each interview. One of the managers later facilitated an affinity mapping workshop to organize our insights.
Outcome
Because the UX debt process was complex, we did find some difficulty in making the process relatable to our stakeholders. Looking back, a user journey map would have been better at communicating the problems and opportunities at particular stages of the process.
After presenting the findings to the UX leaders, their conclusion was that a partnership would be on hold until the UX team could set up their operations team. In the meantime, our task force would focus on the lower-hanging fruits that were within our locus of control. These included:
- Enabling Product Experts (PEs) to support Designers on Prioritization
- Aligning UX Debt and Backlog monitoring across teams
- Conceptualization the future of UX Jiras
As a signal of the UX leaders' commitment to working on fixing the problems with us, they have mobilized their teams to catch up on UX debt. In the following quarter (Q1 2022), 119 UX Jiras were resolved, almost 80% of the total in 2021.
Appendix
Affinity Mapping was used to organize and sort the findings by themes
The next steps after the research were communicated with a timeline