Onboarding A/B test increases tracking number creation by 25% and increases retention for 90 day user cohort
Duration: 1 month| Role: UX Designer |Team: Josephina Kaiser (UI design), Pia Kendrick (PM)
Tools used: Figma, Figjam | Methods: Lo-fidelity wireframes, A/B testing, rapid iterations,
Problem
The business had noticed there had been a downward trend in customers creating source tracking numbers. Source tracking numbers are more profitable for the company because users are buying numbers a-la-carte, which increases variable monthly usage costs and there was a correlation in users with more source tracking numbers having reduced churn rates. The alternate to source tracking numbers are website pools, which have a steep learning curve to adopt. With this project, we wanted to help customers through the onboarding flow to create more source tracking numbers, identify Callrail’s value proposition simply, and grasp call tracking faster.
We were implementing an A/B test to test our hypothesis of:
Would removing the option to create a website pool and replacing it with the ability to create source trackers lead to a cohort of users who end up churning less?
Purpose
We have an opportunity to improve the businesses’ usage and churn metrics. Historically, the app has encouraged users to create a website pool as a way to dynamically track visitors to their site. Although this is a differentiator for us, we found a majority of SMB users needed 1:1 support help to achieve this functionality, which means website pools added friction in users setting up. To get all users (tech savvy to beginner) quicker time to value on their marketing attribution goals, I needed to simplify the requirements to complete the onboarding and get users their first lead attributed to illustrate the value.
Challenges
Aligning on an inclusive source tracker list
CallRail serves a diverse breadth of industries, like healthcare, home services, legal, and real estate. Each industry has different advertising platforms that are most relevant to them. How might we show a limited, but relevant list of options to all the industries while being mindful of decision fatigue (Hick’s law)?
Making a lasting impression on a wide range of users
This setup workflow is the first glance into CallRail’s in-app experience and it serves as an unboxing of the app. There is a lot of opportunity to showcase how well CallRail can accomplish the user’s job to be done. How might we address user’s expectations and show that CallRail meets them in the onboarding?
Discover
The UX team utilizes FullStory, a user research tool, to glean insights on completed funnels, heat maps, and user engagement with features. I made segments to track the data on our current onboarding. Here are the highlights:
60%/40% of users completing the first-run through desktop/mobile. The first-run was not mobile responsive.
19.5% of users drop-off the flow in the first step where we ask them what type of tracker they want to create.
15% of users who completed the workflow went to create another number. Users who created another number had a lower churn rate in 90 days.
In the audit of the workflow content, there are missed opportunities for smart defaults to be used. There is also a high click interaction cost by the grouping of the questions.
Takeaways:
How might we be more reassuring, clear, and remove jargon in the first-run workflow to resonate with the variety of technical knowledge user’s possess?
How might we illustrate CallRail’s lead tracking value in one fell swoop?
Assure the flow is mobile responsive
User flow chart
Activate and Finish were confusing to the user
User empathy comic strip
Ideation
Since this would be dreaming up new UI patterns and illustrations, I collaborated with our UI systems designer from beginning to end. After we gathered user research and aligned with our project manager on scope and what we would not be doing, we got to lo-fi sketches to quickly brainstorm different approaches.
In these lo-fi sketches, I am prioritizing 2 jobs to be done on the first screen.
How might we show users the list of ad platforms they can use to track their sources?
How might we give users an intuitive affordance to suggest they can create more than one tracking source?
Pro - Clear content
Con - When user adds a couple numbers, the scroll depth increases.
Pro - Intuitive affordance to encourage users to add more than 1 tracking number
Con - We are asking users to name a number they might not have yet, which introduces a cognitive gap in the sequence of getting a tracking number
Pro - Reduces click affordance by showing all options at a bird’s eye. Helps with recognition and recall by showing logos
Con - We had not aligned on what the final list of advertisement places would be yet. Depending on the list, we would not need the filtering options this design supports.
✅ Approved lo-fi flow
Exploration
After we gained alignment from the project manager, lead UX designer, and development team on the lo-fidelity approach of the entire onboarding number creation workflow, we transitioned our design to mid-fidelity to make more concrete decisions to the experience. This part of the process would fill the content and UI gaps to the design and allow us to get detailed feedback from our team versus strategic.
Following the example of the first screen, this is how we explored the mid-fidelity design starting from the lo-fi
Callouts :
When we aligned on the final list of the advertisement locations, we decided it wasn’t extensive enough to break it out into categories. We decided to order the list from online to offline to help users understand the tracking tool’s capabilities.
We addressed the potential for users who don’t see their advertisement location by adding an option for “Other” and giving a text tooltip at the bottom as a tradeoff to having a limited number of sources to choose from.
Design and prototype
After I finalized the UI and content decisions and documented feedback on why or why not we would be addressing certain feedback, I made a hi-fidelity prototype for the engineers and product department to reference. Below are some callouts to design choices made.
Limit users to 5 selections
We added a disabled variant to the existing selection card pattern. This is clear UI feedback to tell users they are at the selection limit.
Smart defaults and global settings
To help users make less decisions, we auto-populate the number name field with the type of number selected
4 steps to 3 steps
The original workflow had 4 steps. This workflow combines the “Activate” and “Finish” step into one for seamless clarity on what they completed.
Conclusion
This was an exciting project to work on because the excitement was in “lets ship to learn”. We had a hypothesis that there is a strong correlation of decreased source tracking numbers were causing churn and decreased usage, so we investigated into a solution for it that could be proven wrong through A/B testing!
How did it do? Success metrics:
Mobile version addressed negative reviews from home service and real estate investors. This project served as a good advocation to consider mobile design
Users bought 25% more tracking numbers and increased revenue
Onboarding completion rate went up by 15%
Users churned 8% less