Context
What is Plotline?
Plotline is a SaaS product that enables users to publish tooltips, modals, and bottom sheets (collectively known as in-app engagement) without requiring developer involvement
What is in-app engagement?
Typically, these would be onboarding journeys for users or driving new feature adoption. These campaigns could have tooltips, modals, embeds - or any combination of those.
What was my role?
Re-designing the layout and data visualization to address issues with comprehension and consumption of campaign data, along with my fellow designer and the engineering team.
The previous dashboard - more difficult to consume data and no visual preview of the campaign
The redesigned dashboard
The Problem
What was the problem?
Initially - the time we spent explaining the metrics to users on call or Slack.
Over time - lack of confidence in campaign performances, since users didn't understand how metrics were calculated.
Why was this important?
Lack of confidence in campaign performance caused a lack of confidence in the product and dipping CSAT (Customer Satisfaction) scores.
This also threatened our Net Revenue Retention (NRR) on our path to $1M in ARR.
TL;DR
Users assumed their campaigns weren't peforming well despite performing well in reality, which led to dissatisfaction with Plotline. Essentially, we did a poor job of explaining and representing the metrics.
Goal-Setting
Since difficulty in understanding the data and lack of trust in the data were the key pillars of the problem, we articulated the goal or guiding principle as follows:
How might we make the data easier to consume and inspire confidence in campaign performance for users?
The Process
The process I followed is roughly summarized below. It doesn't come close to capturing the chaos and despair of having to convince the engineering team to build the feature though.
Research and Feedback
A collection of the insights we had gathered from scattered interviews for the Dashboard overhaul
primary users
Our primary users were product managers and growth managers who:
responsible for growth metrics such as feature adoption and user conversion
using Plotline for in-app nudge campaigns
looking to find obvious insights in data - wins and losses in impact metrics
looking to easily share info about winning campaigns with their stakeholders
Research for the project came from continuous discovery with users instead of a dedicated subproject, though largely speaking these were semi-structured interviews over Google Meet.
key observations
App Opens, Eligible Users, Target Users, Clicks and CTRs were all part of a funnel but the existing implementation obscured this relationship.
Users had a strong preference for visual explanations over text-based explanations due to limited mental bandwidth in their work.
Growth managers were used to certain standards such as bar charts with color coding for conversions and dropoffs that were being violated by use of a line chart.
Final Design Output & Decisions
1 | Redesigning funnel metrics, temporal views and Unique numbers
observations
Existing implementation obscured the funnel relationship between App Opens, Eligible Users, Target Users, Clicks and CTRs. Also, some metrics were accessible only via CSVs.
decisions
Used Gestalt principles to establish a visual relationship (left to right - like timelines) and gave more importance to Unique users - what users ultimately looked for - over Total counts.
2 | Explaining Key Metrics
observations
decisions
Moved explanations into the product and represented metrics with a Sankey diagram of a hypothetical campaign, establishing relationship to other metrics.
Users had to either search for the answer on the Help Center - or contact us
The current version - explanations sit within the product to not break users' flow
Metric definitions - Expanded
3 | Linking Impact Metrics
Impact Metrics were terminal goal events used to measure impact between control and target users at the end of campaigns.
observations
Impact metrics were hard to scan in one pass. Also, users were more interested in impact percentage in control vs. test than completion rates or journeys triggered.
decisions
Visualizations were made more cohesive and intuitive with color encoding of positive vs. negative impact as well auto-sorting in decreasing order of impact.
4 | Visualization of Stepwise Dropoffs
Stepwise drop-offs showed the percentages of conversions and drop-off in campaigns that utilized multiple UI nudges to guide users.
observations
Users were used to bar chart repesentations of conversion funnels and struggled to grasp relative magnitudes in a line chart.
Data points were titled with the step number, but users wanted to know which UI element was used for that particular step.
decisions
Drop-offs are now visualized as stacked bar charts. Also, followed a convention Growth Marketers were used to on tools such as Clevertap, Mixpanel, etc.
Both conversion and drop-off have separate, selective details on hover, including the UI nudge type used.
The previous version of the explanations - for which users had to go digging or message use on Slack.
5 | Easier Access to Campaign Preview
Campaign Previews gave users an idea of the content of the nudge campaigns and the types of UI elements that were used.
observations
Users could not check which UI nudges were used for the campaign or a particular step from the conversion funnel without going into Edit mode
decisions
Campaign Preview was incorporated into the dashboard to prioritize recognition over recall. Users were now able to click through all steps and check them against conversion funnel.
Impact of the Project
The redesign was integral to Plotline’s retention of some of its biggest clients and conversion of new ones.
I focused on keeping design fundamentals front and centre, to increase trust and reduce calls for explanation of metrics.
Summarizing, we cut down the time taken to access the performance details and content of a campaign, reduced time spent by Plotline explaining the funnel metrics while increasing satisfaction and trust of our users.
Boosted Customer Satisfaction (CSAT) from 7.2 to 9.1 (Arrested a downtrend and flipped to an uptrend with most users being promoters)
AT LEAST 5 hours per week per engineer saved in time spent explaining metrics over calls or sending CSVs to client teams
Prevented any reduction in Net Revenue Retention (NRR) or churn of our biggest clients well before the dashboard became a universal issue
A learning from the project - local components to speed up future changes
