So you’re launching (or evaluating) a Progressive Web App (PWA) – that’s exciting! I imagine you’re going to be looking for performance improvements, as well as boosts across your user metrics like revenue per user and user conversion rate.
Proving the impact of your PWA can answer important questions such as: Which device is the PWA having the biggest impact on? Where in the user journey is it having the biggest impact? Answering these questions and measuring the success of your PWA investment requires a data and analytics implementation that you can trust.
We look at this as a two step process. First, determine your key metrics for success and ensure that you’re tracking whatever is needed to calculate these metrics. Second, make sure everything is tracking as expected once your PWA has gone live. Here we’ll talk through what to consider when gearing up to launch and measure your PWA, as well as how to spot, diagnose, and fix common analytics tracking issues post launch.
Setting Up Your PWA Analytics for Success
Before even launching your PWA, spending some time thinking about what you would like to track is a valuable exercise because implementing a PWA can have an impact across the full user journey! In our experience, we’ve seen an impact across all of these metrics:
- Bounce rates
- PLP/PDP view rates
- Add to cart rates
- Checkout rates
- Checkout completion rates
- Conversion rates
- Revenue per user
- Sessions per user
- Average Session duration
- Return rate
- Initial and subsequent page loads
If certain metrics are important to you, ensure you’re tracking the necessary events to calculate them. For example, if you are wanting to look at continuation rates for your core user journey as a measure of success, then you need to either send events for each part of the journey to your analytics engine (PDP view, add to cart, checkout start, purchase), or you need to send pageview events categorized by the appropriate page types.
Session vs. User-Based Metrics
You should also determine how you’re going to look at certain metrics, in general, you have a choice between session and user-based metrics. At Mobify, we recommend you focus on user-based metrics and supplement those with session-based metrics. This is largely because we observe user behavior changes with PWA launches, such as, decreased sessions per user along with decreased sessions to transaction (i.e. the number of sessions before a user purchases). This ultimately shows an increase to user conversion rate and revenue per user. However, the opposite can be true where we observe an increase in sessions per user. This will show a lower session conversion rate, but more users may actually be purchasing so user conversion rate is up! Looking at both session and user based metrics is a good practice to ensure you’re seeing the full picture. Additionally, if you choose to run side-by-side or an A/B test of your original site compared to the PWA, the test will be split by users to ensure a continuous experience for each. Since the test will be split by user, you should evaluate by user for continuity and clarity.
If you choose to evaluate the impact of your PWA at a user level, then you will need to ensure that your data can be pulled at user level granularity. This will require that a custom dimension or eVar (Adobe analytics custom variable) containing the client ID associated with a user’s cookie is sent to your analytics implementation. On Google Analytics in particular, this requires a custom implementation. We have had to do this many times, so here’s a free code snippet to do your own implementation:
Ensuring Clean Data Post-Launch: Catching the Gotchas
There are a number of analytics issues that can occur with the launch of a PWA; we’ve listed them all here along with the technical solutions. Preventing these issues will give you confidence in your data and enable you to measure the amazing results!
Session Splitting (only relevant to Google Analytics)
As a user interacts with your PWA, they will generate a number of distinct events that are sent to your analytics platform. Your analytics platform will attempt to group these events into sessions to help you analyze the user’s experience on your site. The analytics platform cannot tell if a single event is a part of the same session as another, so it uses other indicators to group events together into sessions. If these indicators are not correct, session inflation can occur.
Session inflation can be caught and pinpointed through analysis, although some different methods may be necessary depending on the scope of the tracking issue. Session inflation will impact all session-based metrics such as: bounce rate, conversion rate, session duration, sessions per user, return rate, and exit rate. There will appear to be more returning users and a decrease in the overall conversion rate.
When session splitting is occurring across the site, it can be easy to spot by looking at top level metrics on the UI of your analytics implementation and using historical context to compare those to expected metrics. You’ll observe bounce rates, session duration and conversion rates plummeting, and sessions per user increasing starkly. However, if session splitting is occurring on a subset of data (such as sessions from a particular channel) you may not observe large changes in your top level metrics, but you can pinpoint session splitting on a smaller scale by checking the impacted metrics in different segments. Segmenting by medium or channel, page type, top pages, and exit pages are good places to start. Beginning with this type of analysis can be useful to pinpoint areas where the issue is most likely to occur and help to narrow the search for this issue in the code.
How to Prevent Session Inflation
We most commonly see session inflation when tracking with Google Analytics. The location field in Google Analytics is used to determine a new session and it’s set based on the document.location when the user first loads the PWA. Changes to this field can cause Google Analytics to treat events as if they are in different sessions, causing session inflation.
In order to avoid this issue, you must do the following:
- Add a new field called page and update its value on every soft navigation. This allows you to track where the user is on your site without using the location field.
- Keep the value of the location field the same throughout all navigations.
These requirements need to be implemented for all events sent to GA, not just pageview events. If any events have the page or location fields set incorrectly, you may encounter session splitting. For more information, you can read the Google Analytics Single Page Application Tracking guide.
Events are one of the main ways that analytics platforms track the user’s progress through the site. A missing event tracking issue is not uncommon and can have a widespread impact on the data you’re observing in your analytics UI. Pageview events are most commonly the culprit of a missing event tracking issue, but it could also be add to cart, checkout start, product view, or transaction events (etc.).
This type of tracking issue can be somewhat harder to spot in the data depending on how widespread it is. If the event is not being sent at all, the issue is easy to spot at a high level. In the case of a smaller scale tracking issue, the missing events will manifest as what appears to be poor performance on that particular metric calculated from that event count.
Let’s use missing add to cart events as an example. We would expect a PWA to have a positive impact on user engagement, so if we see add to cart drop then that could indicate a tracking issue. This should be analyzed through an AB/ test, causal impact analysis, or year-over-year – not in pre/post analysis since seasonal impacts can be significant.
Events that cause a hard navigation on the original site but not on the PWA can often be the source of missing events. A hard navigation occurs when the browser reloads the entire page. In contrast, a soft navigation occurs when the URL and the content on the page changes, but the entire page is not reloaded. Almost all navigations on a PWA are soft navigations – that’s what makes the experience so much faster.
How to Fix Missing Events
Let’s look at the product listing page for an example. On the original site, when we want to see the second page of products, we click a link and navigate to an entirely new page. This is a hard navigation. When that hard navigation occurs, the analytics script on the desktop site will fire, tracking a new event.
On the PWA, we might use the LazyLoader component to load more products as the user scrolls down the page instead. In this case, no hard navigation occurs, so no event is automatically tracked.
To fix missing events, identify where they’re missing and then send them manually. In this example, we can fix the issue by sending an event when new products are added to the page.
Event duplication can impact metrics similarly to missing events. Common offenders are pageview events and add to cart events, but it can occur with any event being sent to the analytics platform. To spot this type of tracking error using data analysis, you can use a combination of techniques from session splitting and missing events.
If the event duplication issue is widespread, it will be obvious in top level metrics that use that event. With pageview duplication for example, there will be a large drop in bounce rate and an increase in pages per user, it may also impact transactions and product views. With a smaller scale event duplication, we could see better than expected performance (the opposite of what happens with missing events). In this case, it can be easy to ignore event duplication since the results are positive giving little reason to look for a data issue, but some small checks can be done to ensure you’re looking at data that you can trust.
One option could be looking at the user funnel for places where there’s a much higher continuation rate than followed by a higher drop-off rate. If this happens for one particular event, say add to cart, this could indicate a duplication of add to cart events.
Another option would be to look at the observed lift across user funnel metrics. It is common to see the conversion rate lift reflected earlier in the user funnel to a similar magnitude. If there’s a 20% lift in the user conversion rate, then we should see approximately the same lift for product view rate, add to cart rate, and checkout rate. If any of these events are showing much higher improvement then that particular event should be investigated for potential duplication. Alternatively, a much lower conversion would indicate a possible event duplication with transactions.
How to Avoid Event Duplication
There’s no single cause for duplicated events, so you’ll need to do some digging to find the issue here. Once you know which events are duplicated, identify all of the places in your code where this event is being called. This might be happening in your analytics connector or within a tag manager such as Google Tag Manager or Tealium. If you see an event being called in more than one place, it might be a red flag for a duplicated event.
For example, say you have a project where you are using both Google Analytics (GA) and Google Tag Manager (GTM). Both of these analytics tools are integrated using an analytics connector. When a pageview occurs, the GA connector sends a pageview to GA, and the GTM connector sends a pageview event to GTM. GTM will then forward this event onto the tags it is managing. If GTM also sends that pageview event to GA, you have just gotten a duplicate pageview event. For this reason, we recommend using either the GA connector or the GTM connector, but not both at the same time.
Best of Luck!
Although there are some common gotchas to look out for, we hope that with these tips you can confidently rely on your data to measure the impact of your PWA and validate continued iterations for a truly fast and immersive user experience. Get in touch if you’re interested in learning more about launching a PWA and common gotchas to be aware of.