A+ web performance is essential for any retailers who want to compete in today’s digital world, but even the best retailers are struggling in the quest to measure their web performance. And even if the measuring is going well, they are often struggling to actually make the improvements that will have the biggest impact.
Part of this conundrum is understanding the types of improvements that will satisfy your shoppers, your web team, and your bottom line – and what metrics, tests, and tools are best to measure them.
Metric Types and Tests
A quick breakdown of metric types and measurement techniques
The software for implementing Usability Tests isn’t great, so Usability Tests today require getting your UX researchers in a room with shoppers to measure and record what they’re doing. This is useful to do, but a bit of a lagging indicator of performance. Luckily there are more efficient tests that will help you understand how your site is performing.
When I think about technical timing metrics, I think about two ways to measure and evaluate them. One I call a Lab Test, and the other I call a Field Test.
Lab tests are all about creating a controlled environment, and running a test in a really specific circumstance. By controlling for several different factors you can get a really close eye on what’s actually impacting your performance. The metrics you typically measure in Lab Tests are first meaningful paint, time-to-interactive, and last painted hero.
Lab Tests are also called Synthetic Tests, because they’re not designed to test real-world conditions. But they are designed to give you good, quick insight into the kinds of performance your users are seeing.
What we’re trying to understand with Field Tests is whether there are any outliers in performance. We’re measuring similar metrics as with Lab Tests – the difference is that instead of running them with control over as many variables as possible, Field Tests are run on the devices of actual shoppers.
Often what we find is that our Lab Tests missed some specific conditions that are present with shoppers out in the wild. For example, we might have customers using devices we never imagined, and those devices might have a different performance profile, because they have different memory, CPU, or network conditions than we accounted for in the Lab Tests.
Field Testing is going to allow us to understand how performance is distributed across our entire userset. The real challenge with Field Tests is that you’re left sorting through a massive amount of data. Understanding the signal from the noise can be more than a little challenging.
The Best Tools for Measuring Web Performance
Lighthouse is a tool from Google that’s gained a lot of popularity over the last few years. It falls into the Lab Test category. It’s great for ensuring websites are adhering to the best practices that users respond to the most.
With Lighthouse you get scores across a number of areas, and what’s great is that you can dive deeper into each of them.
When we dig into the performance category we can see things like first contentful paint, first meaningful paint, time-to-interactive, first CPU and more. You’ll also get a list of recommendations on ways you can improve your performance.
One of the best things about Lighthouse is that it’s built right into the Chrome browser. In general, it’s a great tool for helping you understand your site. I particularly like this tool because it’s recent and new, which means it’s attune with modern performance practices and it’s actively updated and debugged. If you haven’t tried it, definitely give it a shot.
Web Page Test
Web Page Test definitely has fewer designers working on it’s interface, but I want to give it a shout out because it’s an incredibly useful tool for developers to be able to understand the profile of their site from a performance perspective.
The UI might be slightly outdated, but it’s still a great tool.
When you run Web Page Test on your site you’re going to get back a bunch of metrics (including some of the same ones as with Lighthouse) but there’s a couple of things that make it different.
Web Page Test is easier to share with your colleagues than Lighthouse results, which is great for getting on the same page as far as performance goes. Web Page Test is also considerably more configurable than Lighthouse, which can be really useful, but they are both great examples of Lab Tests.
Chrome User Experience Report
When we think about Field Tests, we want to be able to understand our performance across all of our different users. Field testing is sometimes called Real User Monitoring – and there’s lot of vendors out there who have great proprietary solutions to approaching this – but the tool I want to focus on is actually a free tool from Google called Chrome User Experience Report.
Since 2017, the Chrome browser has been configured to send a bunch of telemetry data back to Google, who’s made a subset of that data available to developers (and others) in order to better understand the performance of their sites across many devices. They’ve also built a number of tools on top of it, to make it more accessible to anyone just getting started with performance.
One of the features that’s recently been updated and configured is PageSpeed. This new version is relatively recently launched to replace the old Lab Testing version of this tool, and now uses insights from all Google Chrome users.
When we enter a particular URL or site into this tool, we get two metrics: first contentful paint and DOM content loaded. We can also see that our speed results are classified (average, slow, etc.).
Google Speed Scorecard
This tool is great for quickly benchmarking the median speed of your site against competitors because you can add other domains to compare against.
If you’re in athletics, for example, you might compare against, Nike, Adidas, and Under Armour. You can also filter by region and network conditions. These bits of field testing data can be really useful in addition to your Lab tests.
Picking the Right Test
As you’re doing more testing and making improvements, keep in mind that Lab Tests are the best for preventing teams from releasing changes that could introduce performance regression. Field Tests are the best for measuring what actual shoppers are doing out in the wild.
To help you understand more about measuring and improving your web performance, I created this 55-minute webinar. It takes a deeper dive into everything you’ve read today, and expands on performance best practices, customer perceptions of performance, and specific technologies you can add to your stack to improve your mobile web performance.