Mobify > Blog

Smartphone Browser localStorage is up to 5x Faster than Native Cache (New Research)

2013-05-13T10:00:00-07:00

As a web developer you know that creating engaging web UX means quickly delivering content that captures your visitor's attention.

The storage API, including localStorage, was designed to help web developers provide more app-like behaviour for the web than cookies previously allowed.

But it turns out you can also use localStorage to make your mobile website faster!

How much faster? Well, you can load some critical path resources such as a JavaScript that's required for your UX up to 3 - 5x faster using localStorage than from the browser's native cache. Using millions of data points from Real User Monitoring (RUM) I had the following surprising results.

20% trimmed mean results of native browser cache vs localStorage performance. Mean performance for localStorage is always faster than the browser's native cache and for iOS 5 & 6, dramatically so.

What does this mean? It means you should absolutely be using localStorage for essential resources rather than depending on the browser's cache. This is a bold claim I'm happy to back with data, and I'm also very curious to hear from the community on your own test results. For more details on how I collected and analyzed my results, read on.

Background: the web community weighs in on localStorage

Web developers may have read some discussion in early 2012 when web storage APIs experienced a flurry of concern around performance.

The most concise summary of the developer community's performance concerns are expressed in Chris Heilmann's post on Mozilla Hacks. Chris's arguments hinge on four key criticisms of localStorage:

localStorage is synchronous (can block rendering)
localStorage performs file I/O (unbounded worst case performance)
Browsers load the data into memory on the first request (wastes user-agent RAM)
localStorage is persistent (if you never visit a site again, disk space is wasted)

These criticisms notwithstanding, localStorage remains attractive for mobile web developers for several reasons:

Browser native caches are shared between websites and are often small on mobile devices - as Guy Podjarny has determined, sometimes smaller than 5MB
Some browser caches are not persistent (contents lost when the app exits)
Native browser cache performance & behavior of mobile user-agents is highly variable
Latency on mobile is high - good caching is required for good performance
localStorage content eviction policy is directly under developer control, browser native cache is not

That localStorage is synchronous, performs file I/O, uses RAM & storage on user-agents is not in dispute. However, what is important is to find out what the performance is like in actual smartphone browsers executing scripts on real websites in the wild.

Experimental design

The experiment focused on the time taken to load a 25KB (uncompressed size) JavaScript payload. The content of the JavaScript payload was the Dust templating library with one small modification - after the library is loaded, a hook was executed to notify the RUM test that loading was complete so that time to load could be measured accurately. There were two experimental conditions, described below. Data collected included the user-agent, condition & the timing result.

Condition 1: browser native cache performance

The objective of condition 1 was to benchmark performance of the browser native cache on smartphones. In this condition, the JS payload is loaded twice, the first time to populate the browser cache and the second to measure timing for instantiating a script from browser cache. The payload is delivered with both a one month future EXPIRES and CACHE-CONTROL: max-age=2592000 headers set (if you're not familiar with cache control headers, or want a refresh, this is a great introduction). In this condition the content of the payload is set using the src= attribute of a script tag.

Condition 2: localStorage performance

The objective of condition 2 was to benchmark performance of localStorage on smartphones. In this condition, the JS payload is first written to localStorage after being JSON-encoded. After the payload is stored, I read back the content of localStorage and measure the time taken for the JavaScript contained in the payload to execute the callback hook. In this condition the content of the payload is inserted inside a script tag as a textnode.

Experiment validity threats

The most significant threat to the validity of this experiment is that in both conditions I am retrieving the data immediately after writing it. A subsequent test might involve reading data several days later when other browsing activity has taken place. However -- this modified experiment is unlikely to favor the native cache as it is much more likely that content will be evicted from the native cache after several days of user browsing activity. localStorage does not suffer from this eviction problem. In this respect, I believe these results understate the performance improvement available from using localStorage.

The second validity threat is that performance may vary if localStorage is fully utilized. Will Chan measured desktop localStorage performance for relatively small storage sizes - and saw very good results on desktop as well. What's interesting is that he found that performance degrades considerably as the amount of data stored in localStorage in desktop Chrome increases. More testing is required to determine the impact of a heavily utilized localStorage cache on mobile.

Results

For data analysis, I applied a 20% trimmed mean to both native browser cache and localStorage conditions discarding 10% of the lowest and highest measured results. This was done to reduce the sensitivity of the results to outliers and is considered a robust estimator.

This candlestick plot shows the results of my performance test. For each browser type and condition, I plot the mean (red mark), the 95% confidence interval (dark gray box), and the min/max values (whiskers).

The figure above summarizes my results. As you can see the mean performance of localStorage is better than browser native cache behaviour in all cases. In many notable cases, such as iOS 5 and iOS 6, localStorage outperforms mean native cache performance by a factor of 5. Summary values are in the table below.

Smartphone OS	localStorage Mean (ms)	native browser Mean (ms)	localstorage StdDev	native browser StdDev
Android 2.2	299.94	509.32	164.76	537.95
Android 2.3	133.50	154.76	57.65	215.13
Android 4.0	97.05	103.44	31.80	179.36
Android 4.1	75.60	85.57	21.60	132.03
Android 4.2	55.28	83.76	23.28	121.51
iPhone iOS5	21.61	132.90	8.64	177.69
iPhone iOS6	15.71	73.86	7.27	109.32

Variance in localStorage performance is much smaller than browser native cache variance. This can be seen by the much smaller confidence interval present in the localStorage condition. It's also readily apparent in the standard deviation column of the table above. This is especially true for Android 2.2 devices where native cache behaviour varies wildly, but the effect is present in all devices, including newer smartphones running both the iOS and Android operating systems.

On Android the minimum time (bottom whisker on the candlestick plot) measured for native cache performance beats the minimum time from localStorage. This suggests that under ideal circumstances the native cache is faster, but my real world tests suggest these ideal circumstances are the exception rather than the rule and mediocre or worst-case behavior from the native cache is common. On iOS 5 and iOS 6 the minimum load time for localStorage and native browser behaviour are nearly identical, but mean time for localStorage is significantly faster.

For completeness, I've included in the appendix the histogram data from each of the OS versions measured for both browser native cache and localStorage conditions. Features of note are described in the captions. These graphs provide more insight into the mediocre & worst-case behavior of native caches and localStorage.

Conclusion

localStorage is an attractive alternative to the mobile browser's native cache for critical page resources with much better average-case performance (2-5x better for recent OS releases) than the native cache. Because web developers have explicit control over the eviction strategy from localStorage, key resources can be preserved or updated in a more optimal way than is possible with generic native browser caching strategies.

Final caveats

There are very significant performance gains to be realized by using localStorage. Keep in mind, however, that there are also tradeoffs when using localStorage.

localStorage space is relatively small, 5MB on most browsers, and it can only store string data. This is effectively halved because localStorage stores strings as double-byte characters (UTF-16).
localStorage reads and writes do block the rendering thread. Where possible reads/writes should be done after initial page rendering is complete. You may want to avoid operations that are likely to cause performance problems such as a very large numbers of operations (thousands or tens of thousands of reads or writes), as well as read/write operations larger than 1MB.
Browsers should do a better job of exposing storage space used by websites taking advantage of the web storage APIs to smartphone users. Be a good neighbour and don't use localStorage recklessly, use it for key resources that are in your critical path.
You'll need to manage your website's cache eviction & resource updates to localStorage manually.

Your Turn

Do you use localStorage API in your work? What do you use it for and how does it perform?

Let me know by leaving a comment!

Appendix

Additional data plots are available in the appendix.

2 Fake Companies and a Killer Unmasked: The Story of Mobify's First Hackathon

2013-05-03T12:00:00-07:00

What do you get when you combine a hackathon with a murder mystery? The answer is 'Morbify', (probably) the world's first murder mystery hackathon.

So how did it work? Well, if you've been to a hackathon before, you'll know that they typically have a theme. After much deliberation, we decided to make ours the gamification of everyday tasks.

And if you've ever been to a murder mystery before, you'll know that they typically have a crazy story that unfolds during the event. So we went ahead and unleashed a team of our best writers onto the challenge of creating the most hackathon appropriate murder mystery ever made.

Here's what hackathon attendees knew about the event in advance:

You've just been hired by trendy video game company, Shankworks Inc. Congratulations! On the back of their wildly successful first game - StabCat - Shankworks Inc. has been on a hiring spree. You are one of 30 new employees starting at Shankworks HQ on the same day!

Eager to eclipse the success of StabCat with an even better product, all new employees have been issued a challenge - create Shankworks Inc's next big viral hit by creatively gamifying some everyday tasks. You'll work in teams, and the team with the best product by the end of the day will be showered with glory.

A sticker of Shankworks Inc's runaway hit, StabCat.

A killer case of corporate collapse

As murder mysteries tend to do, the story of 'Morbify' revealed its sordid twists and turns in a series of staged events.

It began over drinks at EXP Bar, where 'new employees' (read: hackathon participants) met the CEO of Shankworks Inc., played by Mobify's own CEO, Igor.

Amazing things are happening at @shankworks_inc! We're celebrating the new hire of 30 new employees! twitter.com/Shankworks_Inc…
— Shankworks Inc. (@Shankworks_Inc) March 23, 2013

Alas, the good times weren't to last. Upon arriving at Shankworks HQ in the morning, attendees soon discovered that Igor had been murdered during the night:

Sorry to get your hopes up #shankathon-ees - looks like our CEO Igor has, erm, called in sick today.
— Shankworks Inc. (@Shankworks_Inc) March 23, 2013

While the majority of time during the day was allocated to furiously gamifying everyday tasks, every hour, for five minutes, hacking came to a halt while a scene unfolded in the middle of the event space. Each scene revealed clues about the characters and explored motivations they might have had for committing murder.

High school drama lessons definitely came in handy as tempers flared, accusations were made, and tears were shed. Here, Shanksworks Inc. CTO, Jacky Sabatier, is exploding at Inspector Henckels for publicly announcing Igor's demise.

The murder mystery descended into madness throughout the day. By mid-afternoon, the official @Shankworks_Inc Twitter account had been hijacked by the killer, who had also tweeted out pictures and videos of multiple additional murders he/she committed during the day:

Ha. Good luck working for @the_doodle_inc , traitor #shedeadtoo #shankathon yfrog.us/f5ylblmkmugvde…
— Shankworks Inc. (@Shankworks_Inc) March 23, 2013

Another murder! Angelina Arthur-Price, Creator of StabCat, didn't make it through the day.

Sorry David - you guessed right. “@davidmfay: I think I know who the killer is! #shankathon”yfrog.us/mj9b3huxnqeoqb…
— Shankworks Killer (@REAL_Shankworks) March 23, 2013

A third murder! David from the Shankworks marketing team met with a sticky end.

After all the everyday tasks had been gamified, it was up to hackathon participants to identify the killer using the not-so-subtle hashtag #thekilleris. When all the guesses were submitted, the culprit was identified and apprehended on the spot:

Inspector Henckles outlining his case.

The criminal mastermind and 'Shankworks Killer', Candace Shiv, being led away to prison.

As someone who has never been to either a hackathon or a murder mystery, I can say that Morbify was a crazy, weird, and extremely enjoyable experience to be a part of. What's more, the quality of the projects that came out of the event were absolutely amazing– more details to come soon!

If you didn't get a chance to be a part of Morbify, make sure, you sign up for Mobify's next hackathon! For all the latest updates on events and going-ons around Mobify HQ, please feel free to follow us @Mobify and @MobifyCareers.

A Beginner's Guide to HTTP Cache Headers

2013-04-30T10:00:00-07:00

This article offers an exploration into HTTP caching headers and associated CDN behaviour. If you're looking to understand how caching headers fit into the modern web, or are just curious about what everyone is talking about around you, this resource is for you.

If you already understand the benefits of cache headers and are looking to extend your knowledge further, I'd recommend referring to the W3 documentation.

What Can Cache Headers Do for You?

Simply put, caching allows you to store your web assets on remote points along the way to your visitors’ browsers. Of course the browser itself also maintains an aggressive cache, which keeps clients from having to continually ask your server for a resource each time it comes up.

Cache configuration for your web traffic is critical for any performant site. If you pay for bandwidth, make revenue from an e-commerce site, or even just like keeping your reputation as a web-literate developer intact, you need to know what caching gets you and how to set it up.

In the case of assets, things like your company logo, the favicon for your site, or your core CSS files aren't likely to change from request to request, so it is safe to tell the requester to hold onto their copy of the asset for a while. If your visitors were small children in the backseat asking, "Are we there yet?" repeatedly, this would be like telling them, "No, and we won't be for 20 more minutes, so remember my answer."

By cutting down on the requests your server has to deal with, you are able to handle more requests, and your users will enjoy a faster browsing experience. Generally, assets like images, JavaScript files, and style-sheets can all be cached fairly heavily, while assets that are dynamically generated, like dashboards, forums, or many types of web-applications, benefit less, if at all. If your concern is performance, your dynamic content will be shunted to a bare minimum of AJAX-type resources, while the rest of your assets will be heavily cached.

For Clients & CDNs

Historically, cache settings were all about the client's browser, so we shouldn't forget the benefits of properly considering the client and the way it keeps assets around during a session and on return visits. However, these days, with the advent of Content Delivery Networks (CDNs), a bigger concern is how caching is handled on the intermediary points of web traffic.

What are CDNs?

In case you don’t want to read the wiki article , I'll give you the crib notes here. Essentially, CDNs are servers, (note the plural), that sit between your end user and your server. Each of these servers are designed to cache your content according to the cache rules you set in the various HTTP headers outlined below.

When configured properly, CDNs will deliver content to your end user from the fastest, (and typically closest) server available. Additionally, CDNs act as a buffer between you and your users. The number we are most concerned with is the cache hit ratio, which describes the percentage of requests the CDN was able to answer out of its own cache without having to bother our servers. Depending on your traffic and architecture, this number can get well into the high nineties, although even at lower figures you'll experience a gain. (Keep in mind that a low frequency of requests will mean a larger percentage of requests go back to your server, so the ratio is only meaningful when understood together with your cache times and the general load of your site.) A word of warning though; if you just setup your cache and fail to configure your caching headers correctly, it's quite possible to end up paying twice for every request.

A basic flow of data through a CDN. Your webservers provide content to the CDN intermediary servers, which have Points of Presence in various geographic regions. Lazy network diagram makers will usually just draw a cloud here.

Beyond their intended use for caching, CDNs also have a pleasant side-effect; provided you're dealing with a website, or a particularly well-crafted web application, in the event that your servers experience a momentary outage, your CDN may be able to buffer the experience for your end users, ensuring they never even notice.

The Basic Headers

So, you know what a CDN is, and you know you want to configure your response headers to make use of them. Most web servers make it trivial to set response headers, so I'll leave it to you, Google, and the man pages to figure that part out. For now, let’s move onto what headers you should care about.

1. cache-control

If there were a default super-header for caching behaviour, this would be it. Typically you will see a string of settings for this header along the lines of:

cache-control: private, max-age=0, no-cache

These settings are called cache response directives, and are as follows:

private | public

Essentially they let intermediary caches know that a given response is specific to the end user and should not be cached. Do not make the mistake of assuming that this in any way provides you with some kind of security or privacy: Keep using SSL for that.

no-cache

When used alone, this guy lets you specify that caches should revalidate this resource every time, typically using the "etag" header outlined below. The fun bit comes when you specify a field name after the no-cache directive, which lets caches know that you can cache the response, provided that the named fields are stripped out; cookies would be a good example of why you might want to do this. I should note that older User Agents won't respect this trick, so you shouldn't depend on it too much.

no-store

This guy lets you specify that caches should not store this response. I know that may be surprising given the name, but there it is. Actually, if the cache in question is following the rules, it will also ensure that no part of the request is stored either. "no-store" was designed with sensitive information requirements in mind, and so is kind of like the G-Man of cache headers.

max-age

Traditionally, you would let caches know when an asset is expired by using the aptly-named "expires" header, discussed below. However, if you want to be more explicit, you may set a max-age, in seconds, which will override the expires header. Further reasons to use this directive are discussed below under the Caveats section.

s-maxage

Using our keen deductive skills, we can see some similarities between this header and the last one. "s-" is for shared, as in "shared cache", as in CDN. These directives are explicitly for CDNs and other intermediary caches. When present, this directive overrides both the max-age and expires header, and most well behaved CDNs will obey it.

must-revalidate

This one is fun, although not applicable to most of us. For the sake of completeness, and in case your dev-team has some kind of trivia night where free beer is involved, we'll discuss it. Essentially, if your responses include this directive, you are telling the cache that it needs to revalidate a cached asset on any subsequent request, and that it may not, under any circumstance, serve stale content (which is sometimes a desired behaviour). Of course, I say "under any circumstances", but what I really mean is that there's a big fat asterisk next to that claim. If your users are under "severe connectivity constraints", (perhaps they are browsing from low-earth-orbit), then their user agents may serve stale content, provided they pinky-swear to tell their users that they've done so. Apparently this directive exists because some protocols require it, typically involving transactions.

no-transform

“Transform into what?”, you’re surely asking. Some proxies will convert image formats and other documents to improve performance. Presumably this was thought to be a feature that you should have to opt out of. If you don’t like the idea of your CDN making automated guesses about how your content should be encoded or formatted, I suggest including this header.

proxy-revalidate

Essentially the same as the "must-revalidate" directive, except it's just for the shared caches. Why didn't they call it "s-mustrevalidate"? I'm sure there exists a mailing list somewhere where you could find that debate, but for now, just know that like "s-maxage", this directive is designed for intermediary proxies and not user agents. The idea here is that you validate each end-user only once between the proxy and their agent, but each new user should revalidate back to the server. I suspect if your service requires this feature, you probably already know about it.

As always, you should check out the spec for these settings if you need any clarification.

2. expires

Back in the day, this was the standard way to specify when an asset expired, and is just a basic date-time stamp. It is still fairly useful for older user agents, which cryptowebologists assure us still roam in the uncharted territories. On most modern systems, the "cache-control" headers "max-age" and "s-maxage" will take precedence, but it's always good practice to set a matching value here for compatibility. Just make sure you format the date correctly, or it will be evaluated as an expired date:

Thu, 01 Dec 1983 20:00:00 GMT

Try to avoid setting the value to more than one year out as that breaks the specification, (see below for a discussion on cache time settings).

3. etag

Short for "entity-tag", the etag is a unique identifier for the resource being requested, typically comprised of the hash of that resource, or a hash of the timestamp the resource was updated. Basically, this lets a client ask smarter questions of the CDNs, like "give me X if it's different than the etag I already have."

There's a neat trick you can do with etags, which is to make them weak validators. This basically tells the user that although they are not the same, the two resources are functionally equivalent. Support for this feature is considered optional though, so you will want to do some testing with your providers, (discussed below).

4. vary

Oh wow. This one is fun. The “vary” header is extremely powerful, but can trip up what would otherwise be a simple caching scheme. IE has had issues handling the "vary" header in the past, and at one point, even Chrome was handling this in a funny way. Essentially, "vary" lets the caches know which of the headers to use to figure out if they have a valid cache for a request; if a cache were a giant key-value store, adding "vary" fields appends those values to the key, thus changing which requests are considered valid matches for what exists in the cache.

You would commonly set this to something like "Accept-Encoding" to make sure your gzip'ed assets get served where appropriate, saving you all that bandwidth you might otherwise waste. Additionally, setting:

vary: User-Agent

will put you in the SEO good-books if you happen to be serving different versions of your HTML/CSS depending on the User-Agent of the request. Google will note the header and have the Googlebot crawl your mobile content as well.

5. pragma

Another beast from the days of yore, the "pragma" header does many things, and most of them are honoured by newer implementations. The directive we're most concerned with is:

pragma: no-cache

which gets interpreted by newer implementations as:

cache-control: no-cache

I would not generally recommend worrying about it, but for the sake of completeness, there it is. No new HTTP directives will be defined for "pragma" going forward.

Some Caveats

Now that we've gotten some of the standard and expected behaviours out of the way, we should probably mention that not every CDN or User Agent will behave according to the specification, which I'm sure is not news to anyone familiar with browser compatibility issues. For this reason, it is a good idea to test your services before launching a live resource to make sure the behaviour you get is what you expected; it will save you money.

Also, you may have noted that a lot of the headers seem to be either duplicated or overlapping. Some of this is because there are subtle differences between the different methods, and also because the web is shifting over from HTTP/1.0 to HTTP/1.1, which uses the "cache-control" much more heavily. Generally it is safe to set both and let the CDNs and User Agents figure out which one they care to listen to.

1. Compression

Remember that "Accept-Encoding" example for the "vary" header we talked about? That's your new best friend if you intend to serve compressed content, which I hope you do to increase performance and save on bandwidth. CDN providers that get a request with "gzip" as an accepted encoding are expected to request the compressed asset from the origin server, or to serve a cached version of that compressed asset. Historically, this has been a sticking point for some CDNs, and for people wishing to use something like S3 to serve their files, although modern CDNs are able to perform the compression operation themselves if need be.

Things to watch out for when the CDN serves compressed assets is that they'll often ensure that both uncompressed (AKA identity) and gzip'ed version are in their cache, regardless of which was requested. There is a time delay as they perform that operation, so any testing you do will have to take that delay into account.

2. SSL

A CDN is basically a Man-In-The-Middle, meaning you need to think about your HTTPS traffic and how it gets back to your server. Many CDNs will pipe a request for https://somesite.com/asset to your servers as http://somesite.com/asset, so if your server logic depended on that being SSL, either reconsider your logic or ask your CDN to redirect to an HTTPS origin URL. Most CDNs are capable of doing so, with varying degrees of flexibility.

But What about Dynamic Content?

Generally the rule of thumb for dynamic content, like the HTML files of a WordPress blog, is to set “cache-control: no-cache” and prevent the CDNs or User Agents from storing the asset. For most applications, serving your own dynamic content is probably fine, however if you need to boost performance, read on.

Typical Dynamic Content

HTTP/1.1 200 OK
Server: Apache
X-Rack-Cache: miss
ETag: "e6811cdbcedf972c5e8105a89f637d39-gzip"
Status: 200
Content-Type: text/html; charset=utf-8
Expires: Mon, 29 Apr 2013 21:44:55 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 29 Apr 2013 21:44:55 GMT

An example of a section of a header for dynamic content. Note the missed cache status, and the cache-control directives all set to 0 seconds.

Most dynamic content has a shelf-life and is not nearly as volatile as we assume. Things like "Active Users" are probably valid for 10 or 20 seconds, depending on your site. Dashboards that report daily figures could certainly stand to be cached for a few minutes. News feeds could certainly stand to be cached for a while, especially if you remember to set an "etag". If your site is seeing lots of load, it may be worth trying out a short cache on your dynamic assets.

An Analysis of Cache Time

So what types of cache times should you consider setting? Again, this will require you to consider things like the rate of traffic to your site, how sizeable your assets are, and how large your cache is, (yes, there are space allocations, so don't go over them).

Additionally, you should consider the main tradeoff: speed and performance vs. control. If you want to update an asset and need the new version to be live immediately, you may run into trouble if you thought a one-year cache time was a good idea, especially if you set that for Users (max-age) and not just CDNs (s-maxage).

The longest you can set your headers and still be following the specification is one year, or 31536000 seconds, but this is not a very good idea. That's like getting a face tattoo; it stays around forever, barring expensive or painful removal surgery, which accurately describes the annoyances of having to trick caches into refetching an asset through name changes or hoping you remembered to correctly set your etag and that your users and CDNs implemented them correctly. If your servers can't stand to have your CDNs ask once a day if your profile picture has changed you should upgrade your servers.

Sane Static Content Headers

HTTP/1.1 200 OK
Cache-Control: no-transform,public,max-age=300,s-maxage=900
Content-Type: text/html; charset=UTF-8
Date: Mon, 29 Apr 2013 16:38:15 GMT
ETag: "bbea5db7e1785119a7f94fdd504c546e"
Last-Modified: Sat, 27 Apr 2013 00:44:54 GMT
Server: AmazonS3
Vary: Accept-Encoding
X-Cache: HIT

A sample of cache settings for a static asset served from S3. Here we see that the cache has been asked to store the asset for 900 seconds rather than the 300 seconds set for User Agents. Also note the "x-cache: HIT" header, indicating the CDN served the request.

There is one exception to the “thou-shalt-not-set-one-year-headers” commandment, or more accurately, there’s a bit of a hack to get around their pitfalls: if you’ve configured your site to generate resource names, you can rename your assets each time you publish a new version. Typically, this will involve an incrementing version number, a date-time stamp, or a hash of the contents, much like an “etag”, being appended to the file name, so that you end up with things like “core.E89A123FFDB...432D687B.css” and the like. It’s not pretty, but really, who cares? Also, this lets you set one-year-headers without worrying about updating your resources.

A handy table best explains the cache time trade-off. Assuming a web asset get 500 requests per minute, then the following Hit Ratios are possible for each cache time:

Cache time (minutes)	Hit Ratio	Request to Origin / Hr
1	99.8%	60
5	99.96%	12
20	99.99%	3
60	99.997%	1
86400	99.9998%	<1

There. It's spelt out. What kind of Hit Ratio do you need? Typically 60 seconds to an hour is an ideal trade-off. For pseudo-dynamic content, you can still use CDNs, just start working in the under-60-seconds range, as appropriate for the request.

Testing Your CDN

So, onto the testing. Always check that the headers are coming through CDNs the way you expect when setting up new services. Typically a CDN will insert some kind of "x-" header to indicate that it hit, missed, or served expired content. What we're looking for is a convenient way to look at this and other headers, and to set them on our requests for testing. There are more than a few tools out there to help, though these are the ones I’ve found most handy.

1. Web Inspector

Probably the most accessible method, simply right-click on a webpage in Chrome, click Inspect Element, navigate to the Network tab, hit refresh, and click on the HTML asset at the top. If it's not selected by default, checkout the Headers tab to see all your request and response headers. Chrome also lets you set the user agent you request with, and gives you the option to not use your local cache, which is handy when you're testing.

2. Charles Proxy et al.

Tools like Charles Proxy will let you route traffic through them and manipulate DNS lookups, record headers, and see performance statistics, all in a clean, easy-to-use GUI. These tools are generally more task-specific than the Web Inspector and other browser tools, and generally worth the small licensing fee if you work with web requests in any significant way.

3. cURL

Quick, easy, and highly flexible, cURL lets you make web requests directly from the command line and check out the responding headers. Helpful flags include -A for user agents, -b for cookies, -F for form data, -H for setting headers, and -I to request the header only. Very handy for sanity checks, and extremely powerful.

4. hurl.it

hurl.it is essentially cURL with a prettier interface, and in a webbrowser. You’re able to set the headers you pass in to the request, as well as view the full header and body response. Although you don’t quite get the robustness of the command line, it’s still very useful for a quick header examination, and is probably the fastest way to get access to headers if you’re in a hurry.

5. Python and Requests

Requests is a great Python package for making web requests, and is useful for quick checks from the REPL, or from slightly more complex scripted checks. Using Requests comes with the added benefit of letting you write a test-suite for your web assets to assert the status of response headers.

Notes from the Field and Parting Thoughts

So now you’ve made it all the way through our quick guide to cache headers, and you’re eager to get going on setting your own. Excellent. However, now it’s time to put some of this in perspective.

Most web servers, like Apache and Nginx, will do a lot of this work for you. Really, the “cache-control” header is the only one you’re going to have to work with. Web browsers are typically set up to cache aggressively to improve user experience, so often you’ll be fighting to prevent caching rather than ensuring it. Generally this means that you set a “/static” path and configure its caching headers to allow caching up to some reasonable interval like 300 seconds. Next, you would ensure that your root path “/” has “cache-control: no-cache” enabled, or better yet route dynamic content straight to your servers and only have “/static” use your CDN. This is a healthy starting point for most purposes. If your hosting bill is astronomical, or you get to use operations cost savings as beer money, then consider tweaking your settings.

CDNs will often give you the option of breaking protocol and will cache for whatever period you like, regardless of the headers. They also take liberties with the protocols depending on their interpretation of expected behaviour, which is why it’s important to test the headers you get out of your CDN and compare them to your servers. Consider the expected behaviour outlined here to be a baseline and keep an eye out for variance.

Good luck, and happy caching!

For more tips and tricks on improving latency checkout our analysis of Mobile Web Performance.

Mobile Web Design for Content-Heavy Websites: 8 Clever Techniques from Golf Digest

2013-04-17T16:05:00-07:00

Adapting a content-heavy website for mobile devices can seem like a difficult design challenge. How do you successfully squeeze all of that valuable content into such a small viewport?

Many publishers are choosing to ignore this challenge, happy to let users pinch, zoom and squint their way around a website. And if the content—for example, an image gallery—fails to work at all, it's the user's fault for browsing on a smartphone.

Other publishers, however, do not feel the same way.

Golf Digest is a monthly golf magazine published by one of the world's leading publishers, Condé Nast. The property has 6.3 million monthly print subscribers and over 1 million unique visitors to the website each month. It's kind of a big deal in the golfing world.

GolfDigest.com is a great place to go for getting up to date with the latest news, cutting-edge equipment and in-depth golfing stories. It's also a fantastic resource for improving your game on the go— featuring video tips, image galleries and articles by some of the world’s leading golfers.

As it turns out, GolfDigest.com is also a solid example of how to design a brilliant, content-heavy website for mobile devices. The Golf Digest development team has launched a full-featured mobile destination for their readers and the result is a fast and attractive companion website for any golfer on the go.

Why is the mobile website so successful? Below, you'll find 8 mobile web design best practices found on GolfDigest.com that create such a rich mobile experience.

1. Hint and reveal with progressive disclosure

Mobile devices are small and websites tend to present a lot of information. Readers can easily experience information overload when there are too many details displayed on their small screens. Progressive disclosure hints at content beyond the frame of the device, and then reveals that content when the user swipes, taps or scrolls to access it.

By using progressive disclosure, you'll showcase the important parts of the page—your valuable content—while at the same time highlighting actions available to users.

Golf Digest hides the site navigation to simplify the header on mobile devices. A quick tap is all it takes to reveal the navigation and move to the next section.

2. Ads don’t have to be a nuisance on mobile

Ads that are poorly integrated on mobile devices can be a user experience disaster waiting to happen. To avoid this scenario, you need to think about how users navigate your website and make sure that your ad system doesn't ruin the user journey. This is especially true if you're serving up in-depth, high-value content that users might want to focus on.

The Golf Digest mobile website is a great example of how to elegantly integrate your existing ad system into a content-heavy website without breaking the mobile user experience.

3. Carousels make the world go 'round

Carousels are user interface elements that show additional content when a user swipes horizontally. They provide an excellent way to showcase images and highlight featured content.

Golf Digest has a large image carousel in the header of the homepage. Readers can swipe between stories to browse the most relevant new content for their interests.

4. Sticky navigation speeds up browsing

Sticky navigation refers to navigation that remains in place no matter what part of the page is being viewed. Recent studies have shown that users really appreciate the accessibility of sticky navigation, as it can make browsing a site up to 22% faster. Smartphone audiences typically move through websites quickly, so a sticky navigation can be a particularly useful feature.

GolfDigest.com provides users with a sticky navigation that makes it easy for them to get to their next destination. In sections of the website where users need to quickly jump between written, video or image-based lessons, providing a persistent way to navigate is an excellent design choice.

5. Large touch targets = a UX hole in one

While it seems counterintuitive to make everything bigger on a device with a limited screen size, in reality none of us are as dexterous as we’d like to be on mobile devices. Unfortunately, our fleshy fingers are much less precise than the pin-point click of a mouse.

With this in mind, make your website's touch targets big. We recommend at least 40px by 40px on smartphones with a 10px margin around the targets.

Each of the content links on GolfDigest.com are large and easy to tap, whether you're casually browsing on the couch or quickly sneaking in a few quick tips on the course.

6. Forget the desktop and augment your content for mobile

On mobile, simple tends to beat pretty. Simple also tends to be pretty, so that helps too! Take a moment to look at your website’s content on a mobile device. Is the information communicated effectively? Does the existing design augment or distract from the message?

GolfDigest.com features a compelling image gallery that leaves behind the desktop design to focus on solving the gallery design challenge on mobile. To maximize the space allocated to images, extra information about each slide is hidden in a drawer underneath. Users can also get additional information about the parent article by clicking the 'i' in the upper left corner, while the 'x' in the upper right corner brings users back to the page that you came from (a clever use of the browser's history API).

7. Be lazy! Let the OS do all the work

Creating design elements that look pretty across all screen sizes, browsers and breakpoints can be a total pain. For DOM elements such as dropdowns, radio buttons and sliders, you can easily draw upon existing native UI elements without having to worry about styling your own. These native elements have large touch targets and look great on any device.

Golf Digest provides a fallback to native OS elements in their dropdown menus.

8. Adapt for portrait and landscape viewing

The majority of mobile devices can be used in a portrait or landscape orientation, so websites need to be responsive to the orientation the device is in. Pages should adjust using a fluid grid layout when a customer switches from holding their phone in portrait mode to landscape mode (or vice versa).

Golf Digest's flexible grid-layout adapts the content to different sized phones and orientation changes.

The Bottom Line

People visit your website to interact with the content that's there (duh!). But hosting great content only half the battle—showcasing it in a device- appropriate design is also extremely important. Whether users visit your website to learn, read, browse, shop, listen or watch—they expect your content to be accessible and digestible on any device.

Put your users first and take steps to optimize the design of your content. They'll thank you with their time, money and loyalty!

Do you have a favourite publication that provides a great experience on mobile? Let us know in the comments!

GolfDigest.com was built by the Golf Digest in-house development team using Mobify.js, our open source JavaScript library. If you'd like to launch a mobile website as awesome as GolfDigest.com, feel free to check out the Mobify.js developer documentation to get started!

Red state, blue state 2012: iOS vs Android in the USA

2013-03-21T16:05:00-07:00

When it comes to politics, the US loves dividing up the country according to political parties. But who is winning the battle of the mobile operating system?

In this article, we’ve broken down mobile website traffic in the US according to mobile OS. Our sample data comes from Mobify's publisher network, which is composed of over 20,000 sites with almost 200 million visits in 2012.

Why are we only featuring iOS and Android? Simple — when combined the two mobile operating systems made up a whopping 93% of all mobile page views in 2012. The next mobile operating system on the list was Blackberry OS, with a 4% share of mobile web traffic.

Results

To view the dominant operating system in your state, check out our interactive chart below. The more red the state, the higher the percentage of Android mobile page views in 2012. The more blue the state, the higher the percentage of iOS views. States close to 50 / 50 will be a shade of purple.

The eagle eyed observer will notice that the biggest difference between iOS and Android exists in South Dakota and Nebraska. Nebraska has 70% Android penetration, while South Dakota has a stronger iOS penetration – also at 70%. Pennsylvania, New Hampshire, and Virginia are all split evenly at approximately 50 / 50.

What does this mean for your mobile strategy?

Since 93% of traffic on the mobile web in 2012 was generated by iOS and Android, focusing on these mobile operating systems is going to be key to a successful mobile strategy.

When launching a mobile website, be sure to spend the extra time testing on the major browsers that run on Android and iOS like Mobile Safari, Chrome, Opera and Firefox.

This post focused on the breakdown of mobile web traffic in 2012 according to mobile operating system.

If you're looking for a breakdown of web traffic according to device, feel free to check out our previous article, Which devices dominated the mobile web in 2012?

Evolving Mobify part 5: HTTP Basic Authentication

2013-03-13T11:22:00-07:00

Today we’re happy to announce the latest in Mobify’s biweekly update cycle!

This round’s update consists of one feature: adding support for HTTP Basic authentication. Why only 1 feature this time? We’re working hard behind the scenes on some major new features that will be released soon!

HTTP Basic Authentication

HTTP Basic authentication is a method for a browser to provide a user name and password when making a request — you’ve probably seen sites where a built-in browser dialog box pops up and asks for a username and password.

Now you can use Mobify to take your site mobile, even if your site is protected by basic authentication.

To use Mobify and HTTP Basic authentication, type the following string into the URL field in the Mobile Site Settings in Mobify Cloud:

http://USERNAME:PASSWORD@www.site.com/path.html

Where

USERNAME = your username
PASSWORD = your password
www.site.com = the URL of your site

This feature is particularly useful for those of you who have staging environments protected by HTTP Basic authentication. It’s important to note that you won't see a password dialog in Mobify Studio or when you click Preview, but users on your production site will still be prompted.

This is already live on all Mobify accounts — log into cloud.mobify.com and try it out yourself. If you haven't played with Mobify in a while, a lot has changed — now's a great time to see what we’ve been up to!

If you’d like to see a live demo of Mobify features in action, register for our biweekly webinar.

More on previous updates to Mobify are available here.

If you have any questions or feedback, email us at support at mobify.com and we’ll get right back to you!

4 Real-World Business Impacts of a Responsive Design Approach

2013-03-13T10:00:00-07:00

Responsive design is quickly gaining recognition as the dominant approach to building new websites. With good reason, too: a responsive design workflow is a fantastic way to build tailored web experiences for different screen sizes and resolutions.

However, going responsive comes with its own set of difficulties. In this article we highlight the four main challenges of a responsive design approach, and address the business impact of each challenge. If you're thinking about going responsive, here are some critical facts you should know in advance!

Responsive Performance

A website is considered 'responsive' when it dynamically changes its layout based on the screen size of the device that loads it. The properties that control these changes are written into your website's HTML, CSS and JavaScript.

In a responsive design, each time a page loads on mobile it is also loading instructions for how to look on desktops and tablets. Every time a user visits your site they download the full page content by default – no matter which device they are using.

We conducted an analysis into 15 top responsive e-commerce sites which revealed that the average responsive site home page consists of 87.2 resources and is made up of 1.9 MB of data. If 87 resources seems like a lot, it is. The reason the number is so high is because together they dictate how the web page should render on all desktop, smartphone and tablet devices.

However, since your customer is only using one device at a time to access your website, they have to wait for these resources to download before the page becomes usable on their device.

It almost goes without saying: we all hate waiting for websites to load. When your customers are forced to wait, the result is a drop in conversion rates and user satisfaction. On smartphones, the conversion rate drops by an extra 3.5% when users have to wait just one second. By the 3 second mark, 57% of users will have left your site completely.

The best way to speed up performance on your responsive website is to make sure that only the necessary resources are sent to customers’ smartphones, tablets and desktops. For maximum performance, you can also compress and concatenate JavaScript and CSS resources. This can be done manually by your in-house development team with tools such as SASS for CSS compression and Uglify for JavaScript compression. You can also use Mobify's own automated JavaScript and CSS acceleration tool.

By using services like the above, you'll ensure that the number of HTTP requests is dramatically reduced and customers spend less time waiting for pages to become usable.

Responsive Images

Images pose one of the biggest problems in responsive design. If a responsive design uses one markup across devices, how do you ensure that only big, beautiful images are served to your Retina Macbook Pro, while making sure that an old smartphone is sent smaller images that make sense for its lower-resolution screen?

The business impact of the image problem is significant. For image-rich websites, page performance on mobile can become significantly degraded, with a similar impact on conversion rates as detailed in the previous section. In addition, the wasted bandwidth of sending weighty images to the wrong devices can cost thousands of dollars a month.

"To say that a picture is worth a thousand words is really an understatement. In our case, pictures are worth hundreds of millions of dollars. You have to create a new way to load those images dynamically and in real time without affecting the user experience, so having the fastest possible load time is very important to us.”

Richard Cohene, Beyond the Rack

The good news is that there are a number of different solutions to the responsive image problem. The bad news is that these solutions frequently require you to make significant changes to your existing site, or come with performance challenges of their own.

At Mobify, we've created a solution that lets you dynamically optimize your images for all screen sizes and resolutions. Using a single JavaScript tag inserted into your HTML, you can deliver responsive images through our global CDN. This means that you can always send the right images to the right devices, and do so at remarkable speeds – without needing to change your website.

A responsive e-commerce website, pre-optimization. It loads 136 resources, and is 9.02 MB.

The same responsive e-commerce website, after script and image optimization. It includes 23 resources and is 581.81 KB, leading to a much faster load time, reduced bandwidth costs and much happier customers.

Responsive Content

Going responsive ensures that your existing ‘desktop content’ is also available on smartphones and tablets. This is a great approach, because users on these devices expect to have access to the same information across different devices. It also means that every time you update your content, it is automatically reflected across desktops, smartphones, and tablets.

However, a responsive design often gives designers only two choices for how to present a page's content. They can either:

Load existing ‘desktop content’ and show all of it
Load specific desktop and mobile content and hide one or the other

The challenge stems from the fact that while mobile users expect to have access to the same content across all devices, they also expect the information to be presented in an easily digestible format. You could show all of your existing content, but that doesn’t provide a great experience for people reading on smaller screens. It’s also possible to load both your ‘desktop content’ and ‘mobile content’ and selectively hide the elements you don’t need on a particular device, but that just adds to the performance problems detailed above.

The business impact of this problem varies between industries. If you monetize your website through advertising revenue, a dip in pages per visit due to inferior content layout is bad news for your bottom line. If you have an e-commerce website, a poor mobile user experience could result in reduced mobile conversion rates at all points in your funnel.

The best solution to the responsive content problem is to try and make your content adaptive. Rather than envisioning a model where you have ‘desktop content’ and ‘mobile content’, think about how you can publish content so that it is useful on any platform.

Sometimes this can mean changing existing content so that it doesn’t break when viewed in different contexts (e.g. a headline isn’t clipped on a smartphone screen). Other times it can mean creating additional content, structures and metadata that allow you to create content once and then publish everywhere.

Development Time and Resources

One of the most frequently cited problems with responsive design is that it is difficult and expensive to shoehorn an existing website design into a responsive framework. Since you’re integrating several designs into one markup, it can take several times as long to get a design to market. You have to employ advanced techniques to make sure the design works perfectly on every browser, while also making sure that it will degrade gracefully in older, legacy browsers.

The business impact is represented by the additional time, QA and resources it takes to make your site responsive. the more expensive the project becomes.

The trick to implementing a responsive design is to do so in a way that is sustainable and realistic in a world of limited resources (time, people, money). For example, it is generally cheaper and easier to go responsive when rebuilding a site from the ground up, rather than trying to manipulate an existing design.

Another solution is to forgo a single responsive design in favour of using adaptive templates. Rather than trying to create one design that is responsive to every device type, adaptive templating allows you to break up the design into multiple responsive templates that are targeted to different device screen sizes and resolutions.

Unlike a single responsive design, you can test and launch adaptive templates without needing to see if one design decision is going to break another part of the design. It’s a great way of reducing the time, development, and QA cost of going responsive.

Conclusion

Here at Mobify, we’re huge believers in the principles of a responsive (and adaptive) web. The idea that a website should dynamically respond to the device that is accessing it is a core part of our company philosophy, and we’ve built responsive design best practices into our platform to help websites quickly and easily launch amazing experiences for every screen size and resolution on the market.

If you’re thinking of launching a responsive website and would like more information on how to avoid the challenges of a responsive approach, please contact Mobify’s Director of Customer Success, Ben Terrill.

Talk to Ben Today

8 Influential Women in the Mobile Web

2013-03-08T10:00:00-08:00

Today is International Women's Day and we couldn't resist highlighting some of the amazing women working in the mobile Web.

Some of these women are making the web more adaptive. Others educate businesses, designers, and developers on the importance of the mobile channel and how to get the most out of it.

Mary Meeker, Partner at Kleiner Perkins Caufield Byers

Twice a year, Mary Meeker of Kleiner Perkins Caufield Byers shares her insights about the current state of the Internet in a presentation called Internet Trends.

The Internet Trends presentations are highly anticipated and provide outstanding insight into global macro trends and factors that influence the Internet economy — e-commerce, mobile and communications.

One of Meeker's bravest predictions back in 2008 was that mobile web usage would surpass desktop web usage by 2014. Right up to our latest numbers, her prediction is proving correct.

Julianne Bowman, Principal Consultant at PA Consulting

With 15 years of experience in UX for online and mobile, Julianne has become a reputable source of insight into product innovation and mass adoption of new technologies.

Follow Julianne on Twitter

Julie A. Ask, Vice President, Principal Analyst at Forrester

Julie has worked as an analyst for over 12 years, bridging her technical and business expertise to produce great research reports, such as the recent "2013 Mobile Trends For eBusiness Professionals" and "2013 Mobile Trends For Marketers."

See Julie's profile on Forrester

Lyza Danger Gardner, Co-Founder at Cloud Four

Lyza has over 15 years of web development experience, both front-end and back-end. She has recently written an authoritative book on mobile development and the impact it has on business, "Head First Mobile Web."

Follow Lyza on Twitter

Stefanie Reiger, Co-Founder at Yiibu

Stephanie is the co-founder of Yiibu, a company that designs and develops elegant, informative and easy to use interactive media experiences for diverse platforms. She also has a strong creative background in corporate communications, marketing and advertising.

Follow Stephanie on Twitter

Angela Steele, CEO at Ansible Mobile

Angela heads up one of the most successful mobile ad companies, Ansible Mobile. Ansible does mobile advertising by such clients as Microsoft, Intel, Intuit, Johnson & Johnson, and is a part of IPG.

Jesse Haines, Head of Marketing at Google Mobile Ads

Jesse has been at Google since 2005. She's Google's #1 most powerful woman in mobile ads due to that fact alone. Jesse's goal is simply to persuade advertisers to get the basics right: A huge number of them don't have mobile-optimized web sites, and mobile users often prefer those for mobile shopping to apps.

Sara Wachter-Boettcher, Editor-in-Chief at A List Apart

Sara Wachter-Boettcher is a content strategist, writer, and the editor-in-chief of A List Apart. An advocate for meaningful, memorable, future-friendly content, Sara is the author of Content Everywhere from Rosenfeld Media, a frequent conference speaker, and an occasional blogger.

Follow Sara on Twitter

Did we miss anyone? Let us know in the comments!

4 Preliminary Steps to Optimizing Your Site for Mobile

2013-03-05T10:00:00-08:00

At first glance, adapting a website for mobile and tablet devices can appear like a complex task.

What content should be the most prominent on your mobile website? Which pages should be optimized first? How should the layout change for different devices?

It's easy to get overwhelmed.

We created this article to arm you with the important data you'll need to begin launching a successful mobile website, as fast and as painlessly as possible.

Site Analytics to the Rescue

Here at Mobify, we follow the belief that "when in doubt, look at the data."

When optimizing your website for smartphones and tablets, the best place to start is always with the site analytics.

You already have visitors coming to your site from mobile, even if it's just a small percentage at this point. By studying these users, you can uncover insights into what matters the most to them, and how you can better serve their needs.

In this post, we'll be using Google Analytics for our research, but other website analytics platforms should have similar functionality.

Step 1. Segment out smartphone and tablet visitors.

Research has shown that people behave differently on different devices. And the context often matters a lot more than the screen size.

Let's begin by looking specifically into our mobile traffic. Here's how:

Go to the Reporting section in Google Analytics.
Switch to Audience > Overview in Standard Reports.
Click on Advanced Segments.
Check All Traffic, Mobile Traffic, and Tablet Traffic segments.
Click Apply.

A new report will be generated showing how many visits you get from each device type, as well as other useful information.

This is a great place to start learning about your mobile traffic. Look at the metrics such as Bounce Rate or Pages/Visit and see how they compare between device types.

Step 2. Learn about your top content.

We also want to find out which pages your mobile users visit the most and which pages they land on when coming from different sources.

For example, your Home Page may be the most visited page on your website. But your Contact Page may be the one most people see first when they arrive from search engines.

To find out what your top content is, follow these steps:

Go to Content > Site Content > All Pages.
Look at 3 segments (All, Mobile, Tablet) separately and pay attention to the differences. See anything surprising yet?
Repeat these actions for Content > Site Content > Landing Pages.

You can use this information to prioritize your design and development effort. We suggest aiming for at least the top 5 most visited pages and the top 5 landing pages. This should cover about 40-80% of your mobile traffic.

Step 3. Select the most important devices.

Next, let's identify which exact devices people use to access your site.

To begin, deselect all Advanced Segments in Google Analytics and go to Audience > Mobile > Devices.

This information is useful when prioritizing your mobile adaptions. Should you build a mobile site or tablet site first? Do you have to tackle both at the same time? This report will help you identify which devices to target.

Step 4. Set up benchmarks for performance.

Now that you know who your mobile visitors are, what content they view and which devices they use, you're well-equipped to start making informed decisions about going mobile.

The last thing you should do is make a report of all the important metrics and set up benchmarks and goals.

Here's a list of metrics we use here at Mobify:

Conversion rates and revenue
Mobile visits and percentage of total traffic
Bounce rate for different devices
Pages / Visit
Praise vs. Complaints (this one's qualitative)

Keep track of these numbers over time to calculate your return on investment.

Wrapping Up

In order to ensure long-term success, your organization needs a well-designed and high-performing mobile website.

Digging into your site's analytics can help you uncover valuable insights and make better business decisions when going mobile.

For more resources and practical tips on going mobile, check out our free 2013 Mobile Success Pack.

Vancouver JS Meetup Recap

2013-02-28T09:56:00-08:00

On February 12th, the PayrollHero dev team in Whistler travelled down to Vancouver to attend the Vancouver JS meetup. We reached town a bit early, so we spent some time coworking from Launch Academy. Later in the evening, we went to SFU Harbour Center where the event was being held. Around 150 people attended the event. Two top notch speakers presented in the meetup. Robert W. Hurst of Chloi presented a talk on LucidJS and Peter McLachlan of Mobify presented a talk on Mobile performance.

Continue reading on PayrollHero