Apr 27, 2010

Developing a web browser support plan

When I started working at my current company, our QA department was testing our website with every web browser they knew of. Certainly 100% cross-browser pixel-perfect rendering is every developer's dream, but it is often a nearly impossible goal to achieve (unless you have a very basic design). With very limited time and resources, I knew we had to pick and choose our battles. My goal was to establish guidelines so developers and QA knew in advance what to expect.

Know Your Audience

The first step is to determine what the visitors to your website are using. This is important as different sites can attract different types of users. Websites whose content revolves around web technologies will typically attract a larger than normal share of non-IE web browsing traffic. I would guess that MSN is likely at 95% or more IE traffic. Apple.com likely sees very large numbers of Safari users, as would many i-Pod/Pad/Poodle/Pomegranate/etc websites. Sites aimed at a youthful audience may draw more alternative browsers and mobile devices than one aimed towards an older crowd who may be less tech savvy and more likely to use the web browser their computer came with.

There are many statistics packages available, and honestly I do not have enough experience with them to recommend any in particular, so I will leave that exercise up to you. I will say that they are invaluable tools. Not only will you learn what web browsers your visitors are using, you will learn where your bandwidth goes, what pages are drawing the most visitors... etc. Discussing the full benefits would be another article entirely, so let's just say having such a package installed is a must for any website with any decent amount of traffic.

In our case, we had both Webalizer and AW Stats in place. Unfortunately they weren't configured very well and user agent data wasn't of much use. The one piece of information we were able to gather was that we were overwhelmingly an IE-visited site (perhaps 85-90%).

Be wary of the chicken and the egg problem when looking at your own website statistics. Most of your visitors may use IE, but that could be because your site is unusable, unreadable, or just plain broken in other browsers.

Know The Web

With that information in hand, the next step is to look at the web as a whole.

Gathering statistics was fairly easy as the wonderful Wikipedia community has already done most of the work, as found on the usage share of web browsers page. This should be your one and only reference. Personally I weigh Net Applications numbers more highly than any of the other metrics sites simply due to the massive dataset they use.

One serious mistake I frequently see is developers who quote W3 Schools web browser stats. If you take one thing away from this article, it should be this: do not use W3 Schools web stats. It pains me to see how many developers quote their website. There is even a disclaimer on that page which indicates the results are only for the W3 Schools websites which draw a much higher percentage of visitors who use alternative browsers. So please, just don't do it.

Know Your Region

Web browser stats can often be biased towards a North American audience. If you have (for example) a Chinese-language website and 99.5% of your viewers are in China, then it stands to reason that you should strongly skew your support towards the most popular browsers in China.

This information is often available, but it can be hard to find, and often it has to be paid for. Net Applications is one such source, but there are likely more.

Examine The Data

With statistics in hand, analysis can still be somewhat nebulous. What if your website shows 10% IE visitors, regional data shows 80% IE users, and global data shows 55% IE users? Unfortunately there is no right answer as to how to interpret the data. If you want to focus on keeping your existing visitors happy, you should probably weigh your current visitor data more highly than the other data sets. If you want to attract more local business, maybe you need to skew more towards regional data. If you want to form a global empire, perhaps focus on use the global stats. There is no right answer. Try to come up with a fair interpretation that your organization can agree on. One blanket recommendation I can make would be to skew your current viewership most highly no matter what your goals are. You don't want to alienate your current customers or readership.

Web Browser Versions

Supporting multiple web browser versions can be a nightmare (most notably for Internet Explorer where there can be huge rendering differences). Utilizing your collected data you must decide not only which web browsers to support but which versions to support. Once again there is no right way as to how to do this. Perhaps only a small percentage of your visitors may use IE6, but if your website is hugely popular, that can still be a very large number of people.

For our support goals, we differentiated between IE and everything else. For IE, we would support every version that holds more than 5% overall market share. (Unfortunately for us that still includes IE6.) For other web browsers we would only support the latest version. Our reasoning is that if someone has enough knowledge to find and use an alternate browser, they would most likely be capable or likely to update the browser. IE6 is still in use mostly due to corporate infrastructures. It is doubtful other web browsers have anywhere near the same level of tie-in that IE does, so upgrading should not be an issue.

Determine Support Levels

With finite resources it is important to classify different problems into levels of priority. Your main site menu being broken in IE is a far more serious issue than some interior page images moving over a few pixels to the left in an old version of Opera. The former needs to be fixed right now, but the latter can wait, perhaps indefinitely. It appears we can quite naturally divide this problem domain into two parts: which browsers are affected, and what is the problem severity.

Browser Support Levels: Your web browser market share data comes into play here. You should assign each web browser (and/or browser version) a support level. What levels you assign or how you manage them isn’t important as long as you have a clear hierarchy as to what is important to your organization. In our case, we had the following:

  • Full Support: Every problem reported for these web browsers is considered valid
  • Partial Support: Problems have to meet a certain severity in order to be considered valid
  • Unsupported: Most problems will not be addressed (these web browsers are not even used by QA for testing)

Problem Severity Levels: How severe your organization considers the problem to be. This can be a tough one to nail down. For example, you can't group all graphic problems together, as some graphics may convey important data, may have the corporate logo or identity on them, or perhaps lead to a download or purchase page. We came up with:

  • Severe: Website core functionality or core business is affected, main landing page issues, or the corporate identity is diminished or adversely affected
  • Major: Non-core functionality or appearance issues that detract from the site but do not affect core functions or income generation
  • Minor: Everything else. Occasionally these problems are so small that they are deemed not worth fixing due to financial or time constraints.

Assign Support Levels

Data in hand, you can now assign support levels to different browsers. Compare your usage data with the problem severity levels and determine what your support goals will be. This has to be agreed upon by all parties involved.

We had decided that IE (all versions) and Firefox (latest) would have full support and would address Severe, Major, and Minor issues. Safari (latest) would have Partial support and we would address Major and Severe issues. Other browsers would be unsupported, but, while we would no longer actively test those browsers, we would still address any Severe issues when brought to our attention.

Review Periodically

It is important that you revisit web browser statistics periodically to ensure you are staying up to date with current trends. How frequently you check will vary based on your business and organizational comfort level, but likely you will want to check every 3-12 months. If you check more frequently you probably won’t see any noteworthy changes and by checking less frequently you will risk falling too far behind the curve. Our review period was set at 6 months and it has proven to be effective thus far.

Final Outcome

It is important to note that creating and assigning web browser support levels helps all aspects of the business. The developers know what they need to do to deliver a satisfactory product. QA has their workload managed by not having to support every web browser on the planet. Administrators will have more accurate estimates and will feel more empowered to make decisions on what issues are fixed in what order. It really is a win-win scenario for everyone involved.