There seems to be a growing confusion about how best to monitor Websites.

There are three main categories of monitors out there:

1. Pure availability monitors
2. Availability and Performance monitors
3. Availability, Performance and Quality Assurance monitors

Simply put -

type 1 will do little more than Ping your Web server to make sure it's up and running
type 2 will issue GET requests and make sure your website can server up content and that the performance is acceptable
type 3 will do what the type 2 monitors do but will also check the quality of the web page to make sure it Functions, is Standards Compliant and meets Accessibility Standards.

Most of the main players in the market offer type 2 - Availability and Performance monitoring.  They'll check a given URL to make sure it responds to requests and that the responses perform acceptably.  (definition of "acceptable" performance can usually be defined by the customer as a threshold and alerts are sent if it is exceeded).  They will also make a lot of noise about how they can test the designated URL from multiple locations over your choice of geographic locations.

The level of detail provided goes all the way down to the DNS lookup time for a GET request for an image on the page.  So I can check the performance of individual  GET requests from multiple locations around the world and then analyse the data and identify any "slow" spots.

I've been working in the Website performance arena since 1996.  I've worked with companies that have resold these services.  I have to tell you that I'm just not convinced by the arguments.

Let's take my example of the level of detail that's available.  I can measure the DNS lookup of individual items on one of my web pages from multiple locations around the world.

The DNS lookup is done by the first router involved in sending the GET request.  It takes longer if this is the first request for the URL in a while as it has it wont have it cached.  So when you issue the first GET request for the URL it may take, say, 2 seconds to do the lookup.  That's quite an interesting piece of information but as the monitor service is going to issue the same request 5-10 mins later the DNS lookup will probably be much quicker the next time as it'll be cached.  (of course it may be a different router this time so it may be 2 seconds again, however pretty soon all the routers nearest the server issuing the GET requests will soon "recognise" the URL and instantly resolve it to the IP address)  So this information very quickly becomes meaningless.

The route the data being sent back from your Web server in London to the Monitoring Server located in, say, Los Angeles will not take the same route each time.  Some of these routes may have different latencies or may have a greater error rate on them causing a greater proportion of data packets to be re-sent thus making some responses longer than others.  Again quite an interesting piece of information but what on earth are you going to do about it ?

In fact I'd argue that testing form multiple parts of the world is full of interesting information but is telling you about things that you have minimal control over.  You can't ring up your ISP and ask them to make sure that data you send out to a request from Ulan Bator bypasses Uzbekistan's routers because they are a bit slower than the ones in Kazakhstan.  (please don't assume that I'm sad enough to have checked these suggested routings !)

Even if you bring this closer to home, what are you going to do if you find that responses are marginally slower from a monitor making calls from a point of presence on the BT network compared to one on the Cable and Wireless network when you're hosted on L3.

Analysing things down to this level of granularity means that it takes a while to do each monitor and you gather a relatively larger amount of data each time.  So most of the vendors can only offer a monitoring frequency of every 5 minutes.  That number hasn't changed in a decade yet the importance of your website has increased by many orders of magnitude.  This level of delay before being made aware of a problem is becoming less acceptable.

The information you get from a website monitor should be about those things that YOU can influence and in a time frame that allows you to maximise your availability.

You need to focus your view on what you should be monitoring for - what's actually important to your business.

It's all very well having a web page with startlingly fast response and download times but if a partially sighted person can't read it, or it doesn't render in Firefox version 3 or the link to an important page doesn't work - well you might as well have not bothered. 

Sitemorse's current monitoring service offers type 3 coverage.  So it'll give you a view of your availability and performance (alerting you by email or SMS if there's a problem) but will also run all the Sitemorse tests against the page and allow you to set thresholds against them and generate alerts.  That way you'll know that the web pages are available, performing well, are functioning, are abiding by coding standards, are accessible, meet your metadata requirements, are free from spelling mistakes etc. etc.

Watch out for announcements over the coming weeks about new monitoring options from Sitemorse.  But just to let you know - if you want to measure your performance from Ulan Bator I'm afraid you'll  be disappointed by our announcements !