I've received a few emails asking why SOCITM, Shaw Trust and AbilityNet assess sites as achieving high levels of accessibility and yet these same sites do not rank highly in the Sitemorse Surveys.  People find it curious, particularly as SOCITM use Sitemorse data as part of their assessment.

A recent email came in specifically asking about Manchester, Oxfordshire and Mendip councils.

The simple answer is that whereas the aforementioned organisations primarily assess a site's accessibility,  Sitemorse, on the otherhand, checks for many more things (Function, Accessibility, HTML code compliance, Performance, Metadata, Spelling, eMail links, PDFs etc)  and scores the site across all of the tests we perform.  So we are rating the sites by far more criteria, hence the potential for different ratings.  This is particularly true of Manchester and Mendip.  They achieved an OK score for Accessibility but scored poorly for Performance and Function. (Function is primarily checking links, so they both had problems with broken links).  We still found issues with their AA compliance on some of their pages and this is another difference between manual testing and automated testing.

It isn't practical, or cost effective, to test a large number of pages when assessing a site using manual testing.  Whereas Sitemorse is exceptionally good at checking hundreds or thousands of pages on a website where the only way to test is by looking at every single page and every item on every page to check them against the guidelines and standards  However, manual testing is very good at assessing a site where it is necessary to make the more subjective judgements that are necessary when assessing a site.  How intuitive is the navigation of a site ?  Is something that is ideally suited to a manual check and very difficult to accurately assess using automated methods.  That is why we always advise our clients to use a mixture of manual testing as well as using Sitemorse.

And, of course, sites change over time.  So SOCITM's assessment of sites that was carried out at the end of 2008 for the Better Connected report was valid at the time but isn't necessarily accurate come May 2009.  So on-going testing and monitoring of a website's quality and accessibility is essential.  Again something that plays to the strengths of automated testing  and is again not very cost effective when looking at manual testing.