Performance is a crucial issue for anyone who runs a company website - the rewards can be great and the consequences of failure can even impact the organisation's 'bottom line' financial performance. 

But a website's performance does not just depend on the power of the web server involved. There are a number of elements that will affect the performance of any site, starting from the very first decisions made. 

Managers with responsibility for operating a website should also beware of internal testing that can often mask performance issues  - tools that are part of the content management system or run by the company's IT department often can see the website from the inside, rather than from the point of view of the end user.

At Sitemorse, we often liken the performance issue to a car . There is no point in comprehensively testing the performance of a car if it's running, for example, on the wrong type of fuel. Or if the wrong tyres are fitted. A flat tyre will hamper performance even more!

A website is only usually as good as its weakest link, so a badly-planned site, like a badly-designed car, is going to be left behind by its competitors before the end of the race. 

Problems often come down to the same old things - bad links, images larger than necessary, poor code that makes the site work harder than it needs to. 

Talking to potential clients, we are often surprised to hear they are not finding problems with their websites. Frankly, this issue is quite well-known, and we are not the only ones testing, and finding the same problems. One manager we spoke to recently said we must run our tests on their 'bad days' , but it does seem unlikely that major companies would allow the same level of carelessness in other important areas of communication, such as annual reports. 

The real problem here is the nature of the web, a much more complicated operation than printing and distributing documents and literature, for example, where standards have been gradually built up over a long period. 

Defining a precise colour for a brochure, for example by giving a pantone reference , is not feasible for the web, because the huge variety of monitor set ups, lighting etc mean that colours are experienced differently by different users. 

The example holds for almost everything about a website - there are probably more contributors to a website than to any brochure, perhaps scattered internationally or across a wide area, using different set-ups, equipment, browsers, just to quote a few examples. 

A good web manager knows a little about lots of things, and needs to develop his or her own way of developing checks on everything they are told from design and technical agencies, fellow employees, members of the IT and other specialist departments and even company management. Few have the big picture when it comes to a website, and there can be a tendency to not see the wood from the trees, particularly when all the messages are good ones ( such as, "our new site is online and breaking all records". 

The Sitemorse Web Managers Toolkit provides hard evidence of problems and can be a very useful check for web managers and editors wanting to see their site from the users point of view but who don't want to spend their entire day clicking on links or running free tools over the site. 

Our free Snapshot tool will warn you about performance issues found on any tested webpage. The Performance icon gives information of the elements of the page being tested that can affect load, such as large images, for example. Clicking this icon will give a more detailed report so the web content editor can make the necessary changes quickly - and then re-test the page to ensure the changes have fixed the problem.  The free service can be extended to check for IP/trademark infringement and alert you to brand issues or spelling problems. 

Snapshot is not a download but requires saving a small 'bookmarklet' - Get started with Snapshot.

Geoff Paddock is a web consultant who has managed corporate websites for ICI , Wolseley plc and a number of less well-known clients.

We are always coming across comments from prospective customers about the vast capabilities of CMS products,  Mostly it's about how they don't need to use Sitemorse because their CMS produces error free, accessible and standards compliant pages.  Most of us know that that isn't the case and that however good a CMS is there will still be problems to resolve that Sitemorse is ideally suited to help with.

Recently I've seen performance mentioned as something that certain CMS vendors are claiming they can help monitor.  Some CMS vendors (for example, PaperThin and Sitecore) provide built-in reporting capability for determining time-to-render for various content elements.  Now that seems very strange to me.  And the biggest problem I have with the claims is that render times is a small part of the performance of a Webpage and something that is running within your infrastructure can't get a "REAL" picture of what performance an end user of your site is experiencing.  You need to be out there on the internet to experience that.  So for me they could, at best, only tell you a small part of the story - one which if you take a look at the performance of your website from your office PC you'd get the same view.

The reality is that your website visitors don't live in your Data-centre and so aren't sitting with a high speed LAN connection to your Webservers.  They use a variety of technologies to connect to your website all of which can have an influence on THEIR experience of the performance of your website.

Diagram courtesy of Gomez Inc. 

As you can see the further away from your Data-centre (where your CMS is located) the more, potentially dodgy, technologies are used - like mobile devices or dial-up connections -  more issues start to arise with regard to both performance and availability.  So, just as a CMS can't offer you the whole solution to your quality and compliance issues it can't offer you the whole story when it comes to measuring your website's performance.

Further to my pointers about how to improve the performance and quality of your website and my blogs on Google's musings about a website's speed affecting it's rankings I thought a few more pointers about things to consider when looking at improving the speed of your Webpages might be helpful.

Reduce File Sizes

You can find out the sizes of your files by looking at the "Site Inventory" page in your Sitemorse Audits or looking at the "Page assets" info of a Snapshot report and see the impact on performance by looking at the "Performance" info.

Use a HTML Compression tool

There are multiple methods of reducing the time it takes to send a file from the server to the client. Gzip is a compression tool used on servers to compress files in order to save those precious kilobytes. It is the most popular and effective compression method at this time, reducing the response size by about 70%. It can be used to reduce the size of any type of file, however as images and pdf files are already compressed, it is typically best not to attempt to compress these with gzip as there may be loss of quality and can potentially increase file sizes.

Don't Scale Images In HTML

Just because you can set the width and height of an image in HTML does not mean you should! If you want to display an image that is 100px wide and 100px high, then the image should be 100×100px rather than a scaled down 500×500px image! This will reduce the size of the image therefore make it load faster.

Optimise your CSS and JavaScript

Removing unnecessary code from JavaScript and style sheet files will reduce the file size, thereby improving load times. Minification takes this a step further by removing all comments, new lines, tabs and spaces. This improves response time as the size of the downloaded file is reduced drastically. A popular tool for minifying JavaScript code and CSS is YUI compressor.

Reduce Server Calls

You can find out the number of CSS and JavaScript files by looking at the "Site Inventory" page in your Sitemorse Audits or how many are on a particular page by looking at the "Page assets" info of a Snapshot report.

Combining style sheets (and JavaScript files for that matter) into as few files as possible will reduce the number of calls being made to the server. Combining files is more challenging when the style sheets and scripts vary from page to page, but the improvement in response times makes it well worth the effort.

CSS Sprites combine background images into a single image, reducing the number of image requests, and using CSS can show only the parts as desired using the background-image and background-position properties.

Script Locations

You can find out the sequence of how files are loaded by looking at the "Performance" info of a Snapshot report as well as seeing the impact they have on the page's load times.

Where you import your CSS and JavaScript can make a huge difference on how long a page takes to load, and how long it appears to take.

CSS At the Top

Moving style sheets to the HEAD makes pages appear to be loading faster as the page will render progressively. Not only is this stated in the HTML specifications, but by placing them near the bottom of the document, many browsers will be unable to render the page correctly as these browsers block rendering to avoid having to redraw elements of the page if their styles change.

JavaScript At The Bottom

Browsers are able to download from multiple sources at the same time (usually two downloads in parallel per host) allowing a page to load a smidgen faster. The problem with JavaScript is that they block these parallel downloads. Moving them to the bottom of the page gives everything else time to load. Sometimes this is impractical as some script may be needed in order to insert content into the page, for example if the script uses document.write. A work-around for this is to create a function at the bottom of the page and use an on-load command, where you want the content, to call a function when the page has finished downloading.

Flush

In PHP the function flush() allows a partially ready HTML response to be sent back to the browser. This is useful while the back-end is putting together the rest of the HTML page and is most noticeable on busy back-ends, where a script requires a lot of time to pull together a lot of resources and complete making the page before sending it to the browser. A good place to consider flushing is right after the HEAD as this is usually the easiest part of the page to create and allows the browser to start including any CSS and JavaScript files the browser requires while it waits for the rest of the HTML page, which the back-end is still constructing.

For example:
...<!-css, js->
</head>
<?php flush();?>
<body>
...<!-content->

I saw this on the techradar.com website on Monday 16th.  It looks as if it is clearing up the confusion and then confirms, in the last paragraph, that the confusion still exists.  But useful as it does mean that if it isn't already, then speed will be an issue for 2010.

Google has stated it is considering using the speed which a website loads in its search algorithms, with slower sites set to be pushed further down the results page in favour of quicker ones.

Google's Matt Cutts revealed the news in an interview with WebPro, admitting that 'a lot of people' within the company favoured adding a speed factor into the way in which a page is ranked.

"We're starting to think more and more about should speed be a factor in Google's rankings," said Cutts

A good experience

"Historically, we haven't used it in our search rankings, but a lot of people within Google think that the web should be fast - it should be a good experience," he added.

"So it's fair to say if you are a fast site maybe you should get a little bit of a bonus and if you're a slow site then, you know, users don't want that as much."

Already a factor?

The speed of a website and the way in which it is affected by Google rankings is already a hot topic on the web, with .Net editor Dan Oliver explaining to TechRadar that many believed it was already a factor.

"It's great to hear that speed is going to play a part in Goggle's algorithm for 2010, but we think it may have been a factor for some time.

"In fact, Google's communication and public affairs department told .net magazine months ago that 'all other things being equal, it's fair to say that a speedy site will rank better'."

Performance- resolving some of the issues

Take a look at my blog post http://blog.sitemorse.com/2009/10/its-about-more-than-just-cpu-c.html on using information provided in a Sitemorse audit to improve your site's performance

Akamai Technologies, Inc., the leader in powering video, dynamic transactions and enterprise applications on-line, recently released key findings from a commissioned study (www.akamai.com/2seconds) conducted by Forrester Consulting on behalf of Akamai examining eCommerce web site performance and its correlation with an on-line shopper's behaviour. The most compelling results reveal that two seconds is the new threshold in terms of an average on-line shopper's expectation for a web page to load and 40 percent of shoppers will wait no more than three seconds before abandoning a retail or travel site.

Additional findings indicate that quick page loading is a key factor in a consumer's loyalty to an eCommerce site, especially for high spenders. 79 percent of on-line shoppers who experience a dissatisfying visit are less likely to buy from the same site again while 27 percent are less likely to buy from the same site's physical store, suggesting that the impact of a bad on-line experience will reach beyond the web and can result in lost store sales.

In 2006, Akamai conducted a similar study (www.akamai.com/4seconds) to understand consumer reaction to a poor on-line shopping experience. The 2009 study is a follow up to examine how customer expectations around on-line shopping have evolved in the past three years.

Based on the feedback of 1,048 on-line shoppers that were surveyed, Forrester Consulting concluded the following key findings:

• Consumers become impatient when pages take longer than two seconds to load. 47 percent of consumers expect a web page to load in two seconds or less, representing a significant evolution in consumer expectation over the 2006 study, which showed customer expectations at four seconds or less. Forrester found that 40 percent of consumers will wait no more than three seconds for a web page to render before abandoning the site.
• On-line shopper loyalty is contingent upon quick page loading, especially for high-spending shoppers. 52 percent of on-line shoppers stated that quick page loading is important to their site loyalty, up 12 percent from the 2006 study.
• Shoppers often become distracted when made to wait for a page to load. 14 percent will begin shopping at another site, and 23 percent will stop shopping or walk away from their computer.
• Retail and travel sites that under perform lead to lost sales. 79 percent of on-line shoppers who experience a dissatisfying visit are less likely to buy from that site again, up 17 percent from the 2006 study. 64 percent would simply purchase from another on-line store, up 16 percent from the 2006 study.
• Mobile is an emerging shopping channel, and performance is a key to consumer adoption. While only 16 percent of consumers have shopped via mobile or smart-phones, consumers are interested in using these devices for research and shopping activities in the future. One third of consumers report wanting to shop via their smart-phones in the future.

The study also shows that consumers continue to be frustrated by on-line shopping sites' performance, which leads to an increase in site abandonment. 23 percent of dissatisfied on-line shoppers attributed their dissatisfaction to the web site being too slow or taking too long to load, while 57 percent of on-line shoppers insist on a rapid checkout process, up nearly 10 percent from three years ago. One-third of shoppers who abandoned a recent shopping session were dissatisfied with a retail or travel site's performance. These results demonstrate that a poorly performing web site can be damaging to retailer's brand and reputation.

"Today's consumers demand a fast, engaging and secure on-line shopping environment when searching for a product on-line. We see a direct relationship between on-line revenues and site performance and therefore, we have to ensure our site performs well and loads fast" said Michael Cooper, Vice President and General Manager, Home Depot.com

The Forrester study also states that usability, site content and speed are key factors to on-line customer loyalty. By taking the time to improve the overall site content and functionality, retailers can bolster the overall brand and image of their company. For example, adding rich, interactive and dynamic content to a retail site can help reflect the in-store experience for the consumer, which can lead to increased satisfaction and loyalty from the consumer.

"The takeaway from this study is that site performance remains a major factor for keeping visitors coming back to a retail site. On-line shoppers demand - and expect - quality site performance which is a requirement for optimal on-line success," said Pedro Santos, Chief Strategist for eCommerce at Akamai. "With two seconds as the new benchmark for a retail or travel site to load, it leaves little room for error to maintain a company's loyal on-line customer base."

To read and download a complimentary copy of the full study, please visit www.akamai.com/2seconds

People comment about the fact that we check (and probably more importantly score) external links on the websites we audit.  So I thought a blog posting on our thoughts behind why we do this would be helpful.

When we "look" at a web page during an Audit, we run all of our checks on the page. We then check all of the links on the page.  If it's an external link we simply download the Base Page of the linked page and do two things - a) check that the link is working and b) check the performance.  We do this because these are links from your pages that people may click on and we consider that any link on one of your pages, whether it is an internal link to another page on your site or to one of your partner sites, is important - otherwise why link to it ?  It's also based on the premise that if people click on a link on one of your pages they are not necessarily going to go into any deep thought process about who's responsibility it is if that link fails or is horribly slow.  It's on your page and therefore it's YOUR fault.  Once they get onto another site's page it's much more obvious that it's someone else's site and any subsequent problems are theirs not yours.  Which is why we don't fully audit external pages but simply check the link to it and it's performance.

A typical view is "I can understand flagging external response errors such as 404s because we have a control over that - we either link or not".  Then surely this is equally true of a page who's performance is consistently awful - link or don't ? Like everything in Sitemorse we are attempting to report things that are useful to you.  A report that tells you what the 10 slowest items on your site are is undoubtedly a useful piece of information that you can decide whether or not to act upon.   (It isn't a subjective measure from our point of view, we're simply taking the 10 slowest items whilst making no judgement on them as to how slow they are.  They are just slower than everything else we tested during the audit.)  And whether it's an internal or external page, it's just the remedy that's different.  If it's a really important link that's always slow then there's not much to be done but it doesn't mean you shouldn't be made aware of it. 

Let's not confuse the scoring that we do with the value of the information that we provide.  We appreciate that people are focussed on how well they do in the Surveys and whether their scores are improving.  But let me put this in perspective,  When we are calculating the average response times and average download speeds from which we calculate the performance score we take the timings from every single GET request we issue as we traverse the site.  Let's say we issue an average of 10 GET requests per page and we audit 250 pages.  That's 2,500 GET requests.  One external link that is a bit slow won't shift the average speed figures by any significant amount and will have NO impact on the performance score for that audit run.  e.g. If the average response time across the 2,500 requests was 0.5 seconds it would need almost 10% (that's 250) of them to change to 1.5 seconds in order to change the average to .6 seconds.  So one, two or ten slow external links will make no difference to the score.

Another Blog posting on external links called "Broken links are broken links" is here http://blog.sitemorse.com/2008/06/broken-links-are-broken-links.html

 Continuing the theme of Performance that I've covered recently I have seen a number of surveys recently that highlight the impact of poor performance on your business.

I have had a number of conversations recently about the performance figures produced by our scheduled (weekly or monthly depending on the subscription) reports, Page Credits or Immediate Reports not being consistent.

Any performance measurement is only ever a snapshot view at the time the measurement is taken.  So even comparing measurements that are only one minute apart could give quite different values.  There are just so many things that can affect the timing.  So trying to do direct comparisons between two measurements is fraught with dangers.  Even looking at trends is not ideal because the sampling frequency is too low and we run the reports at different times of the day.  What you can do with our measurements is get an indication of where any performance issues might be.  Whether it's server response times or download speeds.  There's a posting on this subject at (http://blog.sitemorse.com/sitemorse-features/performance/).  We are currently looking at improving the accuracy of the download speed figures and I'll post something on this in the coming weeks.

A really simple reason for a difference might be whether the page is cached or not.  If it's not then the first request will take a little longer than normal and if you test again one minute later then the page will be cached and so will be faster.

It could be the number of users on the site.

I could ramble on for hours about the different things that come into play when you start to think about the implications of the number of concurrent users on a website.  But in effect each additional user puts it's individual load onto the site.  All sites have a "tipping point" where performance deteriorates at a greater rate than with fewer users - so a site might be fine with 200 concurrent users but 210 adds 20% to the response times and 250 doubles the response time.  and as soon as response times increase significantly the number of concurrent users increases because each user takes longer to do what they want to do so they are still on the system when the next lot of users log-on.  And then performance gets worse and so on and so on.

Configuration

You can get even simple things like a parameter in the Webserver that limits the number of concurrent users  (in order to mitigate the effects of what I describe above) to say 200.  If you're number 201 then you have to wait for a free slot.  It'll only be a very short wait but, with everyone expecting sites to respond very quickly these days, it may feel like a long time.  And there are lot's of parameters throughout your infrastructure that can affect response times.

Internet "travel" time

Only a minor factor if you're connecting to a UK site from the UK.  But even so the "route" that the IP packets that make up your web pages will probably take a different route from the Webserver to the browser.  In fact, even individual requests for ITEMS from the page might be large enough to be spilt into multiple IP packets and each might take a slightly different route.  Then you might have a glitch with one of the packets that might need to be resent (all handled by the IP protocol) which will affect the response times.

If you then introduce a factor like one test was run on the Development system and one on the Live system and, even though they are identical pages, we introduce a whole host of other factors that affect response times.

There are several ways we measure and report performance in the Sitemorse reports and by looking through some of the client feedback in our recent survey it may be useful to clarify what we do, how we do it and how we report what we find.

The main thing to remember is that we are taking a snapshot of a sites performance.  i.e. if we provide you with a monthly report then the performance figures only relate to the time we were running the report.  We can't definitively say it is representative of your sites performance.  BUT we don't select particular times to run your reports, the times will vary as will the day of the week.  So it should be reasonably representative.

We produce two sets of figures the average figures over the whole test and the figures for the Home Page.

Let's look at the Home Page results first.

We look at the total response time and the speed of the download.  The overall time is obvious how we gather.  Download speed is captured by starting a timer each time we start receiving the first block of data in response to our GET requests and stopping it when the last byte is received.  We know the size of what came down and how long it took so dividing one by the other tells us how many kilobytes per second were received.

Point to remember - line speeds are always quoted in kilobits per second and web page sizes are always quoted in kilobytes.  So you need to divide the kilobits figure by eight to get a rough idea of how the download speed relates to your page sizes.

We then match those against what we consider reasonable times for people connecting via a modem, ADSL or a corporate LAN.  We set these thresholds as 14 seconds, 6 seconds and 4 seconds respectively.  From a modem point of view the limiting factor is the modem.  It can receive data a maximum of about 5 kilobytes per second.  So if we're aiming at 14 seconds you will ALWAYS fail this test if your Home Page is larger than 14 x 5 = 70 kilobytes.  Most Home Pages these days are larger than 70KB so they fail our test.  That doesn't make the test useless, it just means that you have the information about how your Home Page performs for the small number of people that use a modem to browse your site.  You can make the decision to take account of this point and reduce the size of your Home Page, or ignore it.

A 512 kilobit ADSL connection will download at approx. 64 KB per second and a one megabit corporate connection will run at about 128 KB per second.  So if your connection is faster than these you have to look at page sizes of less than 384KB for ADSL and 512KB for corporate to ensure that you pass these two.

We have considered changing these thresholds to reflect the changing landscape of broadband usage and connection speeds but there's so much controversy about what speeds people are actually getting AND quite a lot of sites fail the current tests as they are, so updating them would increase the failure rate.  We'll look at changing them when a high percentage of sites are passing the current tests.

The average figures are captured by starting a timer when we issue a GET request and stopping it when we see the first byte of data.  This gives us the approximate time your infrastructure took to respond. We then start another timer which we stop when the last byte is received.  We do the division and come up the the download speed.  These two figures are then averaged over every GET request done throughout the entire report.  The objective is to attempt to give you an idea of where any performance issues lie.  If you have an average RESPONSE time of 45ms you do NOT need to look at your infrastructure.  Likewise if you have a DOWNLOAD SPEED of 560 kilobytes then you can smile broadly.

If we report RESPONSE times averaging 1350ms the only further information we can give you is the slowest TEN items.  Because we are looking at this from out on the internet we can't gather more detailed info on whether it's your Webserver, Application server or Database server etc that is causing the bottleneck.

More information on how we score sites for the various tests is available here secure.sitemorse.com/benchmark.html