I have had a number of conversations recently about the performance figures produced by our scheduled (weekly or monthly depending on the subscription) reports, Page Credits or Immediate Reports not being consistent.

Any performance measurement is only ever a snapshot view at the time the measurement is taken.  So even comparing measurements that are only one minute apart could give quite different values.  There are just so many things that can affect the timing.  So trying to do direct comparisons between two measurements is fraught with dangers.  Even looking at trends is not ideal because the sampling frequency is too low and we run the reports at different times of the day.  What you can do with our measurements is get an indication of where any performance issues might be.  Whether it's server response times or download speeds.  There's a posting on this subject at (http://blog.sitemorse.com/sitemorse-features/performance/).  We are currently looking at improving the accuracy of the download speed figures and I'll post something on this in the coming weeks.

A really simple reason for a difference might be whether the page is cached or not.  If it's not then the first request will take a little longer than normal and if you test again one minute later then the page will be cached and so will be faster.

It could be the number of users on the site.

I could ramble on for hours about the different things that come into play when you start to think about the implications of the number of concurrent users on a website.  But in effect each additional user puts it's individual load onto the site.  All sites have a "tipping point" where performance deteriorates at a greater rate than with fewer users - so a site might be fine with 200 concurrent users but 210 adds 20% to the response times and 250 doubles the response time.  and as soon as response times increase significantly the number of concurrent users increases because each user takes longer to do what they want to do so they are still on the system when the next lot of users log-on.  And then performance gets worse and so on and so on.

Configuration

You can get even simple things like a parameter in the Webserver that limits the number of concurrent users  (in order to mitigate the effects of what I describe above) to say 200.  If you're number 201 then you have to wait for a free slot.  It'll only be a very short wait but, with everyone expecting sites to respond very quickly these days, it may feel like a long time.  And there are lot's of parameters throughout your infrastructure that can affect response times.

Internet "travel" time

Only a minor factor if you're connecting to a UK site from the UK.  But even so the "route" that the IP packets that make up your web pages will probably take a different route from the Webserver to the browser.  In fact, even individual requests for ITEMS from the page might be large enough to be spilt into multiple IP packets and each might take a slightly different route.  Then you might have a glitch with one of the packets that might need to be resent (all handled by the IP protocol) which will affect the response times.

If you then introduce a factor like one test was run on the Development system and one on the Live system and, even though they are identical pages, we introduce a whole host of other factors that affect response times.