The methodology involved at this time was only really as much as the available technology would allow. This wasn't very much and going back eight years at the time it was little more than a panel of Testing User participants in a classroom where their reactions and opinions were monitored. To draw meaningful conclusions still very much depended on some quite weak theories.

Anyway the point is that in 2003 I conducted Usability Testing studies for over 20 clients and was not asked once to include a comparative analysis in the Usability Test. I look back now and find it quite bizarre that at the time I didn't suggest or was asked for it. Why was this?

Well User Testing was very much in its infancy and I was only aware a couple of pieces of literature which even hinted at the possibility of Usability Testing as a comparative competitor analysis. Maybe I should have included it more from the off but at the time the only organizations remotely interested in Testing User analysis were e-commerce sites, and at the time very little trust was placed in e-business. If you had an e-commerce site the chances were that at least in the earlier you had automatic e-loyalty because those willing to give up their credit card details to the web in the first place were very unlikely to extend that trust to a competitors website. There was simply no point to compare your sites to a competitors. Now as time went on as we know this statement has become more and more invalid up until today where I sit writing this blog the opposite is true.

Comparative competitor analysis is essential. In depth competitor analysis forms a significant part of the Testing User experience and gives clients real insight into the successes and failures of a competitors website. User Testing to compare you User Testing your competitors web presence is now a very logical move as unlike eight years ago a competitive website is now a necessity.

The Testing user analysis contains in-depth competitor analysis as part of a multiphase approach.

Of course the nature of your competition will vary from client to client with the added difference that unlike eight years ago it's no longer a 'maybe' for an e-commerce site, but a serious analysis for organizations with even basic reference and resource sites.

How valuable though is the competitor data you will receive back?

In all honesty clients find the data concerning their own site to be worse than their competitors. Yes they will probably get the better User Testing feedback, but remember the motives you have for wanting to compare the two in the first place is probably based on the competitor having or are perceived to be having more success.