When designing a web site, we need to understand the demographics, psychographics and other characteristics of our audience. Elements to consider includes such technical considerations as the bandwidth, browser, and computer being used.
In this example, I will use Omniture SiteCatalyst but the same type of analysis could be done with any solution. For the sake of this demonstration, let's assume it's a B2B site, so visitors comes from very specific partner companies.
I was faced with a very good case where analysis, instead of reporting, made all the difference. Take a simple statement: 15% of the visitors to the web site use Windows 2000/IE 5. We know this could lead to slightly different design considerations.
If we are simply reporting, we're stuck with 15%... is it worth designing for this platform? How can we tell?
Since we suspect Windows 2000/IE 5 use might be declining, let's look at the usage trend for our visitors. We already have a clearer indication, going from nearly 16% down to 12% in 5 months. Plus the fact Omniture can tell us the Internet average is about 6%, indicating our site receives a larger than average share of Windows 2000 visitors.
Could there be just a handful of visitors (companies) using Windows 2000 and influencing this reporting? Now we need to create a Custom Datawarehouse Report using Omniture's Advanced Analysis. We can ask for the number of visits by OS, grouped by Company, with a breakdown by month. Once we get the CSV file loaded in Excel, a simple Pivot Table can give us a very good look at what is going on.
Now we can spot that only a couple of companies, some of them representing a fair share of the traffic, have much higher usage of Windows 2000 (42% in the above example). This makes sense because it appear one of those companies is a banking environment, and we know those corporate environment traditionally takes longer to adopt more recent technologies. Furthermore, we see there's also a clear negative trend, from 122 visits in October to 54 in February (while XP is generally growing).
Now that we have done a more thorough analysis, we're in a much better position to decide if we need to invest in designing for this platform. From a simple report stating "15%" with very limited context, we were able to identify more detailed behavior and address the impact on a limited subset of our visitors.
In the spirit of the 10/90 rule coined by Avinash Kaushik, getting the number was easy. The analysis is where the value is: how is the site structured? who's your audience? which other reports and metrics will help you get the right decision? Too often, people are complaining about the tool: it's not working, it's not configured correctly, it doesn't give me my numbers... I guess that's where experience and an analytical mindset can make all the difference.
(Ultimately, the recommendation was to avoid integrating costly exceptions in the design and the development process.)
Named one of the most influential industry contributors by the Digital Analytics Association. With over twenty years’ experience empowering organizations to analyze and optimize their online channels, Stéphane has cemented his position as a leading voice for online analytics and optimization.
3 comments:
Hi Stéphane
First it pains me to see anyone have to use a pivot table to drill down into their web analytics data. I think there is a reason MS Excel has never been sold as a web analytics tool. But hey thats another topic.
I agree that investigating the data using analytics tools is vital when a business question has to be answered or an enomaly explained.
However I think that web reporting is the single most thing that organisations need. Can you imagine a finance director trying to create a pivot table using data from Omniture? Hmm...I think this is where dashboards and emailed reports come in which is all just simple reporting.
It can be very enticing to start slicing and dicing your data with every dimension and metric available but if you don't have the inital report that showed that blip or downward trend then you could very easily get lost in the sea of data.
Matt Hopkins
http://www.webanalyticmatt.com
Thanks for the comment Matt. I agree with you, we shouldn't rely on an Excel download and a Pivot Table to analyze data... I used Websidestory HBX in the past and it's very easy to do segmentation and correlations (that's really one of their strength). In my example, Omniture Discover would have been a better solution than an Excel download. And I can't agree more that dashboards and reports are very important. I'm not sure I would give direct access to a finance director (it depends on the "maturity" of the organization...) but for sure, you're right we need the first report that will blip on our radar. But then, after that anomaly is spotted, we need someone to dig more and explain it. And to me, that's where analysis, more than reporting, makes all the difference.
Hi Stéphane
Super post – I am in general a great fan of factual posts that exemplify in detail! – And as a vendor I partly disagree with the former comment that it is a bit sad that one have to use Excel! :-) I actually find that necessary for a number of detailed analysis’s – where the tools would be far to cluttered if one were to accommodate ALL potential analysis options. (we would not want IndexTools., Omniture or Webtrends to turn into Excel)
Cheers
Dennis
Dennis R. Mortensen, COO at IndexTools
My take on: Web Reporting vs. Web Analysis!
Post a Comment