Wednesday, July 16, 2008

Selecting a web analytics solution: top criteria and a hint

I met John Lovett from Jupiter Resarch a couple of times and we had a few email exchanges about the state of the web analytics market. One of the discussion point was around the use of WASP for market and competitive research.

In his most recent post, "It's not the tool, it's the craftman", John's choice for the title stresses two important points:
  1. People are more important than the technology
  2. Craftsmanship marries the "science" and the "art" of doing web analytics

Top three criteria for selecting a web analytics solution

Based on John's research:
"Only one-third of analytics clients surveyed listed standard features and functionality as a priority when selecting vendors. Flexibility of reporting, ability to service business needs and accuracy of information all outweigh features in the buying deliberation process." John Lovett, JupiterResearch
1. Flexibility of reporting
2. Ability to service business needs
3. Accuracy of information

Accuracy of information!!!

This is among the top 3 criteria, yet vendor don't provide very good tools, and not much help, in making sure the data being collected is accurate in the first place. Of course we can trust their data warehouse and their tags to be of topmost quality. The issue is not there.

The issue is in all the implementations that are improvised or overly simplified. The implementations that are done by people who were just introduced to web analytics and read the vendor implementation guide, believing and hoping everything will work without a glitch. That's probably one of the factor why some people say "web analytics is hard" (ok, I hear you: don't go there again! one hot topic at a time - "engagement" - is enough! But read Jim Novo's friction model for a more down to hearth view).

Well known analytics practitioners have commented on the challenges of web analytics tagging. But what I'm witnessing usually revolve around the tagging itself:
  1. Missing tags, especially in transactional areas of a site (often non-template based)
  2. Improper values being assigned (special characters, duplicate page names, etc.)
  3. Polluting valid data with tests (development vs production and all kinds of tests)
  4. Lack of standardized taxonomies/domain of values
That's exactly the challenge WASP is gradually addressing, and this, regardless of the tool you are using. Other alternatives exists: use very technical tools with weird names like "proxies" and "debuggers" along with some black magic IT skills or put your hand in your pocket and get Maxamine hire Accenture. (BTW, looks like they should eat more of their own dog food, the link "Learn more about Accenture Digital Diagnostics" on the Maxamine Accenture Digital Diagnostic home page is broken! - sorry... I could not resist!)