- The facts about web analytics quality assurance
- The methodology usually employed to ensure quality
- The solution: WASP v1.0 coming up!
- Additional references: posts by fellow bloggers
- eMetrics Toronto: round table discussion on this topic and speaking about web analytics maturity
The factsJohn Lovett, from Jupiter Research, said that "42% of web analytics clients surveyed reported data accuracy as an important factor when selecting a vendor". Yet, a torough quality assurance of web analytics tags implementation is rarely part of the initial implementation and data quality is impacted as the site evolve and tags are left out of the loop.
I have personally conducted tests on dozens, if not hundred of sites, including vendor sites. So far, I have not found a single site that didn't have at least one of those common issues:
- Missing tags
- Missing data elements
- Using the wrong account id
- Following a semantic approach to page titles, links, site sections, etc.
- Passing the wrong data: invalid numeric values, string with invalid characters, long strings, etc.
- Not following the "domain of values" of some tags: for example, if you always send 1,2,3 to represent a membership levels, sending a value of "X" or "4" is an error.
MethodologyWeb analytics implementation is much more complex than most people think. Quality assurance remains a cumbersome and complex task. You basically have 4 alternatives:
- Blind faith: Trust and wish everything is right... or slight variation on the same theme... check only the pages that are part of your KPIs and funnels and hope for the best
- Crawlers: Use a high-end tool such as Maxamine (now Accenture Digital Diagnostics), which is very powerful and targeted at larger companies or something like Web Link Validator, which would still require a fair amount of technical know-how. For Google Analytics there's also SiteScanGA but it also has some limitations (incomplete crawls if the site isn't fully indexed by Google, basic checking, etc.)
- WASP: The Web Analytics Solution Profiler was specifically built for quality assurance of web analytics implementations. The sidebar offers page-by-page view of the tags as you surf and the crawler allows you to check a full site or section of a site.
The "Crawl from..." wizard will guide you through the steps to ease the process of crawling a whole site or a specific site section. Since the crawler runs "in context" of a real browser session, there is no better way to test. Options such as timeouts, filters, excluding your own traffic, pause/resume and data management policies makes it a unique approach to web analytics quality assurance of tagging implementations. The built-in data explorer makes it easy to spot untagged pages or wrong values being sent (see screen capture) and you always have the option the data to export to Excel and other formats.
Additional referencesSome references about the challenges of tagging:
- Manoj Jasra: Web Analytics Implementation: Items Overlooked
- Judah Phillips: Part 1: Your Web Analytics Data Quality May Stink and Here's Why!,
Part 2: Your Web Analytics Data Quality May Stink and Here's Why! and
Web Analytics and Data Collection: The Page Tag
The Ultimate Web Analytics Data Reconciliation Checklist
- Threads on the Yahoo! Web Analytics Forum: What is a good tool/methodology to audit a large site for tags?
Speaking at eMetrics TorontoI will be at the next eMetrics Marketing Optimization Summit in Toronto (March 29-April 1) where there will be round table discussions on various topics. I will be a facilitator for the discussion on "How Trustworthy is your Data?"
I will also be presenting about "Establishing your Online Analytics Maturity" in the "Jump Starting your Online Analytics Implementation: A Four Part Series". The session will be very interactive and you will get out of it with your own maturity assessment map! Get more information on the topic of web analytics maturity assessment here and here.