Thursday, November 20, 2008

Web Analytics Maturity Model and Critical Success Factors

Update: White paper, self-assessment & more at
During the Industry Insight day at last month eMetrics in Washington we did a round table where we were tasked by Jim Sterne to answer three key questions:
  • What's the hard part?
  • What do we need from the tools?
  • How do we convince upper management?
Since I already took position regarding the concept of "web analytics is hard", I figured I should look at something else...

We got interested in "showing a long term goal and a map", basically, "use a maturity model to show what we can achieve". I had already used the excellent maturity model from Bill Gassman, of Gartner (click picture thumbnail) as a starting point with my clients. But speaking with Bill himself, and a few others, I came to the conclusion there could be a model looking further than the level of use of the analytical tools.

Critical Success Factors

What I want to develop is a web analytics maturity model encompassing the most important factors of success, basically, the Critical Success Factors of web analytics.

Wikipedia defines a CSF as:
Critical Success Factor is a business term for an element which is necessary for an organization or project to achieve its mission. They are the critical factors or activities required for ensuring the success your business. The term was initially used in the world of data analysis, and business analysis. For example, a CSF for a successful Information Technology (IT) project is user involvement.
Isn't it nice? A close relationship between data analysis and business success! While we focus a lot on KPIs, I think I have never heard anyone talking about CSF in web analytics!

The concept of CSF is also close to SixSigma, a methodology for continuous business process improvement by eliminating defects, and a strategy I recommend for web analytics.

Web Analytics Critical Success Factors

As a web analytic advocate, I get asked about Google Analytics vs other tools all the time. I kept on saying "it's not about the tool, it's about the craftman", but that wasn't enough. I think the tool should really be the last of our worries.

So now, I use a Web Analytics Maturity Model of my own to evaluate the current "state of the union" and manage more realistic client expactations. And the results are shocking!

It's a work in progress, but I have so far identified six critical elements of success, each scaled from 0 to 5:
  1. Management: do you have a web analytics champion? At which level within the company?
  2. Ressources: are you staffed correctly? Do they have the right background and training?
  3. Objectives: can you define SMART web analytics objectives?
  4. Scope: do you aim to do web analytics to measure the marketing performance of a micro-site or reengineer your business processes to optimize your online business?
  5. Process and methodology: do you improvise or use a proven methodology for continuous testing, learning and improvement?
  6. Tools and their use: now it's time to talk about wich tool is the best... for this specific situation!
I have defined a set of criterias for each dimension and scale. I'm actually working on developing recommendations for each scenario.

Using the model

Tell me, where is web analytics most succesful?

The first one is theory, the second one is a real case!

Out of the initial evaluation, I can quickly identify:
  • if the current web analytics culture is at risk
  • if the goals (the reason the client called me in the first place!) is realistic
  • recommendations that goes beyond selecting a tool and implementing it...
  • (if I really want to work with them!)

The next steps

First and foremost, as a web analytics advocate interested in the field of education & research, it's an area I want to explore further. In the coming months I will evaluate each of my clients and others who are interested. I will also present this topic at eMetrics Toronto, on March 29th, and a longer version at eMetrics Sam Jose on May 7th.

To finance this study, I'm also offering to evaluate your own business and deliver a comprehensive analysis & recommendations: contact me if you are interested.