Tuesday, July 3, 2012

You say Analytics Maturity Quotient? I say wtf!

From: Jim Sterne
Subject: RE: What is your organization's Analytics Maturity?
Am I correct that you have seen this??

People in the web online digital analytics know of my work and my passion for the topic of analytics maturity.

I won't disclose the original source of the email Jim forwarded me in order to protect the "innocent". The reminder of the message is a nicely written marketing pitch about yet another maturity model from an agency... inviting you to fill out a survey in exchange for a promise to eventually get a summary of the results.

Never heard of them – and honestly, I frown when I see this type of article… Pointing to a bunch of high-profile company names and a well known and trusted news site. +1 for building credibility!

I’m surprised this news site even published such a superficial article… The survey as some blatant usability errors and is highly subjective. I don’t pretend the Online Analytics Maturity Model is perfect - far from it - but at least I don’t position it as a magical solution. Filling their survey gives you… nothing but a promise to get the results!

Oh! And there is a paper, conveniently called a "framework". I downloaded it – of course, after filling a lead form… and surprise, surprise! There isn't much there other than marketing fluff… It basically repeats the same article and the survey.

But worse of all. The article proposes a fancy math formula - a clever way to sound more scientific and rigorous:

L for Leadership, P for People, D for Decision Making Process (instead of P for Process, which would have been too simple... but that letter was already taken! doh!), etc. So you need 0.3 more people and 0.2D, and some sulfur and a fresh frog eye.

Where the hell is this formula coming from? What was the methodology to come up with it? What are the supporting arguments?

I have seen numerous models claiming to address “analytics maturity” but in almost every instance, those lack any serious research, supporting arguments, and end up being a lure for leads… I have even seen agencies doing a rip-off of my work without citing source or reference despite the fact I have always openly said the OAMM is there to be used, reused and abused. That’s how it can stand the test of fire – and I’m all for agencies and vendors to use it – as long as they cite the source and provide me with feedback if they think there should be any changes.

My take

It as been said the first benefit of a maturity model is the conversation it sparks. But please, PLEASE... do your homework and seek a model that have a minimal level of decency!

Some facts:

  • a summary of the OAMM paper is available (without any requirement to give your email or other info) and the full version is available upon request;
  • when you fill out the OAMM self-assessment - as did over 500 organizations so far - you get immediate feedback and we will not bug you with a sales pitch (unless you specifically ask for a follow-up!);
  • you can read about the numerous elements covered by the OAMM in my posts at Online-Behavior.com;
  • you can enroll in a workshop to learn more - next workshop at eMetrics Boston, September 30th;
  • you are welcome to use the model and expand upon it - the only thing I ask is you cite proper reference and if you have suggestions for improvement, please let me know!
In the coming days, I will make the OAMM self-assessment available as a widget. Agencies and vendors will be able to easily integrate it to their own site while making sure their data is available to them while contributing to the larger benchmark pool.