Tuesday, July 3, 2012

You say Analytics Maturity Quotient? I say wtf!

From: Jim Sterne
Subject: RE: What is your organization's Analytics Maturity?
Am I correct that you have seen this??

People in the web online digital analytics know of my work and my passion for the topic of analytics maturity.

I won't disclose the original source of the email Jim forwarded me in order to protect the "innocent". The reminder of the message is a nicely written marketing pitch about yet another maturity model from an agency... inviting you to fill out a survey in exchange for a promise to eventually get a summary of the results.

Never heard of them – and honestly, I frown when I see this type of article… Pointing to a bunch of high-profile company names and a well known and trusted news site. +1 for building credibility!

I’m surprised this news site even published such a superficial article… The survey as some blatant usability errors and is highly subjective. I don’t pretend the Online Analytics Maturity Model is perfect - far from it - but at least I don’t position it as a magical solution. Filling their survey gives you… nothing but a promise to get the results!

Oh! And there is a paper, conveniently called a "framework". I downloaded it – of course, after filling a lead form… and surprise, surprise! There isn't much there other than marketing fluff… It basically repeats the same article and the survey.

But worse of all. The article proposes a fancy math formula - a clever way to sound more scientific and rigorous:

L for Leadership, P for People, D for Decision Making Process (instead of P for Process, which would have been too simple... but that letter was already taken! doh!), etc. So you need 0.3 more people and 0.2D, and some sulfur and a fresh frog eye.

Where the hell is this formula coming from? What was the methodology to come up with it? What are the supporting arguments?

I have seen numerous models claiming to address “analytics maturity” but in almost every instance, those lack any serious research, supporting arguments, and end up being a lure for leads… I have even seen agencies doing a rip-off of my work without citing source or reference despite the fact I have always openly said the OAMM is there to be used, reused and abused. That’s how it can stand the test of fire – and I’m all for agencies and vendors to use it – as long as they cite the source and provide me with feedback if they think there should be any changes.

My take

It as been said the first benefit of a maturity model is the conversation it sparks. But please, PLEASE... do your homework and seek a model that have a minimal level of decency!

Some facts:

  • a summary of the OAMM paper is available (without any requirement to give your email or other info) and the full version is available upon request;
  • when you fill out the OAMM self-assessment - as did over 500 organizations so far - you get immediate feedback and we will not bug you with a sales pitch (unless you specifically ask for a follow-up!);
  • you can read about the numerous elements covered by the OAMM in my posts at Online-Behavior.com;
  • you can enroll in a workshop to learn more - next workshop at eMetrics Boston, September 30th;
  • you are welcome to use the model and expand upon it - the only thing I ask is you cite proper reference and if you have suggestions for improvement, please let me know!
In the coming days, I will make the OAMM self-assessment available as a widget. Agencies and vendors will be able to easily integrate it to their own site while making sure their data is available to them while contributing to the larger benchmark pool.

Post a Comment