How many monkeys does it take to create a dashboard? #bigdata #analytics #bi

monkeys

Serious question.  How many people are involved in creating a dashboard for your organization?  Probably a few out of information technology to get the requirements and several meetings to make sure the logic is correct.  Then you need a sql engineer or two to get the data out of the database because this isn’t one query.  This will take considerable time because they have no idea where the data is unlike your finance team.  Then you need a report designer to two to create it and this will take a bunch of time because they don’t know what the data means, unlike your finance team.  Oh, then you need this on multiple devices like Android, iOS and Windows, so throw in a couple more engineers.  Oh yeah, don’t forget you need the data test engineer too.   And the network security engineers to deploy, auditing people to make sure you’re not leaking data (unlike Clinton) and probably a few more others.  Oops, and don’t forget the business people.

Crazy to think about, but it is true.  It takes a fleet of people, no matter what business insights tool you are using.  From Microsoft, Tableau or Qlik or the thousand big data companies out there.  It is a team of people to develop insights for your business users.  And when they get the data, I bet it is out of date so no one will trust and look at it.  How much does all of this cost your organization every time the data updates or rules change?  It’s not $1,000 bucks, but tens of thousands of dollars a week.  Just think about that and well, someone did.

A friend of my posted this article from Blue Hill Research  on “attaching dollars to data prep”.  They estimate that 10 analysts cost $220,000 annually.   WAIT!  That is just the analyst!  Never mind the sql data engineers, report engineers, test engineers, audit people, trainers and the number of meetings (including doughnuts and coffee)!   Oh my goodness, big data is totally out of control!!

I recommend reading through their research and how they got to the number.  But if you truly want to use data for business insights, I don’t believe their recommendations are the way to go unless you have hundreds of thousands or millions of dollars to spend.  Remember, this is based on the old way of doing reporting and analytics, which every one of the companies listed in their report do it the OLD WAY.  There is actually ZERO innovation in getting the data faster, better and out to more people in short period of time with little investments.  The only innovation any of these companies bring are some new flashy new colors, and drag and drop and animated bars on the bar chart.

I believe it is time to throw out all these OLD analytical technologies, stop dumping hundreds of thousands of dollars down the I.T. hole and start building something that is radically new.   Stop and rethink what you are doing because if you don’t, you’re just another monkey running around in the jungle of big data.

Thanks for listening, Patrick

2 comments

  1. Glad you thought it was interesting. : ) RE: A friend of my posted this article from Blue Hill Research on “attaching dollars to data prep”. They estimate that 10 analysts cost $220,000 annually.

  2. Glad you thought it was interesting. : ) RE: A friend of my posted this article from Blue Hill Research on “attaching dollars to data prep”. They estimate that 10 analysts cost $220,000 annually.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s