Big Data. Little Pictures.

I had the opportunity to speak about content and understanding at the Explainathon, hosted & mosted by Adam Zais of Wistia, on May 22. (Yeah, my birthday.)  No doubt the video of the talk will end up here.

One remark along the way - "Why is it that the answer to 'Big Data' is 'Little Pictures" prompted a followup question via Twitter.  My response was 'because screens are so small right now.'  It was a wisecrack, but I think it's a serious (and darn interesting) topic. 

Big data is one of the unexpected consequences of the digital shift.  Let me put it this way: you viewing this blog post generated at least a hundred data points - most of them probably invisible to you.  

  • We use a brilliant tool called Woopra to see web-site visits in real time.  Your IP, location, and everything else your browser advertises about itself, are captured there. (Yeah, you with the huge monitor - we know the resolution!)
  • We run Google Analytics.  All that stuff is trapped there as well.
  • Google itself grabbed a bunch of data if you got here via search. Bing, ditto.
  • Our web host probably logs most of the same stuff.

And that's just scratching the surface.

So I'm supposed to find the meaning in all of that (multiplied by thousands, across months) by staring at a flat screen?  Little pictures are the best we can do - for now.

I expect to see visualization frontiers expanded rapidly in the next @7-10 years. We'll start cost-justifying solutions that enable us to see more, based on the value of pattern-and-meaning to be had from all that big data.  There's an optimistic and humanistic statement mixed in there.  I think the job of finding (ascribing?) meaning will be human-first machine-second, for quite a while, and I think it'll be primarily a visual job.  Once we know what we're looking for ("look" - a visual reference...), we'll tell the machinery to go look for more stuff like that.

Or maybe this is just a big rationalization for those 29" Dell monitors that would fit SO nicely on my desk.