How to resist feeling overwhelmed by big data

For many of us, our data challenges are so much more mundane than for enterprises, and in many ways, the same as they ever were, albeit much more digital than ever before. We have our “networks” of friends, family, colleagues and acquaintances, we have our memories and our libraries, we have our goals and ambitions, and we have our day-to-day “operations”.

Social networks have certainly grown the potential of data in our lives as connected people. We can know so much about people if we want to, but are we any more able, as people, to hold and process all this extra data? Do our social networks actually help us, or do they simply overwhelm us with opportunities to interact? Is the only way to accommodate all this social big data now available to us, to clear some “brain space” by doing less of something else? The suggestions offered by the social media interfaces help only to acquire more friends, or follow more people, they don’t seem to help you have better relationships with what you’ve already got. In short, they make the “problem” worse, not better.

Our memories and libraries are potentially much larger than before – digital photographs, home movies, music and video are much easier to acquire and store in quantities that would soon have overwhelmed their physical analogues. Media publishers and distributors have been quick, and reasonably adept, at grasping the opportunity to understand our tastes – or at least our habits – and provide recommendations of what else we should watch, read and listen to. 

Our goals and ambitions perhaps remain very much the same – travel, health, security, fun, and so on. But from Weight Watchers to Nike FuelBand, we are now able to gather and use vast data about what we eat, what we do, and how healthy, active and fit we are. Looking after ourselves has never had so much data before – levels of data that just a few years ago were within the reach of only the most elite athletes. For most of us, it’s just overwhelming, and often the most popular and successful services are those that abstract and simplify what is essentially still science into bite sized concepts such as ProPoints or NikeFuel points. 

Personal finance, grocery shopping, home energy, etc. are often utility activities and choices we find it hard to get excited about. For years our banks have faithfully sent us pieces of paper every month, telling us exactly how we acquire and spend our money. But most of us still can’t answer the question: “where does all my money go?”! Of course we could answer it, if only we could be bothered, and just like smart energy meters, or our digital music collections, bank statements online give us so much more capability to explore and use this data than a file of dusty paperwork. The data is there, but how to understand and benefit from it?

I’ve broken it down into a few simple steps:

  1. Observation –the vital first step in making use of big data is to make a simple summary of what is happening.

  2. Discovery – from studying and summarising the data, one would hope that you notice (discover) something you had not realised before.

  3. Insight – this happens when your discoveries begin to coalesce around something of value, when you identify an opportunity to improve your situation. Insights are different from observations and discoveries because you can do something about them.

  4. Taking action – the successful applications will help overcome the mental fatigue of big data, by making simple, actionable recommendations.

  5. Perhaps we’re not that much smarter than we used to be, our brains can so easily be overwhelmed by too much data, leading us to zone-out, and this can lead us to miss even more. Therefore, I suggest that big data should be exposed progressively, beginning with highly simplified and summarised views, offering more complexity and detail only when our interactions suggest we’re ready for it.

There’s a great article here on adjusting interfaces to suit the level of familiarity and skill we have with the system at hand. If we can deliver big data to people in tune with their “flow zone” (the comfortable level of complexity), then perhaps we can all really benefit from the wealth of data at our fingertips.

Read the original post here on 12ahead.com


Join the conversation