Big Data, Good Information and A Way For You To Use It

In this article I want to explore 3 connected ideas. The first is about big data, the phenomenon that now makes available enormous, staggering, volumes of information almost instantaneously. The second is a condition that says information already known to us can limit how big data can be used because there are other types of knowing to expand thinking. And the third idea is the sum of the first two: that the intersection of big data and a different way of seeing information means a model must be designed to utilize large amounts of validated information in a reasonable way.

So the era of big data is here. Imagine Niagara Falls and the millions of gallons of water that shoot over the precipice virtually every minute. That’s the scale of information we envision when thinking of the amount of data we can reach out and grab—or in some cases is pushed to us—everyday.  It would be impossible to know it all. But one benefit of massive volume is, when looked at through a certain lens, we have an opportunity to connect seemingly unrelated bits of data and discover trends, make predictions, even pre-position products and services long before we click, point or touch. It’s the compilation of colossal amounts of data that presents a challenge. How do we pluck just the right information we need from this torrent of bits?  This is the difficulty with information management in the era of big data; it’s like trying to take a sip from a fire hose. Our need is not to get information it’s how to get just the right content to help us work with more accurate and insightful facts and, smarter and faster?

Clearly then we see how working out a process to employ big data and make quality business decisions is difficult. Furthermore consider this context, our second condition:

“There are known knowns” began an answer to a question at a US Department of Defense News Briefing made by Donald Rumsfeld while serving as United States Secretary of Defense in February 2002. Actually, here’s the whole tortured phrase… “there are known knowns; there are things we know that we know. There are known unknowns; that is to say, there are things that we now know we don’t know. But there are also unknown unknowns – there are things we do not know we don’t know.” Though it may seem convoluted it is a “brilliant distillation of a quite complex matter,” said Mark Steyn, a Canadian columnist and echoed by many others, even legions of his detractors.

While good information on its own is valuable its utility when combined with other data to discover other, perhaps new data and still newer meanings is really profound. Sometimes the information is known and we need to fasten it in context, other times we don’t know there is… and what is…  trustworthy information but have to discover it; and more abstract yet, as unknowns the potential of useful but opaque information demands we peer into the future and ask ‘what if’ and proceed to manufacture information on (hopefully intelligent and intuitively perceptive) speculation.

If you’re in the business of solving problems—and who isn’t really—you’ll need an information life cycle model to regard big data and the ‘knowns issue’ to manage a collection of information for maximum use. And beware; too much data without vetting and affirmation, means you might miss the really important stuff, an effect that keeps security services awake at night. And therein lies the third concern of massive information management.

By summary then, we face three elements in our quest to make big data work for organizations:

    1. Gathering information factoring the effect gained when combinations of content reveal even newer more, newer, meaningful data
    2. Respecting knowns and unknowns as  fact and as potential ‘black swans’ (an unpredictable or unforeseen event, typically one with extreme consequences) that can and will skew results if not discovered prior or during information capture or application
    3. Culling the really useful information or data—those bits directly related to the problem at hand—from the gargantuan amount of information flying about making it accessible, contextual and changeable.

Here’s a model than might help us slow down a bit, turn down the faucet and cull out know information and potentially new content when big data offers additional tonnage of content.


The flow chart illustrates how information would be categorically organized; a model for the standardization of an information life cycle in big data world.

Ultimately culling useful information from an almost limitless stream comes down to energy, resourcefulness and commitment. When building a learning course for example, your subject matter experts deliver very specific information as they must do. However, is there other data in text, as visuals, in video that might provide a different way to see the information? Clarifying content by shifting the context just a little bit can often shine a light into corners formerly unseen. Whether one has the time or inclination to make the effort to go shopping for more information is dependent on time and budget, yes, however, when looking to make learning better and richer, drinking from the stream is often a task worth enduring. Creating metaphors mined from a combination of newly discovered information can improve the user experience—and enjoyment—like spinning a kaleidoscope and seeing new patterns. Using a model such as the one proposed might make such an effort more reasonable.


Heroes of Learning Foundation Debut

For the longest time I’ve wanted to learn about the influencers who have changed the course or perspective of professional practice.

In the world of corporate learning, for me, at least its been Roger Schank. He is all over the web so I won’t list all the ways you can reach him. Though he’s been at it a long time he keeps reinventing ways to communicate his purposeful crusade for ways to learn. And his powerful arguments for the how and why learning should be built. The simple fact he resonates for me so powerfully is we come from the same world. I started my career in education shortly after leaving advertising and graphic design for a modest sized-but known firm in NYC. My first teaching year I journaled about all the inequities, stupidities and failures of schools. I held on to this – and the tipping point that compelled me to leave education was the discovery – up in the attic – of my old journals. Guess what? Rereading my writing from that first year I realized NOTHING HAD CHANGED. So I had to.

What I found in corporate learning – and especially entering the field in synchronicity with the exultant rise of technology, is that you can have the freedom to deliver knowledge, skills and even influence behaviors given the opportunity, a wise client, and a modest budget. This was reinforced when I discovered Cognitive Arts,  Roger’s test bed for his philosophy to be played out for real. He taught me how e-learning could work to bring about massive and distributed performance change. His 2 page memo I kind of copped from one of his minions laid out a rubric I could get behind. And I did and still do.

Now I want to understand how others came to their way of practicing. I think it’s important. Do you, in fact,  have guiding principles and if so where did they come from — and from whom. I believe that if we can share the influencers that made our designs for learning what a wonderful service to the field. Think of it as a repository of the good guys processes – and a deep and wide reference for a beginner or an experienced professional to draw from as needed.

So I ask that you consider signing up for the LinkedIn group “Heroes of Learning Foundation” and share your stories with us. Check it out at or if the link won’t take just sign in and go to groups and search us out. This won’t be nearly as powerful until we have a critical mass of participants. I’ve gotten some positive feedback  already but it remains to be seen if you all will give back to the profession and get something in return as well.

Hope to see you on LI.