Are You Sure You Have Sufficient Information?

Merely because the volume of information is growing, it does not mean that we are submerged in a deluge of information, popularly christened ‘information overload’. We feel submerged most probably because we don’t know what we want the information for and hence feel lost. In fact, the problem could well be the opposite – information underload. Sound counter-intuitive? Bill Gates was probably the first to articulate this when he spoke to a gathering of CEOs in the Microsoft CEO Summit in 2005. Arguing the need to access more data in areas such as sales results and corporate budgets, he said: “”I’d say in all of these cases, we are really dealing with information underload,” Gates said in his talk, which kicked off Microsoft’s annual CEO Summit, adding “we still want a lot of information.” (https://www.cnet.com/news/gates-information-overload-is-overblown/, article by Ina Fried, May 19, 2005). The problem was (and still is) that information was not available in one place. As he elaborated (what many will agree with even today), “you have to seek information out ….. it is spread across different systems”. (URL as above)

Granted that today there is a vast (and growing) information on any topic and you could be forgiven for feeling baffled, although I am skeptical about that. As someone who earned a living by finding actionable intelligence from information that is available to everyone, I can say that you should be clear about what and why you are looking for. In May 2006, Gates sent an email to the CEOs who were going to attend the CEO Summit a day before the event. The full text of the email is available at https://www.eweek.com/news/gates-memo-beyond-business-intelligence. To quote: “The problem, really, is twofold. The first is information overload. Faced with the endless deluge of data that is generated every second of every day, how can we hope to keep up? And in the struggle to keep up, how can we stay focused on the tasks that are most important and deliver the greatest value? The other problem is something I call information underload. Were flooded with information, but that doesn’t mean we have tools that let us use the information effectively”.

Starting point

Since then, there has been an enormous growth in tools and techniques to harness data from multiple sources, but we have not been able to overcome the problem, because information has grown faster than can be managed by available tools & techniques. Let us take cancer research. That too is very broad unless we are clear what we wish to research. Let us narrow it to say hypoxia and targeted cancer treatment. We should narrow it yet further to a specific type of cancer – breast cancer or pancreatic cancer, two cancers where this research has been going on for a while. Even this will encounter massive amounts of data produced by already completed (to whatever degree of satisfaction) research. Or gaps in understanding since that too is data, as it will provide pointers to further research.

Research such as this usually goes by collection of primary data, since you need to clearly define what the data is that you wish to gather. In fact, capture, to gather. If this absolutely critical went wrong, the entire exercise will be compromised. In my experience, I have seen this on many an occasion not to be concerned about.

As a teacher, I advise students to think through their research questions first. Even the form in which the question is posed since that itself could affect the information captured. I have seen even post-doctoral students floundering because they have not thought through what is involved in the problem they have encountered which they are keen to research. If you got this right, there is a fair chance (high probability) that you will capture relevant information. And if you have done as do others, you are creating the kind of information that doesn’t need prefixes such as over and under. To emphasize, not only will you get your research going in the right direction, you will simultaneously contribute to future research by creating a database of appropriate information.

As I wrote in a LinkedIn post, “The real risk in an environment of information overload is selective narratives, which will exclude anything even remotely resembling objectivity. Since there is a high volume of information, it is possible to find sufficient ‘information’ to suit the narrative that is being ‘marketed’. Yes, marketed because that is exactly what happens. The search for a balanced, objective narrative has to be deliberately undertaken, which means an active search for alternatives including contrarian views. Good journalism does precisely that. The problem of confirmation bias is real – looking only for information that will support your case. This is why we are going to (always) encounter islands of partial truths. The complete picture – the holy grail of modern data analysis – may be just utopian”.

I will continue this kind of narrative in another post.

Takeaways

The question is not whether there is information overload or underload

Ensuring that there is information adequate to the purpose is the key

Selected narratives are the real challenge