“The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far. The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the deadly light into the peace and safety of a new dark age.” — H.P Lovecraft, “Call of Cthulhu,” Weird Tales, 11, No. 2 (February 1928), 159–78, 287.
While it is obvious that H.P. Lovecraft could not know about big data, this quote is very relevant. Alexis Madrigal of Nextgov’s Big Data blog thinks so as well. That is where I first saw this quote and its being linked to big data. There are many things about big data and how it can be used that are very scary. We have seen recently where is is actually difficult to limit how much data you may end up collecting because of how hard it is to separate out what you need from what is found (ie. NSA and FISA). It is important to not blame the technology as the problem. The problem is its application and use by human beings.
There are many “Big Benefits” from the use and application of “Big Data.” Look at the growth and maturation of the field of bioinformatics and its use in medicine. Sequencing of the human genome is the application of big data. Genomics will change how we are treated for disease.
Big Data will help us in the battle to overcome global warming. Increasingly accurate weather forecasts and improved computer models of the effects of global warming are all applications of big data.
Big data is now showing up in all the hard sciences and in the “soft” such the social sciences. It is impossible to get away from it.
All of us in special libraries, especially in business, technical and research libraries, have seen our jobs change because of interest in big data and because many of us are directly involved in the exploration, analysis, and manipulation of big data sets.
A bit of common sense will help us avoid us the fate suggested by H.P. Lovecraft of ”mad from the revelation or flee from the deadly light into the peace and safety of a new dark age.” As with all technologies, big data is not in itself good or bad. It is in how it is used. As librarians we can help direct its use into positive directions.
Note: These are my own opinions and not the opinions of SLA, Military Libraries Division of SLA, my employer, or the U.S.Air Force or DoD. — Bill Drew