HomeNews and blogs hub

How BIG is big data?

Bookmark this page Bookmarked

How BIG is big data?

Author(s)

Chris Morris

Posted on 7 December 2011

Estimated read time: 2 min
Sections in this article
Share on blog/article:
Twitter LinkedIn

How BIG is big data?

Posted by s.hettrick on 7 December 2011 - 4:23pm

Preikestolen.jpgBy Chris Morris, STFC.

GenBank now contains 100000000000 base pairs. That's big, in the sense each similarity search visits every record, and there are millions of searches a day. But it's not BIG, in the sense that it fits on one disk, and only takes 200s to transfer at 1Gb/s.

An electron microscope tomography image may contain 8 billion pixels. That's big, in the sense that the noise reduction algorithms take polynomial time in image size. But it fits on a USB stick. The data stream from LHC is big; but most of it is of no significance. But it can be reduced to a one-bit answer, such as "Does the Higgs boson exist?" The Sloan Digital Sky Survey contains several terabytes of data. That's big, since it is all unique data of potential interest. But it would fit on a medium size RAID. A Next Generation Sequencing instrument can record 100TB of data in a day. It will quickly be reduced ten-fold, and then later reduced to a consensus sequence - a whole human genome is 1.5GB.

Whether data is big or not depends on what you want to do with it.

Share on blog/article:
Twitter LinkedIn