The term “Big Data” first surfaced about five years ago in connection with the data management and analytical needs of extremely large internet businesses such as Google and Yahoo. These companies needed an economical and highly scalable architecture for the analysis of certain files, such as their logs of web activity. Their data and analytical requirements did not fit closely with the relational database products used in most commercial systems. And, because their data volumes were so extremely large and fast growing – between ten and one hundred times the size of the largest relational databases — they decided to create their own solutions.
As these techniques spread, principally via the use of open source licenses, it turned out that similar requirements existed in many enterprises.