M
uch of the buzz currently generated around big data results
from the new correlations
enterprises are able to derive from the
data streams they can now collect—
and even among sets of previously
analyzed data. The key is finding correlations across as many dimensions
as possible. Think of astronomers using multiple telescopes to observe the
same celestial object, with each frequency band revealing new insights:
some show clouds of interstellar dust,
while others permit a glimpse of what
lies behind. With enough observation,
a ground truth can emerge.
So, just as they are achieving
wide adoption, big data systems are
undergoing significant change—
something we see continuing into the
future. Heterogeneous processing and
reconfigurable logic will drive another
wave of change (though we hope not
in the programming models). Nonvolatile memory is more than likely to
find its place in the data layer between
memory-based objects and the file system. Data modalities and analytical
engines will continue to expand.
All this will result in a platform
that analyzes historical data, reacts
to current data, and predicts future
data— while still maintaining the
key attributes of scalability, resilience, and usability fundamental to
big data systems.