Top 8 trends for big data in 2016
NoSQL technologies, commonly associated with unstructured data,
have seen significant adoption over the last 12 months. Going forward, the
shift to NoSQL databases as a leading piece of the enterprise IT landscape
becomes clear as the benefits of schema-less database concepts become more
Nothing shows the picture more starkly than looking at Gartner’s
Magic Quadrant for Operational Database Management Systems, which in the past
was dominated by Oracle, IBM, Microsoft and SAP.
In contrast, in the most recent Magic Quadrant, the NoSQL
companies, including MongoDB, DataStax, Redis Labs and MarkLogic, are set to
outnumber the traditional database vendors in Gartner’s Leaders quadrant of the
Apache Spark lights up big data
Apache Spark has
moved from a being a component of the Hadoop ecosystem to the big data platform
of choice for a number of enterprises.
Spark provides dramatically increased data processing speed
compared to Hadoop and is now the largest big data open-source project,
according to Spark originator and Databricks co-founder, Matei Zaharia.
More and more compelling enterprise use cases around Spark are
emerging, such as at Goldman Sachs, where
Spark has become the “lingua franca” of big data analytics.
Hadoop projects mature: enterprises continue their move from Hadoop proof of
concepts to production
In a recent survey of 2,200 Hadoop customers, only 3% of
respondents anticipated they will be doing less with Hadoop in the next 12
months and 76% of those who already used Hadoop planned on doing more within
the next three months.
data grows up: Hadoop adds to enterprise standards
As further evidence to the growing trend of Hadoop becoming a
core part of the enterprise IT landscape, investment will grow in the
components surrounding enterprise systems such as security.
Apache Sentry project provides a system for enforcing
fine-grained, role-based authorisation to data and metadata stored on a Hadoop
These are the types of capabilities that customers expect from
their enterprise-grade RDBMS platforms and are now coming to the forefront of
the emerging big data technologies, thus eliminating one more barrier to
data gets fast: options expand to add speed to Hadoop
With Hadoop gaining more traction in the enterprise, there will
be a growing demand from end users for the same fast data exploration
capabilities they’ve come to expect from traditional data warehouses.
To meet that end-user demand, adoption of technologies such as
Cloudera Impala, AtScale, Actian Vector and Jethro Data that enable the
business user’s old friend, the OLAP cube, for Hadoop will grow – further
blurring the lines behind the “traditional” BI concepts and the world of big
number of options for ‘preparing’ end users to discover all forms of data
Self-service data preparation tools are exploding in popularity.
This is in part due to the shift toward business-user-generated data discovery
tools that reduce time to analyse data.
Business users want to reduce the time and complexity of
preparing data for analysis, something that is especially important in the
world of big data when dealing with a variety of data types and formats.
data warehouse growth is heating up – in the cloud
The “death” of the data warehouse has been overhyped for some time
now, but it’s no secret that growth in this segment of the market has been
But there is now a major shift in the application of this
technology to the cloud where Amazon led the way with an on-demand cloud data
warehouse in Redshift.
Redshift was AWS’s fastest growing service but it now has
competition from Google with BigQuery, offerings from long time data warehouse
power players such as Microsoft (with Azure SQL Data Warehouse) and Teradata,
along with new start-ups such as Snowflake, winner of Strata + Hadoop World
2015 Startup Showcase, also gaining adoption in this space.
Analysts cite 90% of companies who have adopted Hadoop will also
keep their data warehouses and with these new cloud offerings, those customers
can dynamically scale up or down the amount of storage and compute resources in
the data warehouse relative to the larger amounts of information stored in
their Hadoop data lake.
buzzwords converge: IoT, cloud and big data come together
The technology is still in its early days, but the data from
devices in the Internet of Things (IoT) will become one of the “killer apps”
for the cloud and a driver of petabyte scale data explosion.
For this reason, leading cloud and data companies such as Google, Amazon Web Services and Microsoft will
bring IoT services to life so the data can move seamlessly to their cloud-based