Big Data in 2017

We have all heard the term “Big Data.” While the business world is trying to come to terms with how to leverage it, I think we will see an increase in velocity and maturity in the next 12 months.

Big Data is transitioning from one of the most hyped and anticipated tech trends of recent years into one of the biggest challenges that our client IT teams are now trying to wrestle and harness.

Internet of Things, machine learning, deep data analysis and mature algorithms are all top of mind for data scientists, but they need to run on clusters. Clusters leverage data locality and horizontal scalability that help with load balancing, replication and failover. Yes, I know, you still have management overhead but I think the pros outweigh the cons.

We will see more tools and offerings like the Data Labs collaboration environment that connects data to business analysts and Allteryx which is a user-driven analytics platform that has some early stage machine learning integrated. There are many tools in between here; I just named a few.

IoT is probably at the forefront of this trend and brings with it an ecosystem of players for managing traffic and caching data. The latter as evidenced by our partner Amazon with AWS Greengrass. You may see an increase in a very chatty network if the right solutions are not mapped out.

Which leads me to note that increasingly Big Data is going to live and be processed in the cloud. There are some stats reporting that as high as 40% of Big Data workloads will be processed in cloud environments.

Flexential is positioned to assist our clients in this migration, from multiple cloud environments (public/private/hybrid) to the critical managed service around security and performance matrices. The penalty of cloud-based architectures, separated compute/storage, will grow more academic.

Okay, I know I might have some active dissenters, so post below and let the conversation begin. Of course, if you have other platforms that should be noted, add your comments. It would be good to hear other perspectives.

With broad-based expertise in operations, strategy, technology and business development, Peter is Chief Digital Officer for Flexential. He leads testing, validation, optimization and integration of new products and platforms that help enable Flexential clients to transform their IT infrastructure and achieve success.