CPG companies left fishing for answers in data lakes instead of building streams of intelligence

Topics: IT, data

The landscape of consumer goods companies increasingly has been spotted with growing data lakes housing a company’s historical data. While these data lakes provide some organization and access to the vast amount of data a company has, there is significant manual work to fish out any insights. As a result, these data lakes become stagnant data repositories polluted by the continued addition of unstructured data, instead of meaningful business growth sources.

Like any environment, to make a data lake a fertile growth source requires regular cleansing and movement. In other words, organizations must have a way to not only refresh the data, but also extract and harmonize data (See also: How to turn data pain into analytics gain) into meaningful, easily usable segments for analysis and future decision making. Since this process historically has been resource intensive, many CPG companies are hesitant to adopt practices and solutions that use the data for advanced analytical analysis or predictive planning. 

While these hesitations are understandable, the reality is that these solutions and providers should be providing          data lakes
automated resources for harmonization, easy-to-use cleansing functionality and experience-based guidance to
your organization. With this, the data lake that currently may be an eye-sore of lost potential becomes a living
source for business intelligence.

An area with great revenue-generating potential largely impacted by data quality and usage challenges is
trade promotion analysis and planning. This significant business investment is greatly impacted because 
any forecasting or ROI calculation is highly dependent on 3rd party syndicated data considered by most to
be unreliable. 
Furthermore, the inability to easily cleanse and harmonize this data with shipment and spending
data leaves companies 
with a flawed understanding of their significant trade investment. 
(See also: Analytics Driven Optimization Starts with a Hike Through the Data Weeds)

This is in part because, while the data lake may have the correct data sets, there is no tool pulling together the data in a useable, easily understandable way. It is a tool, like a Trade Promotion Optimization solution, that centralizes the data to provide a single version of the truth to serve as the foundation of the analysis, predictive planning, and constraint-based modeling that can really impact the revenue generation for your organization.

The investment in data lake technology by CPG companies was an important first step in prioritizing better understanding of their business. However, the sheer volume of data makes working with these data lakes to extract understanding in a timely and useable manner impossible. Now, companies want to prioritize data-driven decision making, predictive analytics, and AI – all potentially revenue-impacting objectives – but are paralyzed by their inability to turn data lakes into a stream of intelligence.

As too many companies push off investment in more advanced analytics capabilities because “we have to get our data in order,” they miss opportunities to make better decisions, be more competitive, and sustain better results. In reality, most companies don’t have a data problem; they have a problem turning data into understandable, accurate and actionable -intelligence. While it is easy to see yourself drowning in a data lake, today, it is equally easy to invest in a solution that will enable you to ride the waves to optimal outcomes.

What others are reading...

Subscribe to receive more industry best practice and updates