Manufacturers are slowly coming around to the concept of Big Data and how it drives business. The adoption of Big Data best practices is already in place in other industries. Common adoption in manufacturing companies is inevitable as a way to solve business problems and effectively compete within their industry.
It’s hard to read much about information technology, analytics, and business intelligence without hearing the term “Big Data.” Unfortunately, it’s grown into such an amorphous term that it doesn’t mean the same thing to everyone. Is Big Data something that manufacturers should focus on? What is it, and how will it drive more profitable business?
What is Big Data?
Let’s start with the most common definition. In 2001, Doug Laney of Meta Group (now Gartner) used “3 V’s” to describe the effect of Big Data:
- Increasing Volume: The amount of data coming in
- Increasing Velocity: The speed of the data coming in and going out, and
- Increasing Variety: The data types and sources.
The definition has changed slightly over the past 14 years, but the essentials are still captured by those 3 V’s. The diagram below depicts the many types and volumes of data that most businesses deal with:
Chances are that your manufacturing enterprise manages big data every day:
- Manufacturing equipment has sensors that monitor quality and safety during manufacture, as well as conditions such as temperature, pressure, tolerance, etc.
- Goods — incoming materials and outgoing production — have RFID tags or barcodes that are tracked as they move through your business and become part of larger assemblies.
- Some manufactured items collect data post-sale and transmit the data back to the manufacturer.
The volumes and the variety of data types qualify this as Big Data.
Now that you know you have it, so what? Does it have any value to your business apart from the original purpose for which it was collected? Should it be kept and, if so, for how long? Should it be organized and added to other business data that you collect, such as your list of customers or your financials?
Capturing Big Data
A common mistake in working with Big Data is isolating it, and believing that it must either be kept separate from other data, or be physically combined with other data in order to deliver business value. Too often, the focus is on the “Big” part, rather than the “Data” part.
Isolating Big Data from the rest of your enterprise data leads to program development and technology adoption that work well for only a very limited number of use cases.
If you’re not able to logically integrate Big Data into your environment today, and if your business users don’t have access to it, or can’t access it using the tools they’re familiar with, then you’re surely losing ground to your competitors who can.
Putting Big Data to Use
Since isolating Big Data reduces its accessibility and value, how should you put it to use? Begin by considering how to incorporate Big Data into your overall data architecture.
A sound data architecture accounts for many different types of data, often stored and retained in different locations and for different periods, while providing access to a variety of users and tools for analyses. Your data architecture should also provide you with methods to move, manage and access data. This is referred to as a Unified Data Architecture, and the diagram below helps to illustrate the key components:
Data Architecture
It’s useful to think of your data architecture as a factory — one that ultimately is producing information that can be analyzed and used to deliver value to your business. Let’s examine the Unified Data Architecture using the metaphor of an Information Factory.
Data Paths
A manufacturer’s factory begins by receiving a variety of raw materials, parts, and assemblies from a variety of sources. As these materials are received, they are tracked, stored, and managed. They are moved to areas in the factory where they can be readily accessed during the manufacturing process. Likewise, data coming into an Information Factory needs to be kept somewhere, managed, and tracked, and this is done in the “Data Platform.”
Maintaining Data Sources
Some data may enter the factory already joined to other data – just as assemblies like a motor enter a manufacturer’s facility. It probably doesn’t make sense to break this data apart, and in the same way, it probably doesn’t make sense to pre-assemble individual bits of data, such as the output from sensors, into groupings of data. As much as possible, the data platform should be able to receive the raw materials – or data – in the form in which it arrives.
Data Warehousing
The “Data Warehouse” is the assembly line of the business. If you manufacture automobiles, then the assembly line combines multiple parts and assemblies using a predefined series of steps and procedures in order to deliver the finished product.
The Information Factory uses the data warehouse to assemble various pieces of data in order to deliver information to your business users. Each time the warehouse produces a piece of information, the user of that information has a degree of confidence that its accuracy today is the same as it was yesterday or last month — because that information was produced by a predefined set of steps and processes.
The data warehouse is designed to handle these operationalized requests for information, and can do so in a very efficient way, handling a high volume of requests and delivering results in a guaranteed way.
Data Discovery
The Discovery Platform is essentially the Research and Development department of the Information Factory. The R&D group in a manufacturer will consider how to modify its assembly line to deliver new products that meet the desires or requirements of its consumers.
Consideration is given to whether new raw materials need to be sourced, and how best to bring everything together in a way that minimizes cost and maximizes value. The R&D group also builds prototypes prior to moving production to the assembly line. In the Information Factory, the Discovery Platform is the place where new questions are asked; where new types of analytics are developed.
The Discovery Platform is a fast fail kind of environment. Many things are tried, but not all are successful. Once a new question or analytic is developed, and it’s clear that it delivers value to the business, it is then moved to the “assembly line” or Data Warehouse, in order to be delivered in a predictable way to the consumers of the information.
The ultimate goal of your Information Factory should be to enable any question to be asked against any data by any qualified user at any time.
Identifying Your Data Analysis Goals
The ultimate goal of your Information Factory should be to enable any question to be asked against any data by any qualified user at any time. Looking at data in this holistic way, rather than partitioning it by type and source, allows high-value questions to be asked.
Consider the benefit of being able to analyze service problems to identify clusters of issues, and proactively contact consumers without the need to issue a recall. Or being able to isolate failures in assembled systems back to an individual component, and to know exactly which customers have those parts in the units they’ve purchased.
The Big Deal
So, is Big Data a Big Deal? It can be, especially when it is part of your Unified Data Architecture, and your business users have the ability to include it in analytics.
If all you are doing is creating unique environments for your data and isolating access to it, it will do little more than create expensive sandboxes that deliver very little return to the business.
If you’re not able to logically integrate Big Data into your environment today, and / or if your business users don’t have easy access to it, then you’re surely losing ground to your competitors who can.
Properly capturing your data, warehousing it, and making it available to your employees, using tools they are familiar with, empowers them and your company to make real-time, informed, valuable decisions.
How does your company handle its Big Data?
Terry Allen is a long-time thought leader in Data Warehousing and a southern regional sales director and executive level consultant for Teradata. Contact him at www.teradata.com or terry.allen@teradata.com.