RKL eSolutionsBlogs

Big Data versus Big Pile of Data

Almost since the introduction of computing systems for business, there has been a sort of underlying belief by many that there is a direct correlation between the volume of available data and the opportunity (at least) to manage more effectively. This belief system can be reduced to a simple, widely-held, axiom:

If we have more data, we can manage more effectively.

Big Data Sage X3

There is, however, a big problem with this assumption.

The first problem is that it is a leap of logic: If we take it in steps, the logic might be spelled out something like this.

  1. If we have more data, we have more knowledge.
  2. If we know more, we can manage better.

Here is the root of the problem exposed. More data does not equate to more knowledge. Allow me to give you a small example.

We are probably all familiar with the widely told story of how Sir Isaac Newton "discovered" gravity as the result of watching an apple fall from a tree.

Well, simply said, Newton did not discover gravity. People had discovered gravity long before whenever they tried to do anything for which gravity was a constraining factor—like lifting an object from the ground onto their horse.

Now, suppose someone who had discovered this gravitational force had taken time to write down tens, hundreds or even thousands of observations about the effects of gravity. They would have had a lot of data about gravity, but they would still have no knowledge about gravity.

That is where Newton comes in.

We call Newton the "discoverer" of gravity because he went beyond collecting data. He came to know and understand gravity—he gained knowledge about gravity—because he developed a theory about gravity and was able, then, to begin put his observations into context. In the context of his theory, he was able to determine where certain cause-and-effect relationships postulated by his theory were correct or incorrect. [See on W. Edwards Deming below.]

It was Newton's theory that was able to transform collected data into knowledge.

Collecting more and more data would never have lead to knowledge. It would have just added to the pile of data.

Dave Stenfort, Director of Operations at internationally recognized Anritsu, talked about this very matter. In an interview with Bob Bowman of Supply Chain Brain, Stenfort had this to say:

"It was really hard to get to the data—to what was the important data versus all the pile of data…. What we had to do… with our data was to get rid of all the data that wasn't important [in order] to get to the data that was really important, so we could act just on the data to support our customer." [Emphasis added.]

The better axiom

Carol Ptak and Chad Smith, writing in Orlicky's Material Requirements Planning (Third edition) [1], point us to the first law of manufacturing:

All benefits will be directly related to the speed of flow of materials and information.

This first law of manufacturing was articulated nearly half a century ago. That, of course, was long before there was the absolute flood of information now made available to enterprises of every size and description through the IoT (Internet of Things) and low-cost, high-speed computers and data storage system.

Today, as Ptak and Smith have said, there needs to be a modification to the first law of manufacturing. The updated first law reads:

All benefits will be directly related to the speed of flow of relevant materials and relevant information.

Better, more effective management does not flow from access to more data.

Better, more effective management flows from access to relevant information. But, additionally, there needs to a sound, underlying theory upon which to base both management's understanding and actions.

W. Edwards Deming [2] offered many strong admonitions to this effect:

"Information is not knowledge. Knowledge comes from theory."

"There is no knowledge without theory."

"Experience teaches nothing without theory."

"We should be guided by theory, not by numbers."

Got improvement?

We find that mediocre performance levels in many of the companies with which we come into contact stems from two crucial factors

  • Management effectiveness is being stymied by an overwhelming flood of data—or, at least, access to data—but they are unable to sort out the relevant from the irrelevant amidst the flood
  • Management has no coherent and effective theory by which to make decisions and set priorities—everything they thought they understood, they have already tried, applied, and found it produced little or no improvement (sometimes they even found themselves moving backward)

Will Business Intelligence applications help?

The short answer to this question is: they might.

Consider what is sometimes called "data mining." That is a term we apply to "digging through" our big piles of data.

However, no successful mining company ever starts their costly mining operations without first having a theory. Their theory, combined with their observations of the circumstances, given them several critical factors upon which to base their investment in the mining effort:

  1. What they are likely to find when they start digging around (e.g., gold, silver, titanium, diamonds)
  2. The likelihood of finding something worthwhile
  3. Where is the best place to start digging
  4. What methods to use for the digging.

If, due to lack of theory, you have no idea what you should be looking for, the likelihood of finding something worthwhile, the best place to start digging, or the best methods to apply, then you should consider you strategy behind making a plan.

New thoughtware before new software

We are able to help clients by introducing them to "new thoughtware" [3] before they make a decision about costly new hardware or software. Chances are we can help your enterprise or your supply chain, as well.


[1] Ptak, Carol A., Chad Smith, and Joseph Orlicky. Orlicky's Material Requirements Planning. Third ed. New York: McGraw-Hill

[2] W. Edwards Deming is the man whose management theories revolutionized and revitalized Japanese industry following World War II. He went to Japan after his management theories had been rejected by just about every manufacturer of note in the United States. In his frustration at one point he said, "Export anything to a friendly country except American management."

[3] The term "thoughtware" is not original with us. We owe that term to Debra Smith and Chad Smith and their excellent book Demand Driven Performance Using Smart Metrics.



Questions? Need Support?
Get in Touch