Data. It is simultaneously the best and the worst thing about being in technology. To many people, it is personal, and to many technology professionals, it is the bane of their existence. While, to me, it is the gateway to an ever-expanding knowledge base, just waiting to be tapped. Come with me while we answer some questions about data. We know that every modern business, organization, or government agency stores data. What is this data? We will give an overview of why data is stored and what kinds of data are stored. We will discuss what factors (including government regulations) limit data storage and, finally, how do businesses, organizations, and government agencies determine what to keep (and for how long) and what to discard (and when to discard it)?
The data needs of every business and government agency vary. I currently work for a company that has data back to 1977. My personal take on this is positive. I believe that data is an amazing thing and, since it is real-estate, you can see trends dating back to 1977. There is currently 10 terabytes online. One of my previous companies had four petabytes of data because they had to keep x-ray data for 25 years. The made munitions and the x-rays for the outside casings were critical to detect defects and have a proof for government clients. A petabyte of data is about 10 railcars of paper, so it is a lot of data. I once heard that five exabytes could hold every word man has even spoken. Corporations are bound by law sometimes, to store the data. Others are just simply packrats. I heard a great quote once that said, “You can store and report anything but you cannot digest everything. You have to have a happy medium.” I like to call “actionable data.”
We are always told time and time again that data is a prized commodity, but it is very easy to be big data rich, yet still insight poor if the information is not harvested and analyzed efficiently. This is becoming increasingly important as the Internet of Things (IoT) connects more and more devices, churning out even more data. You have to pick out what is actionable. The Internet of Things (IoT) refers to a system of interrelated, internet-connected objects that can collect and transfer data over a wireless network without human intervention. The personal or business possibilities are endless. Again, picking out what is relevant and actionable.
Mining big data for actionable insight is a major challenge for most corporations. Searching all your data would be like panning for gold—long and often thankless task with little or no reward—with giant wins few and far between. Ultimately, you need to know what data to keep and what data to discard. This means having the right data scientists and tools in place to make these decisions and rapidly process the data. Yet, there are still organizations that spend more on collecting data than analyzing it. This is an easy trap to fall into.
Organizations are aware of the strategic importance of big data and analytics, but there are still hurdles to overcome. According to a recent survey, 31 percent of senior executives say the timeliness of data in their organization is poor, while 25 percent admit they lack the skills or expertise to make greater use of data.
In addition, 61 percent of executives acknowledged that their organizations should rely on data-driven analysis more and intuition less. At the same time, they did not see their organizations as highly data-driven, leaving them open to being overtaken by competitors.
The lesson here for business is that having too much data can become a fatal flaw, especially because we now live in the Big Data era where everything is being tracked, recorded, and analyzed with machine learning. Can you take action on the data set? If not, why do you have it? Have you ever acted on it before? If not, why are you collecting it? Is it possible to collect a different subset of information or data that would be more relevant? Answer these questions, so you are not wasting space or other people’s time.
Data has become an asset. Entire businesses exist based on Big Data, Advanced Analytics and pure Data Science, but they know what to do with the data. If you do not please, for the love of God, partner with someone that does.
Otherwise, it is possible you can overwhelm and drown your business and your leadership team with too much data. The leadership team sees the data and should be making decisions from it. Be careful, though, that anyone on the leadership team is not the one generating large amounts of data collection and wasting other’s time.
Rather than helping you make decisions or reduce risk, too much data can actually slow you down to the point where you can become paralyzed. It is the old saying about analysis paralysis it is real.
Studies show you only need 75 percent of the information available to make a decision. The goal should be to move faster than the market, not to accumulate more data than everyone else or every corporation out there. The trap that I see so many managers fall into is that they struggle to find the line between having the right amount of data to be able to decide.
I have worked for many bosses and different companies who become obsessed with accumulating more and more data to make the perfect decision but it does not happen. It just ends up bogging down the system and costing tons of cash. Analysis paralysis that lasts weeks or even months can end up letting the market beat you to whatever opportunity might have existed.
In other words, in your attempt to avoid risk by collecting more data, you have put your business into an incredibly risky position. That is why high data needs can become a fatal flaw for leaders. We have not even covered the organizational cost in accumulating and analyzing those masses of data when people could be doing more valuable work. Physical storage, cloud storage, and people cost are only a few costs of collecting mass amounts of data.
Go ahead and feed your computers as much information and data as they can handle. Let the machine process all day long. But when it comes to the data you, need to make decisions and act or take action!
What are you going to do with the data? If you are not willing or able to change your behavior or make decisions with it, that data might not be as valuable as you thought in the first place. It might even be a distraction. Do not lose sight of your goal. Good goal-setting questions are; what does the added data do for me, and what insights do I gleam from collecting this much more? Always think about how much data you need to take action vs. the resource requirements need to collect that data. Do not overload your brain, and at the same time, do not overload your BI team…Actionable DATA!