In short, Big Data is the collection and secure storage of many various transactions types, from a range of sources, that are pooled collectively to help inform businesses and decision makers. Through using statistical data and knowledge it enables you to make better, calculated decisions.
The most important tiers of the Big Data pyramid are:
- Secure Storage
Big Data is a collection of data from traditional and digital sources inside, and outside, of your organisation that will be a source for ongoing discovery and analysis.
Some like to constrain Big Data to digital inputs, like web behaviour and social network interactions; however, the CIOs I work with agree that we can’t exclude traditional data collected from product transaction information, financial records and interaction channels, such as a call centre and point-of-sale. All of that is Big Data, too, even though it may be dwarfed by the volume of digital data.
With very strict privacy laws (POPI) coming into effect in South Africa very soon, and laws which are already established in other countries, you need to ensure you do not infringe on the rights of the entities generating the data for you.
You also need to ask yourself: Are you allowed to use the data? And if so, how will you be securely storing the information? And, have you considered the data’s lifecycle? The security of data needs to be of up-most importance, and something to invest in.
This is how you use the data: Are you going to interpret trends or formulate strategies to launch new products? Or are you planning on learning how to know, where to, and how to sell existing products? Or are you strategising on how to enter new markets?
In terms of analytics, you either need to start at a need, or take the data and generate need by finding parties that have a data analytical requirement.
For example, an insurer might be looking for market trends and will need live averaged data to base new strategies on. On the flip side, you may want to sell a new product, but you will use the data to find a market through data analytics. Another use for analytics of Big Data might be to identify the top 20 trending hedge funds to focus your next investments into.
These are my three important tiers, but you might also hear phrases like ‘volume’ (the amount of data), ‘velocity’ (the speed of information generated and flowing) and ‘variety’ (the kind of data available). Whatever the terms you use or hear, one thing is clear: every enterprise needs to fully understand Big Data – what it is, what it can do, and what it means to you.