What is “big data”? How is it used? What does it mean to business, to you? Confused by the possible definitions? You are not alone. Here’s a nutshell overview.
Chris Herbert, a former IBM executive and special projects director, says there is some logical confusion about a definition for the term “big data” because of the practical lack of separation between collecting, storing, and analysis and use of information. In an e-mail response to questions, Herbert states, “The term Big Data really refers to the vast volume of data (information) in existence. Big data is, therefore, simply, collected information. When used for good, along with algorithms (instructions) that give computers the ability to listen, interpret and act when tasked by humans. Use of big data can save money, time and even lives. The same data, however, can allow your every purchase and recordable action to be followed and used to put together a profile of you for advertisers, potential employers, or governments.
Big data users can know their customers and their shopping habits better than many of those customers may know themselves. Not only are your purchases and other interactions with retailers, utilities, banks and service providers tracked, but with newly emergent algorithms, your habits, purchases, use of sales coupons and how you pay are analyzed to get a bigger and better picture of who you the consumer are and what products you might jump at the chance to buy.
Portland Oregon Savory Spice Shop owners Anne and Jim Brown have carefully chosen their social media advertising to the get the best of big data’s offerings for the launch of their new boutique business. “With our Facebook ads we’re routed to target those groups of buyers and potential customers most likely to want our high-end specialty products,’’ says Jim. “We just had to invest in the ad and then Facebook algorithms and analysis sort through mountains of consumer data to find those that most closely match our buyer profile. Those potential customers then get targeted ads and special announcements from the store.”
The Browns estimate the service saves hundreds of hours of work that would otherwise be done by hand and possibly having to add, or contract for, highly specialized staff to work through all the available information collected on consumers. “The time and cost saving has been real and with every sale the data mined from our big data pile becomes more and more accurate,” says Anne, Jim’s wife and partner in Savory Spice.
Closer to home, Alaska businesses large and small are also tied in to Internet-based data collection analysis and distribution systems. Sales and stock are reported and the information stored for later decision-making.
We’ve seen an explosive growth (e.g., Internet-based data, databases in business, unstructured data in the form of text) in this available information over the past decade primarily because of the tremendous growth of the Internet. To leverage and use the vast amount of data to gain insight, meaning and more information requires analysis—thus the connection between the two thoughts, and the confusion.
As an aside (but relevant), IBM’s Watson computer, which came into the national spotlight by winning Jeopardy, can “read” unstructured data and “understand it.” It then can help direct human actions. Watson’s biggest potential? Medical diagnosis. By looking at data and sets of parameters, Watson assists in diagnosis and treatment recommendations for health care providers and, hopefully, patients.
Internet entertainment provider Netflix uses big data and analytics to determine viewing patterns and decide what types of offerings to promote to its customers.
Big data use even affects a person’s financial life. It affects such things as mortgage terms and pension well-being. How?
Data determines what is traded and when on the world’s stock exchanges—the related algorithms work, in microseconds, to buy and sell millions of shares of stock. If one is just a few microseconds slower than the next guy, tens of millions of dollars can be lost. The data? What stock and how much of it has just been sold. How important is it? Important enough that entire buildings in large trading hubs have been, and are being, transformed from living and working spaces into massive computer server centers as close to Internet sources as possible. Fibre optic cable trenches are being constructed to move big data even faster. As an example, a fibre optic link has been created between New York City and Chicago that will allow the transfer of data just three microseconds faster than previous “traditional” satellite or dedicated copper lines. What does that mean? In a 2011 talk to a TED (technology, entertainment, design) conference, business design consultant and lecturer Kevin Slavin stated that it takes about 500,000 microseconds to click a mouse. “(In stock trading) if you’re five microseconds behind, you’re a loser.”
So, food for thought: big data can be a blessing when combined with the right algorithms. Big data has the potential to save lives and treat disease faster. Big data allows business and industry to make previously limited, labor-intensive calculations about supplies and likely customers so more of what people want will be available on retail shelves.
Big data can also be a nuisance (and worse) when it inspires unwanted and unsolicited advertising, phone calls and email; or misinterpreted, causing denial of credit, incorrect charges, or in serious cases, the damaging reality of identity theft.
What’s it going to be? Probably some of both—the good and the bad.
The optimistic view is that good will reign, that reduction in labor and increase in speed of personal and business transactions is worth taking a chance on the existence of so much information, stored, ready for use by humans, but also by the magic machines we’ve created, computers. The cautious view: that our privacy is endangered, that now more that ever Orwell’s Big Brother is closer to reality because of big data and that soon computers will make daily decisions in our lives and report us when we fail to follow the instructions made to other humans whose sole job will be enforcement.
For now, from a business point-of-view, the good is winning. The benefits of having big data available for analysis and use are outweighing the potential for abuse.
Journalist Ed Arthur writes from Anchorage.