December 6, 2011

Big Data-Not Just Big Storage Or It May Be A Big Headache

Copyright © 2011 Cary J. Calderone Esq.

Time to give the busy professional's definition of the latest technology buzz phrase, "Big Data."  In brief, it is about being able to process and mine very large amounts of data (even petabytes) for business intelligence.  Big Data indexing and database technologies, like Hadoop and NoSQL allow for distributed processing that previously was impossible with standard table-based relationship databases.  However, too many short-term thinkers will try to implement a Big Data strategy by doing nothing more than keeping everything they can and figuring it out later.  This approach is fraught with Big danger.

The most frequent justification for employing a Big Data strategy is to improve business intelligence.   Customer satisfaction and trends identified with Big Data technologies can be used to optimize your supply chain as well as shape your digital marketing efforts.  However, more data can also mean more concerns with US and Foreign rules governing privacy and security.  Will your data be anonymized?  Will critical HIPAA or Credit Information be scrubbed or tokenized sufficiently to adequately secure and protect it?  And last, but not least, are you going to have to analyze your Big Data for E-Discovery?   By and large, if you own and control the data, at some point, a court, at the behest of one of your adversaries, will want you to analyze it for E-Discovery purposes.  Will you be able to defend your practices?   Will you be able to show why you should not have to produce it?  If you have to analyze it for E-Discovery production, will you be able to do so in a timely fashion?  One company that decided to keep all company email realized too late, that their search tools and storage drives were inadequate.  It would take 12 days to get simple keyword search results.  This was definitely not useful for performing Early Case Assessment.  Good DRED practices suggest that before you decide to keep everything, make sure you have the policies and procedures and technology in place to use your Big Data to your advantage, and limit your exposure to those potential problems.  Make sure your Big Data plans provide you with Big benefits that will far outweigh some of the potential costs.   Then, maybe you can avoid Big Delays, Big Problems, Big Costs and even, Big Sanctions.

No comments: