The Concept and Purpose of Big Data Posted by John on 28 September 2015

Big Data is a recently created, popular phrase used to describe the process by which organisations accumulate and analyze vast quantities of information collected from sources at different times and for different reasons. Public and private bodies gather and organise data which is then processed so as to extract from them both hidden information and surprising correlations. The immense amount of data that is expected to be produced in 2015 - five times that of 2010 - illustrates how ingrained this reasonably new phenomenon of interaction with the digital form has become in modern society. The ubiquity, thus, of big data cannot be overstated. With such importance comes economic benefits such as enhanced efficiency, increased productivity and decreased manufacturing costs. Proponents of Big Data strongly emphasise its many benefits, including those for society at large; mentioning specific improvements in national security, fraud detection and disaster relief as worthwhile significant consequences.

Although improvements in data analytics represent a positive development for modern culture, it must be noted that the primary driving forces behind the explosion in data collection quantities are corporate business entities that recognise citizens' personal data as commodity first and foremost. Tech giants such as Google and Facebook now possess information on subscribers' personal tastes and opinions which can then be utilised in order to improve marketing strategy and create personalised, targeted advertising models which provide the primary source of income to these companies and thus subsidise their other, principal business practices. 


An interesting example of data mining being put to good use nonetheless is ‘Google Flu Trends’, described as "the poster child for the positive benefits of Big Data", where relevant individual Google searches are monitored and analyzed to produce accurate forecasts of imminent flu outbreaks. As with most of Google's products, 'Flu Trends' is available free of charge. Similarly, as with Google Maps, Google Earth, Gmail etc. there is undoubtedly a societal benefit that has resulted from the creation of such a product. 

The relationship between big data and the right to privacy requires ongoing scrutiny and modernised regulation, particularly within the EU. Among the many challenges for privacy protection are transparency and assigning accountability which are categorised as key problematic characteristics of data collection. The contradictory nature of transparency inherent in data mining has been referred to as the “Transparency Paradox”. Big data promises to use data to make the world more transparent, but its collection is invisible, and its tools and techniques are opaque, shrouded by layers of physical, legal, and technical privacy by design. If big data spells the end of privacy, then why is the big data revolution occurring mostly in secret? As such, following the opinion last week of the Advocate General in the Facebook case it appears that accountability of those responsible for the vast collection and retention of personal data is set to become an urgent source of concern for information privacy law.