Despite headlines, reports and presentations on the value of ‘Big Data’ when it comes to data, bigger isn’t necessarily better – and it is often not very open…
‘Big Data’
Big Data refers to large and complex sets of data which have come from the use of electronic controls and the storage of data arising from them. ‘Big data’ is not necessarily any more valuable, relevant or significant than any other kind of data. People talk about more than they used to because there is more of it about – so many everyday processes are now sensed, modelled or controlled electronically. Examples:
- money increasingly exists as a digital currency accounted for online rather than as physical coins and notes. Every time digital money moves, it leaves a trail of data behind it
- NHS records are increasingly held on computer – so the data about different medical conditions can be compared and correlated in ways that used not to be possible
- sales in supermarkets are recorded by electronic scanners – which spend all day collecting data
- all sorts of networks for moving people, objects and power around are controlled electronically – using sensors which collect data.
None of these processes was originally designed with the collection of data in mind, but they all produce masses of data. It is used, in the first instance, for managing the processes from which it has been derived. The data can, however, be used and combined in innovative and unexpected ways – to make new information.
Big isn’t necessarily beautiful
The point about Big Data, however, is that the data sets are so large and complex that traditional approaches to data processing tend not to be able to cope with them. Big Data is likely to contain very useful potential information, but unlocking it isn’t easy. Big Data is rarely very open data – it tends to belong to whichever large organisation is responsible for the process from which it derives; it can be structured in a way that makes it hard to re-use; and personal data within it is governed by Data Protection laws which makes it harder to use.
Can you think of some data that you know is routinely collected from your area which could be combined with other data to form really useful information? House prices and rents? Benefits information? Canvassing returns? Utility bills? NHS records? What would be the main problems in getting hold of and using this data in practice?
Now read about
setting up a local data observatory
OR – follow the menu on the right to have a look at other parts of the guide.