The field of information technology is full of similar, sometimes confusing terms. In recent years, we have seen the emergence of several new expressions, including “open data” and “big data”. These two very important terms mark the evolution of digital data in the age of the Internet. Do you know how to define them, and their role in the digital landscape?
BIG DATA: WHEN DATA ACCUMULATES
We live in a connected world where phenomenal quantities of data are stored at all hours of the day and night. According to IBM, 2.5 quintillion bytes of data are created on a daily basis. The term “big data” refers to the information that is accumulated and stored in real time on servers all over the world.
It should be noted that this data, even if it is retained, is not necessarily useful or usable. While it may have the potential to be used, it is estimated that less than 1% of the data is analyzed. Why? Its format and structure often make this task too difficult or too time-consuming. However, new tools are constantly being developed to tap into what is a gold mine for some businesses.
OPEN DATA: WHEN DATA IS OPEN TO EVERYONE
While big data is defined by its size, open data is defined by its use. The idea behind open data is to democratize information for the benefit of a group or population. Distributed under an open licence guaranteeing free access to it, open data must be made available without any technical, legal or financial restrictions.
This democratization is structured with a set of methods and formats that make it possible for data to be shared and understood. Businesses, or even individuals with programming skills, can then analyze the data, identify trends and inform the public. Open data provides tremendous opportunities for collaboration to resolve complex issues.
Check out our post on smart cities in the age of open data
A lot of open data comes from social networks where information is publicly accessible, such as Twitter. In addition, scientists are increasingly publishing their findings in open formats. This leads to very promising collaborations that move science forward.
A COUPLE OF IMPORTANT POINTS
Big data is not always open. Data gathered by mobile phone operators and banks will never be available to the public. Moreover, this data is of benefit only to those who have it, which could raise some ethical concerns.
Open data does not need to be big. Modest amounts of data can have a big impact when it is made public. Municipal information about certain services can help citizens make better decisions or improve their quality of life. For example, wouldn’t you like to know when the swimming pool is least crowded?
CAN THE TWO TYPES OF DATA BE COMBINED?
Open data and large quantities of data can absolutely be used together to provide a fuller, more visual and more understandable picture of the information.
Processing and cross-referencing lots of data, while certainly a technological challenge, allows information to be presented from new angles. The HubCab project from MIT’s Senseable City Lab is a particularly interesting example of this. In order to perform an extensive analysis of mobility in New York City, the team tracked 170 million taxi trips made in the city over one year. The trips were mapped using open data. The goal was to understand travel habits in the city in order to identify patterns that could be used to find ways of reducing the social and environmental costs of the trips. The principle of “taxi shareability” is central to this initiative. Their study found that 40% of taxi trips could be shared!
In conclusion, big data can be used to reach an unprecedented level of understanding and analysis, and open data ensures that this knowledge will be beneficial to everyone. Big data and open data are changing businesses, governments and society. And together, they are particularly powerful.
Source: The Guardian
DO YOU SEE OPPORTUNITIES FOR YOUR BUSINESS OR WANT TO KNOW MORE?