These are some of the relevant factors that have contributed to the rise of new concepts like Internet of Things (IOT for short) and Big Data - terms that have since left the realm of academia and entered the mainstream.
IPv4 to IPv6 Transition
We are right in the middle of a large transition from old-fashioned IPv4 to IPv6, but what does it mean for us? In comparison with IPv4, IPv6 supports 10²⁸ as many endpoints!
This means that it will be possible for every single device (no matter if we're talking about thousands of heat sensors in a forest) to be connected to a network interface - being able to send and receive data from potentially any other Internet-ready device in the world. Any tiny piece of hardware could, theoretically, be uniquely identifiable via an IP.
IPv6 supports 10²⁸ more addresses than IPv4
Explosion of Devices and Data
In addition to the wider amount of addresses available through IPv6, the cost of hardware has gone down over the last few years, while newer and faster CPUs and hard disks have been developed.
This has contributed to what is being called the commoditization of processing power and storage space.
Last year (2013), there were over 10 billion connected devices, and this number will climb as high as 50 billion by 2020, according to an estimate by networking equipment maker Cisco. source
The hockey stick effect
- GI Systems
GIS, short for Georaphical Information Systems is the umbrella term for systems whose objective is to store large quantities of coordinates and/or some extra information related to them.
With the increase in the number of mobile and handheld devices, as well as the aforementioned explosion in the number of overall (including static) devices, it has become ever more convenient to store event locations and/or user actions as defined points in time and space in GISs.
Sensors are becoming economically viable for many industry sectors such as manufacturing, agriculture, energy generation and so on.
Each sensor typically emits data at a predefined rate or when some threshold conditions are met. This means lots of data gets sent to a database and needs to be acted upon, sometimes even in real time.
- Social Media
User-generated content is rising to heights never before seen, now that large populations (which until very recently didn't have access to the Internet) are becoming regular Internet users all around the world.
It is hard to evaluate if social media has been more of a consequence of this phenomenon than one of the causes thereof, but social networks are among the organizations where most data is being kept nowadays - many popular open source tools for big data manipulation originated in places like Facebook, Google and Yahoo.
Disk space has become so cheap that most devices and applications are configured to log everything that can be logged in the off chance it might some day, somehow, be useful for someone.