The Future of Cloud-stored Data is Compressed

The amount of data created and consumed globally is projected to grow to more than 180 zettabytes by 2025. That's 180,000,000,000,000,000,000,000 Bytes. Interestingly, only a small percentage of this newly created data is retained. In fact, just two percent of the data produced and consumed in 2020 was saved and carried over into 2021.

Minimizing Data Costs Without Sacrificing Growth

Companies moving to the cloud is old news, and so is the fact that cloud storage is expensive. But just how does this translate to the bottom line, and what actions are notable companies like Dropbox and 37Signals taking to minimize it?

Dropbox, for example, managed an estimated $75 million in savings over two years by migrating from AWS to its own data centers in 2017. Similarly, 37signals was spending over $500,000 per year on data storage on AWS and Google Cloud before moving its applications to a colocation facility. Environmental impact notwithstanding, the aforementioned organizational optimization efforts clearly point to a trend where near-complete cloud reliance is no longer the modus operandi. 

SMEs and enterprises around the world are becoming aware that their cloud migration strategy has incurred, and continues to incur, unexpected costs, both financially and environmentally. Data centers, which comprise an estimated 7% of the global energy demand, have become topics of national discussions. 

Enterprises Brace for the Flood

Organizations worldwide are increasingly recognizing the need for a Data Center Infrastructure Sustainability Program to reduce their carbon footprint and optimize their storage and networking costs. In addition, Cloud computing statistics from IDC predict a 10.9% growth rate in demand for building better enterprise storage solutions.

One of the leading pharmaceutical and biotechnology companies, AstraZeneca, has tapped into data compression to optimize its genomics datasets. AstraZeneca’s Center for Genomics Research (CGR) aims to analyze up to two million genomes by 2026, which requires minimizing the storage footprint and transfer time of genome data. By using a data compression solution, AstraZeneca has achieved an average data size reduction of 76%, enabling faster and more efficient genomic research.

Similarly, Medicat, a provider of healthcare information technology to universities and colleges, recognized the need for data compression as it faced rapidly growing data demands. This strategic move allowed Medicat to optimize performance and reduce infrastructure requirements, demonstrating the institution’s recognition of the critical role of data compression in managing large-scale data.

With over 20% of all files stored in data centers being compressed files, and considering that 95% of data is expected to be stored in cloud environments by 2025 and that 67% of enterprise infrastructure is already cloud-based, institutional recognition of utilizing data compression technologies to mitigate soaring storage, networking, and environmental cloud costs is clear.

Software or Hardware Compression: Roll the Tape or Execute the Command?

Rising trends in enterprise storage solutions aimed at mitigating escalating costs are novel data compression and tape compression. Looking at the storage challenge from both a software and hardware perspective presents interesting optimization opportunities. Hardware compression, exclusive to tape drives, enhances backup data storage. Software compression occurs on the data’s host machine before data transfer, increasing the remote server’s CPU load but saving bandwidth. Conversely, hardware compression, performed by the tape drive, doesn’t burden the CPU of remote or media servers, but sends uncompressed data, generally requiring more bandwidth.

The Future, Compressed

In the face of exponential data growth, businesses are reevaluating their reliance on cloud storage due to financial and environmental implications. Pioneers like Dropbox and 37Signals have demonstrated significant cost savings by optimizing their data storage strategies. 

The increasing institutional recognition of data compression technologies, both software and hardware-based, is a testament to their potential in mitigating storage, networking, and environmental costs. 

These technologies are not just solutions for today, but strategic investments for a sustainable and cost-effective future.