Directions in Enterprise Data Storage Systems
Businesses’ data storage needs are always increasing, and nothing indicates that we will be storing less data in the future. On the contrary, everything points to an era where data management will gain center stage in computing. Data loss is a disaster and can have significant costs to consumers and businesses; if data is unavailable, even for minutes, it can cause productivity losses or cost millions in revenue. CIOs admit that their biggest expenditure is in managing storage systems, and managing storage is the most difficult part of IT management. This month, Computing Now brings together a few articles that deal with the contemporary issues concerning storage systems and their management.
The need for storage arises from the need for persistence of data or memory. File systems evolved in operating systems to be the bridge between volatile and persistent memory devices. Different file systems evolved in operating systems to address the varying characteristics and capabilities of these devices in the context of application demands for performance.
Ever-increasing demands for access to data have led to different methods of organizing and managing it. The evolution from data or transaction processing to business intelligence is leading to tradeoffs in the way we organize and search raw data. The explosive growth of unstructured data brings new challenges in organization and retrieval of meaningful information.
The implicit demand to eliminate data loss and data unavailability drives the development of highly reliable, highly available storage systems. Distributing storage elements to increase capacity and availability across a wider geographic area is becoming common practice. Additionally, the desire for the lowest cost per Gbyte has resulted in advances in storage devices, systems, and equipment. It has also led to the development of a hierarchy in storage systems in operation. The large-scale growth of data has led system engineers to create storage systems assembled from numerous industry-standard, commodity storage devices. However, these elements must appear as a single, reliable shared storage service to the applications.
Overall, storage systems offer exciting times for engineers and researchers.
The articles in this month’s theme illustrate a variety of challenges storage researchers are addressing. In “A Taxonomy and Survey on Distributed File Systems,” (login required for full text) Tran Doan Thanh and colleagues introduce a framework to compare architectural approaches to building distributed file systems. David Geer presents an overview of deduplication technology in “Reducing the Storage Burden via Data Deduplication.” (login required for full text) This technology dramatically reduces the need for storage capacity and hence capital and operating costs. “Using Intradisk Parallelism to Build Energy Efficient Storage Systems” (login required for full text) by Sudhanva Gurumurthi, Mircea R. Stan, and Sriram Sankar covers an important aspect of storage systems: energy efficiency. “Reliable Distributed Storage” (login required for full text) by Gregory Chockler and colleagues discusses algorithms that implement the abstractions and tradeoffs in developing modern storage systems. Neal Leavitt reviews the trend of organizations to increasingly collect unstructured data and search for information—and say “no” to SQL or relational databases—in “Will NoSQL Databases Live Up to Their Promise?” (login required for full text) Shravan Gaonkar and colleagues present an automated approach to designing dependable storage systems in “Designing Dependable Storage Solutions for Shared Application Environments.” (login required for full text)
David H.C. Du’s article, “Recent Advancements and Future Challenges of Storage Systems,” (login required for full text) is a treasure and will give newcomers to the storage domain a quick overview of the concepts and issues, with an excellent collection of references. (IEEE Xplore login is required for this article.)
We are interested to hear your perspectives on the trends, hard problems, and advances in storage in the coming decade. For instance, what do you perceive to be the hardest and most impactful problem to solve in this decade concerning storage systems? What’s the strongest trend you’re observing about how storage integrates with the rest of the data center elements—servers, networking gear, application software, and so on? In your opinion, how will cloud storage service affect the way storage is used in enterprises? Please let us know in the comments below.