The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2011 vol.22)
pp: 860-873
Lanyue Lu , Rice University, Houston
Peter J. Varman , Rice University, Houston
Kshitij Doshi , Intel Corporation, Chandler
ABSTRACT
The growing popularity of hosted storage services and shared storage infrastructure in data centers is driving the recent interest in resource management and QoS in storage systems. The bursty nature of storage workloads raises significant performance and provisioning challenges, leading to increased resource requirements, management costs, and energy consumption. We present a novel workload shaping framework to handle bursty workloads, where the arrival stream is dynamically decomposed to isolate its bursts, and then rescheduled to exploit available slack. We show how decomposition reduces the server capacity requirements and power consumption significantly, while affecting QoS guarantees minimally. We present an optimal decomposition algorithm RTT and a recombination algorithm Miser, and show the benefits of the approach by evaluating the performance of several storage workloads using both simulation and Linux implementation.
INDEX TERMS
Workload decomposition, graduated QoS, storage system, resource management, scheduling.
CITATION
Lanyue Lu, Peter J. Varman, Kshitij Doshi, "Decomposing Workload Bursts for Efficient Storage Resource Management", IEEE Transactions on Parallel & Distributed Systems, vol.22, no. 5, pp. 860-873, May 2011, doi:10.1109/TPDS.2010.129
REFERENCES
[1] W. Jin, J.S. Chase, and J. Kaur, "Interposed Proportional Sharing for a Storage Service Utility," Proc. ACM SIGMETRICS, 2004.
[2] J. Zhang, A. Sivasubramaniam, Q. Wang, A. Riska, and E. Riedel, "Storage Performance Virtualization via Throughput and Latency Control," Proc. IEEE Int'l Symp. Modeling, Analysis, and Simulation of Computer and Telecomm. Systems (MASCOTS), 2005.
[3] A. Gulati, A. Merchant, and P. Varman, "Clock: An Arrival Curve Based Approach for QoS in Shared Storage Systems," Proc. ACM SIGMETRICS, 2007.
[4] C. Lumb, A. Merchant, and G. Alvarez, "Façade: Virtual Storage Devices with Performance Guarantees," Proc. Conf. File and Storage Technologies (FAST), 2003.
[5] M. Aron, P. Druschel, and W. Zwaenepoel, "Cluster Reserves: A Mechanism for Resource Management in Cluster-Based Network Servers," Proc. ACM SIGMETRICS, 2000.
[6] M.E. Gómez and V. Santonja, "On the Impact of Workload Burstiness on Disk Performance," Workload Characterization of Emerging Computer Applications, Kluwer Academic Publishers, 2001.
[7] W.E. Leland, S.T. M, W. Willinger, and D.V. Wilson, "On the Self-Similar Nature of Ethernet Traffic," IEEE/ACM Trans. Networking, vol. 2, no. 1, pp. 1-15, Feb. 1994.
[8] A. Riska and E. Riedel, "Long-Range Dependence at the Disk Drive Level," Proc. Int'l Conf. Quantitative Evaluation of Systems (QEST), 2006.
[9] N. Mi, Q. Zhang, A. Riska, E. Smirni, and E. Riedel, "Performance Impacts of Autocorrelated Flows in Multi-Tiered Systems," Performance Evaluation, vol. 64, nos. 9-12, pp. 1082-1101, 2007.
[10] E.W. Knightly and N.B. Shroff, "Admission Control for Statistical qos: Theory and Practice," IEEE Network, vol. 13, no. 2, pp. 20-29, Mar./Apr. 1999.
[11] L. Lu, P.J. Varman, and K. Doshi, "Graduated qos by Decomposing Bursts: Don't Let the Tail Wag Your Server," Proc. IEEE Int'l Conf. Distributed Computing Systems (ICDCS), 2009.
[12] "Public Software (Storage Systems Department at hp Labs)," http://tesla.hpl.hp.compublicsoftware/, June 2007.
[13] D. Narayanan, A. Donnelly, E. Thereska, S. Elnikety, and A. Rowstron, "Everest: Scaling Down Peak Loads through I/O Off-Loading," Proc. Symp. Operating Systems Design and Implementation (OSDI), 2008.
[14] "Storage Performance Council (umass Trace Repository)," http://traces.cs.umass.edu/index.phpStorage , June 2007.
[15] J.S. Chase, D.C. Anderson, P.N. Thakar, A.M. Vahdat, and R.P. Doyle, "Managing Energy and Server Resources in Hosting Centers," Proc. ACM Symp. Operating Systems Principles (SOSP), 2001.
[16] G. Chen, W. He, J. Liu, S. Nath, L. Rigas, L. Xiao, and F. Zhao, "Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services," Proc. Symp. Networked Systems Design and Implementation (NSDI), 2008.
[17] Q. Zhu, Z. Chen, L. Tan, Y. Zhou, K. Keeton, and J. Wikes, "Hibernator: Helping Disk Arrays Sleep through the Winter," Proc. ACM Symp. Operating Systems Principles (SOSP), 2005.
[18] D. Meisner, B.T. Gold, and T.F. Wenisch, "PowerNap: Eliminating Server Idle Power," Proc. Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2009.
[19] "HDFS: Architecture and Design," http:/hadoop.apache.org/, Aug. 2008.
[20] S. Ghemawat, H. Gobioff, and S.-T. Leung, "The Google File System," Proc. ACM Symp. Operating Systems Principles (SOSP), 2003.
[21] J.C.R. Bennett and H. Zhang, "${WF}^2{\rm Q}$ : Worst-Case Fair Weighted Fair Queueing," Proc. IEEE INFOCOM, 1996.
[22] P. Goyal, H.M. Vin, and H. Cheng, "Start-Time Fair Queueing: A Scheduling Algorithm for Integrated Services Packet Switching Networks," IEEE/ACM Trans. Networking, vol. 5, no. 5, pp. 690-704, Oct. 1997.
[23] A. Gulati and P. Varman, "RFQ: Redemptive Fair Queuing," Proc. Ann. European Symp. Algorithms (ESA), 2008.
[24] http://www.pdl.cmu.eduDiskSim/, 2010.
[25] D. Narayanan, A. Donnelly, and A. Rowstron, "Write Off-Loading: Practical Power Management for Enterprise Storage," Proc. Conf. File and Storage Technologies (FAST), 2008.
[26] "Flexible IO Tester," http://freshmeat.net/projectsfio/, 2010.
[27] P.J. Shenoy and H.M. Vin, "Cello: A Disk Scheduling Framework for Next Generation Operating Systems," Proc. ACM SIGMETRICS, 1998.
[28] J.C.R. Bennett and H. Zhang, "${WF}^2$ Q: Worst-Case Fair Weighted Fair Queueing," Proc. IEEE INFOCOM, 1996.
[29] A. Demers, S. Keshav, and S. Shenker, "Analysis and Simulation of a Fair Queuing Algorithm," J. Internet Working Research and Experience, vol. 1, pp. 3-26, Sept. 1990.
[30] M. Harchol-Balter, B. Schroeder, N. Bansal, and M. Agrawal, "Size-Based Scheduling to Improve Web Performance," ACM Trans. Computer Systems, vol. 21, no. 2, pp. 207-233, 2003.
[31] N. Mi, G. Casale, and E. Smirni, "Scheduling for Performance and Availability in Systems with Temporal Dependent Workloads," Proc. Int'l Conf. Dependable Systems and Networks (DSN), 2008.
[32] Q. Zhang, N. Mi, A. Riska, and E. Smirni, "Load Unbalancing to Improve Performance under Autocorrelated Traffic," Proc. IEEE Int'l Conf. Distributed Computing Systems (ICDCS), 2006.
[33] J.W. Evans and C. Filsfils, Deploying ip and mpls qos for Multiservice Networks. Morgan Kauffman, 2007.
[34] S. Floyd and V. Jacobson, "Random Early Detection Gateways for Congestion Avoidance," IEEE/ACM Trans. Networking, vol. 1, no. 4, pp. 397-413, Aug. 1993.
34 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool