This Article 
 Bibliographic References 
 Add to: 
Resource Policing to Support Fine-Grain Cycle Stealing in Networks of Workstations
October 2004 (vol. 15 no. 10)
pp. 878-892
Kyung Dong Ryu, IEEE Computer Society

Abstract—This paper presents the design, implementation, and performance evaluation of a suite of resource policing mechanisms that allow guest processes to efficiently and unobtrusively exploit otherwise idle workstation resources. Unlike traditional policies that harvest cycles only from unused machines, we employ fine-grained cycle stealing to exploit resources even from machines that have active users. We developed a suite of kernel extensions that enable these policies to operate without significantly impacting host processes: 1) a new starvation-level CPU priority for guest jobs, 2) a new page replacement policy that imposes hard bounds on physical memory usage by guest processes, and 3) a new I/O scheduling mechanism called rate windows that throttle guest processes' usage of I/O and network bandwidth. We evaluate both the individual impacts of each mechanism, and their utility for our fine-grain cycle stealing.

[1] M.W. Mutka and M. Livny, The Available Capacity of a Privately Owned Workstation Environment Performance Evaluation, vol. 12, no. 4, pp. 269-284, 1991.
[2] I. Foster and C. Kesselman, The Grid: Blueprint for a New Computing Infrastructure. San Francisco: Morgan-Kaufmann, 1998.
[3] F. Berman, G. Fox, and A. Hey, Grid Computing: Making the Global Infrastructure a Reality. Wiley, 2003.
[4] M. Litzkow, M. Livny, and M.W. Mutka, “Condor—A Hunter of Idle Workstations,” Proc. Eighth Int'l Conf. Distributed Computing Systems, Jun. 1988.
[5] K.D. Ryu, J.K. Hollingsworth, and P. Keleher, Efficient Network and I/O Throttling for Fine-Grain Cycle Stealing Proc. Supercomputing Conf., 2001.
[6] K.D. Ryu and J.K. Hollingsworth, Linger Longer: Fine-Grain Cycle Stealing for Networks of Workstations Proc. Supercomputing Conf., 1998.
[7] K.D. Ryu and J. Hollingsworth, Exploiting Fine Grained Idle Periods in Networks of Workstations IEEE Trans. Parallel and Distributed Systems, vol. 11, no. 7, pp. 683-698, July 2000.
[8] A. Barak, O. Laden, and Y. Yarom, The NOW Mosix and its Preemptive Process Migration Scheme Bull. IEEE Technical Committee on Operating Systems and Application Environments, vol. 7, no. 2, pp. 5-11, 1995.
[9] J.C. Mogul and A. Borg, The Effect of Context Switches on Cache Performance Proc. Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), pp. 75-84, 1991.
[10] B.L. Jacob and T.N. Mudge, A Look at Several Memory Management Units, TLB-Refill Mechanisms, and Page Table Organizations Proc. Int'l Conf. Architectural Support for Programming Languages and Operating Systems, pp. 295-306, 1998.
[11] A. Tamches and B.P. Miller, Fine-Grained Dynamic Instrumentation of Commodity Operating System Kernels Proc. Third Symp. Operating Systems Design and Implementation (OSDI), pp. 117-130, 1999.
[12] W.R. Carr and J.L. Hennessy, WSClock A Simple and Effective Algorithm for Virtual Memory Management Proc. ACM Symp. Operating System Principles, pp. 87-95, 1981.
[13] J.S. Heidemann and G.J. Popek, File-System Development with Stackable Layers ACM Trans. Computer Systems, vol. 12, no. 1, pp. 58-89, 1994.
[14] K.J. McDonell, Taking Performance Evaluation Out of the‘Stone Age’ Proc. Summer USENIX Conf., pp. 8-12, 1987.
[15] S. Woo, M. Ohara, E. Torrie, J.P. Singh, and A. Gupta, “The SPLASH-2 Programs: Characterization and Methodological Considerations,” Proc. Int'l Symp. Computer Architecture, pp. 24-36, June 1995.
[16] C. Amza, A.L. Cox, S. Dwarkadas, P. Keleher, H. Lu, R. Rajamony, W. Yu, and W. Zwaenepoel, “TreadMarks: Shared Memory Computing on Networks of Workstations,” Computer, vol. 29, no. 2, Feb. 1996.
[17] P. Keleher, “The Relative Importance of Concurrent Writers and Weak Consistency Models,” Proc. 16th Int'l Conf. Distributed Computing Systems, May 1996.
[18] J.B. Chen, Memory Behavior for an X11 Window System Proc. USENIX Winter Conf., pp. 189-200, 1994.
[19] K. Flautner, R. Uhlig, S. Reinhardt, and T. Mudge, Thread-Level Parallelism and Interactive Performance of Desktop Applications Proc. Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), pp. 129-138, 2000.
[20] L. Smith and C. Laird, Android, Open Source Scripting for Testing&Automation Dr. Dobbs J., vol. 326, pp. 58-61, 2001.
[21] D.H. Bailey, E. Barszcz, J.T. Barton, and D.S. Browning, The NAS Parallel Benchmarks Int'l J. Supercomputer Applications, vol. 5, no. 3, pp. 63-73, 1991.
[22] S. Zhou, X. Zheng, J. Wang, and P. Delisle, Utopia: A Load Sharing Facility for Large, Heterogeneous Distributed Computer Systems SPE, vol. 23, no. 12, pp. 1305-1336, 1993.
[23] R.H. Arpaci, A.C. Dusseau, A.M. Vahdat, L.T. Liu, T.E. Anderson, and D.A. Patterson, The Interaction of Parallel and Sequential Workloads on a Network of Workstations Proc. SIGMETRICS, pp. 267-278, 1995.
[24] P. Krueger and R. Chawla, "The Stealth Distributed Schedular," Proc. 11th Int'l Conf. Distributed Computing Systems, pp. 336-343, May 1991.
[25] SiliconGraphics, IRIX 6.4 Technical Brief, , 1998.
[26] B. Verghese, A. Gupta, and M. Rosenblum, Performance Isolation: Sharing and Isolation in Shared-Memory Multiprocessors Proc. Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), pp. 181-192, 1998.
[27] M. Aron and P. Durschel, Soft Timers: Efficient Microsecond Software Timer Support for Network Processing Proc. ACM Symp. Operating Systems Principles (SOSP), pp. 232-246, 1999.
[28] J. Bruno, E. Gabber, B. Ozden, and A. Silberschatz, The Eclipse Operating System: Providing Quality of Service via Reservation Domains Proc. USENIX Ann. Technical Conf., pp. 235-246, 1998.
[29] G. Banga, P. Druschel, and J. Mogul, Resource Containers: A New Facility for Resource Management in Server Systems Proc. USENIX Third Symp. Operating System Design and Implementation, pp. 45-58, 1999.
[30] D. Reed and R. Fairbairns, The Nemesis Kernel United Feature Syndicate, Inc., 1997.
[31] J.S. Turner, "New Directions in Communications (Or Which Way to the Information Age)," IEEE Comm. Magazine, vol. 24, no. 10, pp. 8-15, Oct. 1986.
[32] J. Csirik, J.B.G. Frenk, M. Labbe, and S. Zhang, On the Multidimensional Vector Bin Packing Acta Cybernetica, vol. 9, no. 4, pp. 361-369, 1990.
[33] T. Faber, L.H. Landweber, and A. Mukherjee, Dynamic Time Windows: Packet Admission Control with Feedback Proc. SIGCOMM, pp. 124-135, 1992.
[34] R.P. Goldberg, Survey of Virtual Machine Research IEEE Computer Magazine, vol. 7, no. 6, pp. 34-45, 1974.
[35] E. Bugnion, S. Devine, and M. Rosenblum, Disco: Running Commodity Operating Systems on Scalable Multiprocessors Proc. ACM Symp. Operating Systems Principles (SOSP), pp. 143-156, 1997.

Index Terms:
Grid computing, cluster computing, networks of workstations, parallel computing, resource scheduling, cycle stealing.
Kyung Dong Ryu, Jeffrey K. Hollingsworth, "Resource Policing to Support Fine-Grain Cycle Stealing in Networks of Workstations," IEEE Transactions on Parallel and Distributed Systems, vol. 15, no. 10, pp. 878-892, Oct. 2004, doi:10.1109/TPDS.2004.58
Usage of this product signifies your acceptance of the Terms of Use.