This Article 
 Bibliographic References 
 Add to: 
Operation Shipping for Mobile File Systems
December 2002 (vol. 51 no. 12)
pp. 1410-1422

Abstract—This paper addresses a bottleneck problem in mobile file systems: the propagation of updated large files from a weakly-connected client to its servers. It proposes an efficient mechanism called operation shipping or operation-based update propagation. In the new mechanism, the client ships the user operation that updated the large files, rather than the files themselves, across the weak network. (In contrast, existing file systems use value shipping and ship the files.) The user operation is sent to a surrogate client that is strongly connected to the servers. The surrogate replays the user operation, regenerates the files, checks whether they are identical to the originals, and, if so, sends the files to the servers on behalf of the client. Care has been taken such that the new mechanism does not compromise correctness or server scalability. For example, we show how forward error correction (FEC) can restore minor reexecution discrepancies and, thus, make operation shipping work with more applications. Operation shipping can be further classified into two types: application-transparent and application-aware. Their feasibilities and benefits have been demonstrated by the design, implementation, and evaluation of a prototype extension to the Coda File System. In our controlled experiments, operation shipping achieved substantial performance improvements—network traffic reductions from 12 times to nearly 400 times and speedups in the range of 1.4 times to nearly 50 times.

[1] D. Bachmann, P. Honeyman, and L.B. Huston, "The Rx Hex," Proc. First Int'l Workshop Services in Distributed and Networked Environments, pp. 66-74, June 1994.
[2] A. Borg, W. Blau, W. Graetsch, F. Herrmann, and W. Oberle, "Fault Tolerance Under UNIX," ACM Trans. Computer Systems, vol. 7, no. 1, pp. 1-24, Feb. 1989.
[3] P.J. Braam, “The Coda Distributed File System,” Linux J., June 1998.
[4] F. Douglis and J. Ousterhout, "Transparent Process Migration: Design Alternatives and the Sprite Implementation," Software Practice&Experience, Vol. 21, Aug. 1991, pp. 757-785.
[5] Free Software Foundation. BASH—The Official Web Site, . 2002.
[6] http:/ 2002.
[7] Coda Group, Coda File System—The Official Web Site, http:/ 2002.
[8] A. Houghton, The Engineer's Error Coding Handbook. Chapman&Hall, 1997.
[9] K. Scott, “Review: AirSoft's AirAccess Keeps Everything in Sync,” Network Computing, vol. 6, no. 7, June 1995, also available from .
[10] P. Karn, Error Control Coding, a Seminar handout, available from 2002.
[11] J.J. Kistler, “Disconnected Operation in a Distributed File System,” PhD thesis, Carnegie Mellon Univ., School of Computer Science, 1993.
[12] J.J. Kistler and M. Satyanarayanan, "Disconnected Operation in the Coda File System," ACM Trans. Computer Systems, vol. 10, no. 1, Feb. 1992, pp. 3-25.
[13] L. Kleinrock, “Nomadic Computing—An Opportunity,” ACM Computer Comm. Review, pp. 36-40, year???
[14] P. Kumar and M. Satyanarayanan, “Log-Based Directory Resolution in the Coda File System,” Proc. Second Int'l Conf. Parallel and Distributed Information Systems, Jan 1993.
[15] O.S. Kylander and K. Kylander, GIMP: The Official Handbook, The Coriolis Group, Also available from, 1999.
[16] Y.W. Lee, Prototypes for Operation Shipping - Download Location, http://manual.gimp.org ~clementsource_code/, 2002.
[17] Y.W. Lee, “Operation-Based Update Propagation in a Mobile File System,” PhD thesis, The Chinese Univ. of Hong Kong, Dept. of Computer Science and Eng., Jan. 2000.
[18] Y.W. Lee, K.S. Leung, and M. Satyanarayanan, “Operation-Based Update Propagation in a Mobile File System” Proc. USENIX 1999 Ann. Technical Conf., June 1999.
[19] Q. Lu, “Improving Data Consistency for Mobile File Access Using Isolation-Only Transaction,” PhD thesis, Carnegie Mellon Univ., School of Computer Science, May 1996.
[20] Q. Lu and M. Satyanarayanan, “Improving Data Consistency in Mobile Computing Using Isolation-Only Transactions,” Proc. Fifth IEEE HotOS Topics Workshop, May 1995.
[21] Q. Lu and M. Satyanarayanan, “Resource Conservation in a Mobile Transaction System,” IEEE Trans. Computers, vol. 46 no. 3, Mar. 1997.
[22] J. MacDonald, “Versioned File Archiving, Compression, and Distribution,” submitted for the Data Compression Conf., an earlier version is available from . 1998.
[23] J. Mogul et al., "Potential Benefits of Delta-Encoding and Data Compression for HTTP," Proc. ACM SIGCOMM, ACM Press, New York, 1997, pp. 181-194. An extended and corrected version appears as Research Report 97/4a, Digital Equipment Corp. Western Research Laboratory, Dec. 1997.
[24] L.B. Mummert, “Exploiting Weak Connectivity in a Distributed File System,” PhD thesis, Carnegie Mellon Univ., School of Computer Science, 1996.
[25] L. Mummert, M. Ebling, and M. Satyanarayanan, "Exploiting Weak Connectivity for Mobile File Access," Proc. 15th ACM Symp. Operating Systems Principles, pp. 143-155, Dec. 1995.
[26] A. Muthitachareon, B. Chen, and D. Mazières, “A Low-Bandwidth Network File System,” Proc. 18th ACM Symp. Operating Systems Principles (SOSP '01), Oct. 2001.
[27] K. Patersen, M.J. Spreitzer, D.B. Terry, M.M. Theimer, and A.J. Demers, “Flexible Update Propagation for Weakly Consistent Replication,” Proc. 16th ACM Symp. Operating Systems Principles, Oct. 1997.
[28] The FreeBSD Documentation Project. CVSup: in FreeBSD Handbook, available from , 2002.
[29] R. Rivest, The MD5 Message-Digest Algorithm, Internet RFC 1321, available from , Apr. 1992.
[30] M. Satyanarayanan, “The Evolution of the Coda File System,” ACM Trans. Computer Systems, vol. 20, no. 2, pp. 85-124, May 2002.
[31] M. Satyanarayanan, “Fundamental Challenges in Mobile Computing,” Proc. 15th ACM Symp. Principles of Distributed Computing, May 1996.
[32] M. Satyanarayanan, M.R. Ebling, J. Raiff, P.J. Braam, and J. Harkes, “Coda File System User and System Administrators Manual,” School of Computer Science, Carnegie Mellon Univ., available from . 2000.
[33] M. Satyanarayanan, J. Kistler, P. Kumar, M. Okasaki, E. Siegel, and D. Steere, "Coda: A Highly Available File System for a Distributed Workstation Environment," IEEE Trans. Computers, vol. 39, no. 4, Apr. 1990.
[34] B. Schneier, Applied Cryptography, second ed. John Wiley&Sons, Inc., 1996.
[35] Cyclic Software. Concurrent Versions System (CVS), available fromhttp:/, 2002.
[36] A. Tridgell and P. Mackerras, “The RSYNC Algorithm,” Technical Report TR-CS-96-05, The Australian Nat'l Univ., available from, June 1996.

Index Terms:
Operation shipping, operation-based update propagation, mobile file systems, surrogate client, application-awareness, application-transparency, forward error correction, Coda File System.
Yui-Wah Lee, Kwong-Sak Leung, Mahadev Satyanarayanan, "Operation Shipping for Mobile File Systems," IEEE Transactions on Computers, vol. 51, no. 12, pp. 1410-1422, Dec. 2002, doi:10.1109/TC.2002.1146707
Usage of this product signifies your acceptance of the Terms of Use.