|
| This Article | ||
| ||
| Share | ||
| Bibliographic References | ||
| Add to: | ||
 Digg  Furl  Spurl  Blink  Simpy  Google  Del.icio.us  Y!MyWeb | ||
| Search | ||
| ||
| ASCII Text | x | ||
| Chye Lin Chee, Hongjun Lu, Hong Tang, C.v. Ramamoorthy, "Adaptive Prefetching and Storage Reorganization In A Log-Structured Storage System," IEEE Transactions on Knowledge and Data Engineering, vol. 10, no. 5, pp. 824-838, September/October, 1998. | |||
| BibTex | x | ||
| @article{ 10.1109/69.729739, author = {Chye Lin Chee and Hongjun Lu and Hong Tang and C.v. Ramamoorthy}, title = {Adaptive Prefetching and Storage Reorganization In A Log-Structured Storage System}, journal ={IEEE Transactions on Knowledge and Data Engineering}, volume = {10}, number = {5}, issn = {1041-4347}, year = {1998}, pages = {824-838}, doi = {http://doi.ieeecomputersociety.org/10.1109/69.729739}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, } | |||
| RefWorks Procite/RefMan/Endnote | x | ||
| TY - JOUR JO - IEEE Transactions on Knowledge and Data Engineering TI - Adaptive Prefetching and Storage Reorganization In A Log-Structured Storage System IS - 5 SN - 1041-4347 SP824 EP838 EPD - 824-838 A1 - Chye Lin Chee, A1 - Hongjun Lu, A1 - Hong Tang, A1 - C.v. Ramamoorthy, PY - 1998 KW - Adaptive prefetching KW - storage systems KW - database management systems KW - storage reorganization. VL - 10 JA - IEEE Transactions on Knowledge and Data Engineering ER - | |||
Abstract—We present a storage management system that has the ability to adapt to the data access characteristics of the application that uses it based on collection and analysis of runtime statistics. This feature is especially useful in the storage management layer of database systems, where applications exhibit relatively predictable access patterns. Adaptive reorganization is performed by the storage management system in a manner that optimizes the access patterns of the system for which it is used. We enhance the log-structured storage system that naturally caters for write optimization, with the addition of a statistics collection mechanism to determine data access patterns of applications. The storage system can serve as a testbed for a variety of statistics analysis and clustering mechanisms. Higher level application-specific data clustering mechanisms can be used to override the storage system's low-level clustering mechanisms. In addition, the analysis techniques and reorganization scheme can be used in other storage systems. Performance results from our prototype show potential response time speedups of up to 83 percent over the basic log-structured file system in the best case, using a combination of storage reorganization and prefetching.
[1] M.J. Carey, D.J. DeWitt, and J.F. Naughton, “The OO7 Benchmark,” Proc. ACM-SIGMOD Int'l Conf. Management of Data, pp. 12–21, May 1993.
[2] On the technical history of System R, see D.D. Chamberlin et al., "A History and Evaluation of System R," Comm. ACM, vol. 24, no. 10, 1981, pp. 632-646. On the history of database design, generally see J. Gray, "Evolution of Data Management,"Computer,vol. 29, no. 10, 1996, pp. 38-46, and A. Silberschatz, M. Stonebraker, and J. Ullman, "Database Systems: Achievements and Opportunities,"Comm. ACM, vol. 34, no. 10, 1991, pp. 110-120. For a brief popular gloss on System R, see S. Lohr,Go To,Basic Books, New York, 2001, pp. 161-68.
[3] C.L. Chee, "Active Storage Management for Database Systems," PhD dissertation, Univ. of California, Berkeley, Aug. 1995.
[4] O. Dexu et al., "The Story of O2," IEEE Trans. Knowledge and Data Eng., pp. 91-108, Mar. 1990.
[5] D. Fishman, "IRIS: An Object-Oriented Database Management System," ACM Trans. Office Information Systems, vol. 5, no. 1, pp. 48-69, 1987.
[6] G.R. Ganger, "Disk Arrays: High-Performance, High-Reliability Storage Subsystems," Computer, vol. 27, no. 3, pp. 30-36, Mar. 1994.
[7] K.S. Grimsrud, J.K. Archibald, and B.E. Nelson, "Multiple Prefetch Adaptive Disk Caching," IEEE Trans. Knowledge and Data Eng., Feb. 1993, pp. 88-103.
[8] R. Katz, G. Gibson, and D. Patterson, “Disk System Architectures for High Performance Computing,” Proc. IEEE, vol. 77, no. 12, pp. 1,842–1,858, Dec. 1989.
[9] S.B. Kim et al., "Threaded Prefetching: An Adaptive Instruction Prefetch Mechanism," Microprocessing and Microprogramming, vol. 39, no. 1, pp. 1-15, Nov. 1993.
[10] W. Kim, J. Garza, N. Ballou, and D. Woelk, "Architecture of the Orion Next-Generation Database System," IEEE Trans. Knowledge and Data Eng., vol. 2, no. 1, pp. 109-124 Mar. 1990.
[11] S.J. Leffler, M.K. McKusick, M.J. Karels, and J.S. Quarterman, Design and Implementation of the 4.3BSD Unix Operating System.Reading, Mass.: Addison-Wesley, 1989.
[12] S.T. Leutenegger and D. Dias, "A Modeling Study of the TPC-C Benchmark," Proc. ACM SIGMOD, pp. 33-40, May 1993.
[13] M.K. McKusick, W. Joy, S. Leffler, and R. Fabry, "A Fast File System for UNIX," ACM Trans. Computer Systems, vol. 2, no. 3, pp. 181-197, Aug. 1984.
[14] B. McNutt, "Background Data Movement in A Log-Structured Disk Subsystem," IBM J. Research and Development, vol. 38, no. 1, pp. 47-58, Jan. 1994.
[15] E. Omiecinski, L. Lee, and P. Scheuermann, "Concurrent File Reorganization for Record Clustering: A Performance Study," Proc. Eighth Int'l Conf. Data Eng., pp. 265-272, Feb. 1992.
[16] J. Ousterhout, Tcl and the Tk Toolkit, Addison Wesley Longman, Reading, Mass., 1994.
[17] D.A. Patterson, G. Gibson, and R.H. Katz, “A Case for Redundant Arrays of Inexpensive Disks (RAID),” Proc. ACM SIGMOD Conf., pp. 109–116, 1988.
[18] E.G. de Paula and M.L. Nelson, "Clustering in Object-Oriented Databases," OOPS Messenger, vol. 3, no. 3, pp. 14-21, July 1992.
[19] M. Rosenblum and J.K. Ousterhout, "The Design and Implementation of a Log-Structured File System," ACM Trans. Computer Systems, vol. 10, no. 1, Feb. 1992.
[20] J.T. Robinson and P.A. Franazek, "Analysis of Reorganization Overhead In Log-Structured File Systems," Proc. 10th Int'l Conf. Data Eng., pp. 102-110, Feb. 1994.
[21] M. Seltzer, K.A. Smith, H. Balakrishnan, and J. Chang, "File System Logging vs. Clustering: A Performance Comparison," Proc. Usenix Technical Conf., pp. 249-264, Jan. 1995.
[22] F.W. Shih, "A File-Based Adaptive Prefetch Caching Design," Proc. IEEE Int'l Conf. Computer Design: VLSI in Computers and Processors, pp. 463-466, Sept. 1990.
[23] M. Stonebraker, "Retrospection on a Database System," ACM Trans. Database Systems, vol. 5, no. 2, pp. 225-240, 1980.
[24] M. Stonebraker,L. Row, and M. Hirohama,"The implementation of POSTGRES," IEEE Trans. Knowledge and Data Engineering, vol. 2, no. 7, pp. 125-142, Mar. 1990.
[25] Transaction Processing Performance Council, TPC Benchmark C, 1993.
[26] P. Vongsathorn and S.D. Carson, "A System for Adaptive Disk Rearrangement," Software—Practice and Experience, vol. 20, no. 3, pp. 225-242, Mar. 1990.
[27] J. Widom and S.J. Finkelstein,"Set-oriented production rules in relational database systems," Proc. 1990 ACM SIGMOD Int'l Conf. Management of Data, pp. 259-270, 1990.
[28] C.T. Yu, C.M. Suen, K. Lam, and M.K. Siu, "Adaptive Record Clustering," ACM Trans. Computer Systems, vol. 10, no. 2, pp. 180-204, June 1985.
[29] T. Zhang, R. Ramakrishnan, and M. Livny, "Birch: An Efficient Data Clustering Method for Very Large Databases," Proc. ACM SIGMOD Int'l Conf. Management of Data, ACM Press, 1996, pp. 103-114.
[30] R.G. Rosandich, "Havnet: A New Neural Network Architecture for Pattern Recognition," Neural Networks, vol. 10, no. 1, pp. 139-51, Jan. 1997.
[31] E. Omiecinski, "Incremental File Reorganization Schemes," Proc. VLDB Conf.,Stockholm, pp. 346-357, Aug. 1985.