July 3, 2006 to July 5, 2006
Ekow J. Otoo , Lawrence Berkeley National Laboratory, UC Berkeley
Doron Rotem , Lawrence Berkeley National Laboratory, UC Berkeley
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/SSDBM.2006.24
Data analyses in scientific domains involve storage, retrieval, processing and visualization of large scale multidimensional datasets. The datasets incrementally grow by appending new data to the dataset without reorganizing the already allocated data storage. The datasets, typically modeled as k-dimensional arrays, are maintained in files where the array elements are allocated in a sequence of consecutive storage locations according to some ordering of an array mapping function. Such mapping functions limit the degree of extendibility of the arrays to one dimension only. To allowing new data elements to be appended to the dataset effectively implies allowing for the arbitrary extendibility of the array. We present a mapping function F.(), that uses additional O(k log E ) storage, where E is the number of array segments appended. The function is realized with approximately the approximately the same order of complexity as a conventional array function. The algorithm presented for addressing elements of datasets in extendible multi-dimensional array files apply equally well to addressing memory resident extendible arrays.
Multi-dimensional array, scientific dataset, extendible array, computed array addressing function, out-of-core array storage.
Ekow J. Otoo, Doron Rotem, "Efficient Storage Allocation of Large-Scale Extendible Multi-dimensional Scientific Datasets", SSDBM, 2006, Scientific and Statistical Database Management, International Conference on, Scientific and Statistical Database Management, International Conference on 2006, pp. 179-183, doi:10.1109/SSDBM.2006.24