loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
The sqlLoader Data-Loading Pipeline
January/February 2008 (vol. 10 no. 1)
pp. 38-48
ASCII Text
x 
 
Alex Szalay, Ani R. Thakar, Jim Gray, "The sqlLoader Data-Loading Pipeline," Computing in Science and Engineering, vol. 10, no. 1, pp. 38-48, January/February, 2008.
 
 
BibTex
x
 
@article{ 10.1109/MCSE.2008.18,
author = {Alex Szalay and Ani R. Thakar and Jim Gray},
title = {The sqlLoader Data-Loading Pipeline},
journal ={Computing in Science and Engineering},
volume = {10},
number = {1},
issn = {1521-9615},
year = {2008},
pages = {38-48},
doi = {http://doi.ieeecomputersociety.org/10.1109/MCSE.2008.18},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - MGZN
JO - Computing in Science and Engineering
TI - The sqlLoader Data-Loading Pipeline
IS - 1
SN - 1521-9615
SP38
EP48
EPD - 38-48
A1 - Alex Szalay,
A1 - Ani R. Thakar,
A1 - Jim Gray,
PY - 2008
KW - Sloan Digital Sky Survey Science Archive
KW - SDSS
KW - astronomy
KW - large-scale databases
KW - database management systems
VL - 10
JA - Computing in Science and Engineering
ER -
 
Alex Szalay, Johns Hopkins University
Ani R. Thakar, Johns Hopkins University
Jim Gray, Microsoft Research
Using a database management system (DBMS) is essential to ensure the data integrity and reliability of large, multidimensional data sets. However, loading multiterabyte data into a DBMS is a time-consuming and error-prone task that the authors have tried to automate by developing the sqlLoader pipeline—a distributed workflow system for data loading.

1. A. Szalay, "The National Virtual Observatory," Proc. Conf. Astronomical Data Analysis Software and Systems (ADASS) X, vol. 238, F.R. Harnden Jr., F.A. Primini, and H.E. Payne, eds., Astronomical Soc. of the Pacific, 2001., p.3.
2. A. Thakar, S. Szalay, and J. Gray, "From FITS to SQL - Loading and Publishing the SDSS Data," Astronomical Data Analysis Software and Systems (ADASS) XIII, vol. 314, F. Ochsenbein, M.G. Allen, and D. Egret, eds., Astronomical Soc. of the Pacific, 2004., p. 38.

Index Terms:
Sloan Digital Sky Survey Science Archive, SDSS, astronomy, large-scale databases, database management systems
Citation:
Alex Szalay, Ani R. Thakar, Jim Gray, "The sqlLoader Data-Loading Pipeline," Computing in Science and Engineering, vol. 10, no. 1, pp. 38-48, Jan./Feb. 2008, doi:10.1109/MCSE.2008.18
Usage of this product signifies your acceptance of the Terms of Use.