loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
2004 IEEE International Conference on Computer Design (ICCD'04)
An Architecture for Fast Processing of Large Unstructured Data Sets
San Jose, CA
October 11-October 13
ISBN: 0-7695-2231-9
Mark Franklin, Washington University in St. Louis
Roger Chamberlain, Washington University in St. Louis; Data Search Systems, Inc., St. Louis, MO
Michael Henrichs, Data Search Systems, Inc., St. Louis, MO
Berkley Shands, Washington University in St. Louis
Jason White, Data Search Systems, Inc., St. Louis, MO
This paper presents a general system architecture tailored to performing searching, filtering, compression, encryption, and other operations on unstructured data streaming from a disk system. The system achieves high performance on such applications by providing for parallelism, hardware-application specialization and reconfiguration, and hardware placement near the disk systems. A limited prototype of a single compute node has been implemented and is described. The prototype is tailored to applications involving complex searching and its performance is compared to a pure software implementation having the same search capabilities. Performance is considered in terms of data set size, query string hit rate and query complexity. Performance results as a function of these parameters are presented and the results indicate that, for data set sizes above 1.4 MB, the prototype compute node is between one and two orders of magnitude faster than a pure software implementation. At high data set sizes, on an individual node, speedups of about 200 and a sustained throughput of 300 MB/sec have been achieved.
Citation:
Mark Franklin, Roger Chamberlain, Michael Henrichs, Berkley Shands, Jason White, "An Architecture for Fast Processing of Large Unstructured Data Sets," iccd, pp.280-287, 2004 IEEE International Conference on Computer Design (ICCD'04), 2004
Usage of this product signifies your acceptance of the Terms of Use.