The Community for Technology Leaders

Guest Editors' Introduction: High-Performance Computing

Charles J. , US Department of Defense
Robert E. Peterkin , US Department of Defense

Pages: pp. 8-11

The March/April 2002 issue of Computing in Science & Engineering featured the use of high-performance computing (HPC) in the US Department of Defense's (DoD's) R&D and test and evaluation communities. It is the mission of the DoD's High-Performance Computing Modernization Program (HPCMP) to create advanced computational environments to help scientists and engineers in these communities facilitate the rapid application of advanced technology into superior warfighting capabilities. Advanced technology is the key. As US Secretary of Defense Donald H. Rumsfeld stated to the US Senate and House Armed Services Committees in February 2004, "we have learned in the global war on terror…what is critical to success in military conflict is not necessarily mass as much as it is capability." In this issue of CiSE, we again take the opportunity to report on a subset of the wide range of activities in the relevant DoD communities for which HPC is enabling scientific discovery and creating opportunities to impact the war on terror.


Why is HPC so important to scientific and engineering enterprises? Some consider scientific computing to be the third element in a triad that includes theory and experimentation. Proper science is done according to the scientific method, the goal of which is to arrive at a theory that describes the behavior of some aspect of the observable universe. Simple theories are presumed better than complicated ones, but simple theories can describe both simple and complicated behaviors. Consider, for example, that the same theory of classical mechanics governs both the Earth's weather and the behavior of a baseball dropped from rest a certain distance above the floor. We can predict the speed of the baseball before it hits the floor to a pretty high level of accuracy with just a pencil and a small piece of paper. A similar approach to predicting this weekend's weather usually isn't very accurate: weather behavior is complex, and to increase the accuracy of weather prediction, we must do a lot more arithmetic.

For complex problems, our answer's accuracy often improves as we do more arithmetic. This is one reason why there usually isn't a good answer to the question, "how big of a computer do you need?" The answer is all too often, "how big of a computer can you give me?" HPC is all about using the best available computers to solve the most complex problems that can be performed on those computers.

To help fulfill our vision to provide a pervasive culture among DoD scientists and engineers in which they can routinely use advanced computational environments to solve the most demanding problems, the HPCMP began in 1992 to fund the creation, recapitalization, and maintenance of large-scale computing centers, scientific software development activities, secure networks linking researchers and users, and environments that foster innovation, discovery, and application to defense. The HPCMP community includes scientists and engineers from government labs, test and evaluation centers, academia, and industrial partners. Program director Cray J. Henry details the elements of the HPCMP in the sidebar.


The HPCMP fosters collaborations between the US Armed Services and Defense agencies to advance the state of the art in using HPC to address today's most pressing defense problems. Such problems include the design of advanced materials, the implementation of remote sensor networks, the improvement of sophisticated machinery, and the development of both conventional and unconventional weapons. Some of the best HPC approaches to addressing these problems are examined in this special issue.

Jerry Bernholc, Serge M. Nakhmanson, Marco Buongiorno Nardelli, and Vincent Meunier describe recent advances in theoretical methods and HPC for first-principles investigations that move us toward the ability to design new materials with useful properties. Specifically, in their article "Understanding and Enhancing Polarization in Complex Materials," they describe computational methods for designing ferroelectric materials that have pyroelectric and piezoelectric properties. Materials with such properties might be useful for new classes of useful sensors.

Thomas S. Anderson, Mark L. Moran, Stephen A. Ketcham, and James Lacombe use a variable grid finite-difference time-domain method to model ground motion from moving vehicles. Their goal, as described in "Tracked Vehicle Simulations and Complete Seismic Wavefield Synthesis in Support of Seismic Sensor Systems Development," is to help build identification algorithms for the development of effective ground sensor systems.

The role of turbulence in both external and internal flows is extraordinarily important to many technologies of interest to the DoD, including the flow around airplanes and inside certain machinery. Advances in algorithms and HPC have made the numerical simulation of highly separated flows feasible, but the value of simulated results depends on the appropriateness of the numerical algorithm and boundary conditions as well as the quality of the numerical grid. The article "A Grid Convergence Study of a Highly Separated Turbulent Flow" by Robert Hansen and James Forsyth investigates the effect of grid density and connectivity on the quality of solution for the relatively simple test problem of flow over a cylinder.

The article, "Study of Tip-Clearance Flow in Turbomachines Using Large-Eddy Simulation," by Donghyun You, Meng Wang, Rajat Mittal, and Parviz Moin discusses the use of HPC techniques to investigate ways to mitigate the undesirable effects caused by unsteady complex turbulent flows in hydraulic turbomachines. The authors use the technique known as large-eddy simulation to gain physical insights about the complex vortex structures, dynamics, and associated low-pressure events that lead to cavitation.

Finally, Timothy J. Madden and James H. Miller describe HPC models of chemical lasers in their article "Simulation of Unsteadiness in Chemical Oxygen-Iodine Laser Flowfields." The issues faced by these researchers are similar to those faced by the authors of the previous two articles in that they require accurate simulation of unsteady flows, but Madden and Miller face the additional challenge of reaction chemistry accompanying the complex flow in the laser-generating flowfield.


The DoD has an expanding list of complex scientific and engineering problems to solve, and the use of HPC is becoming increasingly common. The computational power of the most advanced computers has steadily increased over time for close to 60 years now, but our ability to effectively use ever-increasing computational power based on the most popular current model of building parallel computers might be tapering off.

To address this issue, the DoD has joined forces with other US government departments and agencies that heavily use HPC—including NASA, the US Department of Energy, and the US National Security Agency—in two arenas. The first is the High-End Computing Revitalization Task Force (HECRTF;, which is led by the White House Office of Science and Technology Policy. In May 2004, it published a high-level plan for the nation to meet the high-end computing requirements of the federal science, engineering, and national security communities. This plan includes resources to increase accessibility to both production and large-scale high-end computing systems and R&D to build future generations of high-end computing systems and software technologies. The second is the DARPA High-Productivity Computing Systems Program (HPCS; This program is conducting research and development into next-generation supercomputer systems, with the goal of building economically viable high-productivity computing systems for the national security and industrial user communities. It is focused on providing dramatic improvements in performance, programmability, application software portability, and system robustness.

The HPCMP is substantially involved with these and related strategic HPC activities. Thus, we're optimistic that our ability to move technology from concept to reality in support of national defense goals should continue to advance at an ever-faster pace as our ability to compute complexity continues to increase.

The High-Performance Computing Modernization Program

Cray J. HenryHPCMP program director

High-performance computing has been—and continues to be—a key enabling technology for R&D design and test of new and improved systems. US national military strategy depends on the ability of our nation's science and engineering community to continue to research, design, develop, and produce technologically superior ideas, components, and systems. The use of science-based modeling and simulation is accelerating the pace of innovation, allowing solutions to previously unsolvable problems. Advanced parallel software and the fastest supercomputers provide the competitive edge on which we depend.

The DoD High-Performance Computing Modernization Program (HPCMP) is a major defense acquisition program that exists to enable more than 4,400 scientists and engineers—at more than 170 DoD laboratories, universities, test centers, and industrial sites—to seek innovative solutions on more than 650 active defense programs. The scope of efforts includes

  • designing new materials (fuels, coatings, composites),
  • developing effective sensor systems (sensor simulations, communications simulations, environmental simulations),
  • simulating aircraft and ship performance (analyzing complex vortex structures and hull performance, studying cavitation),
  • modeling explosions and structural responses (design and test of thermobaric weapons, Pentagon retrofit studies, armor and anti-armor systems),
  • developing directed-energy weapons (modeling the complex flow and chemistry in the laser-generated flowfields, molding optical atmospheric turbulence), and
  • weather prediction (ocean modeling, sandstorm prediction).

The HPCMP also operates approximately 20 supercomputing centers across the country, provides wide-area network services for defense laboratories and test centers via the defense research and engineering network (DREN), and supports computational science-based application development at defense laboratories, test centers, and with academic and industrial partners.

The four largest supercomputing centers, referred to as major shared-resource centers (MSRCs), provide multiple large supercomputing systems (distributed memory, shared memory, and vector), robust mass storage and archival systems, extensive scientific analysis and visualization systems, and expert staffs to support the full community. Of the 16 distributed centers, four are allocated distributed centers (which provide specialized support to specific functional communities), and 12 are dedicated distributed centers (which provide dedicated support for specific projects). The number of dedicated distributed centers changes from year to year, based on the DoD's evolving needs.

DREN provides Internet Protocol and asynchronous transfer mode transport services at current bandwidths from DS-3 (45 Mbits per second) to OC-48 (48 times as fast as DS-3), with future capabilities up to OC-768 (16 times again as fast, or 35 Gbit/s). DREN protects information with a security-in-depth model that includes access filters, strong authentication, and active monitoring, while delivering network services such as IPv6 and multicast. DREN also provides connectivity across the science and technology and test and evaluation communities, enabling integrated distributed test events, specialized research testbeds, and remote access to supercomputers.

The development of software applications that can effectively use dozens to thousands of parallel processors is difficult. The performance potential of today's supercomputers cannot be achieved without specifically designed software that combines the skills of both science domain experts and computer science experts. The HPCMP

  • sponsors HPC software application institutes that focus on the design and maintenance of defense-specific applications,
  • manages the development of combined science software applications through annually selected software portfolios,
  • provides expert technical computational science services through partnerships with researchers at dozens of universities,
  • oversees research into innovative software protection approaches, and
  • supports graduate fellowships and undergraduate summer internship programs.

The DoD, through the HPCMP, maintains a world-class supercomputing capability that when applied to the broad range of research, development, and test problems, will help ensure our military advantage on the 21st century battlefield.

About the Authors

Charles J. Holland is the Deputy Under Secretary of Defense for Science and Technology. His research interests include HPC and embedded systems. He has a BS and an MS from the Georgia Institute of Technology and a PhD from Brown University, all in applied mathematics. He's also a member of CiSE's editorial board. Contact him at
Robert E. Peterkin Jr. is the Chief Scientist of the DoD High-Performance Computing Modernization Program. He has a BS in physics from Boston College and a PhD in physics and astronomy from the University of North Carolina at Chapel Hill, where he studied gravitation and elementary particle physics and wrote his dissertation in quantum gravity. His present research interests include HPC and plasma physics. He is a fellow of the Air Force Research Laboratory. Contact him at
63 ms
(Ver 3.x)