« Back

Building Parallel Programs: SMPs, Clusters, and Java

There Is Parallel Life for Java Scientific Programmers!

by Francisco Esquembre

book cover

This book is an excellent text on parallel programming in Java, especially for computational science students and faculty. If you're short of time, simply reread the previous sentence and you'll get the gist of this review; the rest of the following is devoted to the statement’s subtleties (including its use of italics). As an experienced Java programmer of science and engineering applications who has authored and used many Java scientific simulations for pedagogical purposes (some of them computationally intensive), I've frequently asked myself the following question: "Since I have a quadcore computer, and all of my programs are sequential, do the other three cores get bored while my program runs?!" In other words, wouldn't there be a way to parallelize my Java programs to make better use of even modestly powerful hardware?

In principle, if you create a Java program with several threads and your computer has several processors, the Java virtual machine can distribute the workload by assigning the execution of a different thread to each available processor. Concurrency utilities that can accomplish this have been included in the language since Java 5, and Java 7 takes this paradigm further by introducing a fork-join framework of thread concurrency. However, trying to build a full-fledged scientific application on these principles is simply too difficult; most scientifically oriented programmers need a more straightforward approach.

In search of a solution, I joined a course on parallel programming that happened to be on Fortran or C programming using OpenMP and message-passing interface (MPI), and I also attended the Supercomputing 2010 conference. The main message? "If you want to do scientific parallelism, forget about Java and join us in C." Because Java didn't seem to be the right programming language, it looked like I'd have to revert to good old C. Then came this book!

What It Covers

The author of this book, Alan Kaminsky, is an associate professor in the Department of Computer Science at New York's Rochester Institute of Technology (RIT). A highly experienced programmer, Kaminsky has recently designed, implemented, and tested the Parallel Java (PJ) library, which

  • provides an API and middleware for creating parallel programs in pure Java;
  • works on shared-memory multiprocessor (SMP) parallel computers, clusters of parallel computers, and hybrid SMP clusters of parallel computers; and
  • is freely available in the public domain.

My quadcore desktop is an SMP, for example, and you can build a cluster of parallel computers by assembling multiple desktop PCs and connecting them with a reasonably fast Ethernet network. If the computers in the cluster are multicore themselves, then you’ve built a hybrid cluster.

This book is heavily dependent on the PJ library and uses it to guide the reader step by step into writing parallel programs that exploit SMP, cluster, and hybrid configurations. (Neither the book nor the library address GPU programming.) Kaminsky has used his programming and teaching expertise and the object-oriented design of both Java and his library to create a highly readable and informative book.

The book is divided into five parts that cover

  • the basics that everyone interested in parallel computing needs to know,
  • parallel programming on SMP machines,
  • cluster programming,
  • programming hybrid clusters, and
  • as a final bonus, three complete, fairly involved real-life examples.

These five parts span 35 short chapters, and the book ends with four appendices (the first two of which are must-reads). At the end of most chapters, the author provides references for further information, and each section is followed by a series of exercises that or assign small student projects.

Highs and Lows

This book is very informative, and Kaminsky's writing style is highly readable. Having programmed a library that implements parallelism clearly gives him deep insight into the subject. Kaminsky leads readers through a progressive process of turning sequential programs into effective and efficient parallel programs that address a variety of problems. He does this repeatedly and consistently (thus achieving persistence in students' minds), introducing complexity as needed whenever simpler prototypes prove insufficient.

There are two especially valuable features of his approach:

  • He always includes complete program listings, interspersed with comments as needed.
  • He uses computational examples that hold real scientific interest for students, yet are simple enough to be easily understandable. (In my experience with parallel computing courses, this use of interesting examples is not commonplace!)

Although perhaps a bit biased toward computer science students — due most likely to his own teaching — the examples are also interesting to the scientific community: cryptography, fractal sets, Monte Carlo simulations, graph theory, cellular automata, partial differential equations, and N-body problems. These problems introduce situations that necessitate the use of various features of a parallel programming library, including patterns of parallelism, load balancing, synchronized access to shared variables, types of message-passing communications, and so on. He also covers some of the examples repeatedly in the different parts of the book to show how the same problem can be addressed using SMP, cluster, or hybrid parallelism, which is a very informative approach.

Another highlight is that, early on, Kaminsky introduces the use of metrics (and later, time models for clusters) to measure parallel program speed-up and size-up — that is, how parallelism can make your program run faster on a given problem, or successfully deal with larger problems, respectively.

He then uses these metrics and time models for the various programs to show whether a given implementation of an idea will lead to the expected parallel improvement. This subsequently leads to deeper insight and better solutions.

My biggest criticism of the book stems from the same writing style I praised earlier. As I read the chapters, I frequently found the following reasoning sequence:

  • A given scientific problem is stated.
  • Parallel solutions are proposed and implemented.
  • The solution is tested and shown not to work as expected.
  • The next chapter introduces a new topic whose implementation fixes the problem.

After going through this sequence several times, I sometimes wondered: "What will be the next trick?" Inexperienced students might get the impression that learning parallel programming consists of learning a few tricks. Also, coming to the end of the SMP part, for example, you might feel that there are more tricks that haven’t been covered. To remedy this, an instructor could provide supplemental material from a more theoretical book on parallel computing.

A second possible criticism is of a more technical nature and goes in two directions. The first points toward PJ's C-language relatives, OpenMP and MPI. Kaminski says that the PJ library is inspired by OpenMP and MPI, but he doesn't discuss their parallels (pun included) until the appendices. Because many readers might have previous knowledge of these well-established standards of parallelism, they could benefit from this comparison while studying the PJ approach. The second direction points toward Java itself. I was often curious to learn how the PJ library implements certain utilities (such as barriers or safe access to shared variables). You could delve into the freely available source code, but a couple of in-site explanations would have enriched the text.

This book has quickly become one of the Java programming jewels in my library. It communicates which issues are important for parallel computing, and it does so in a clear and interesting manner. It also demonstrates the elegant and natural way in which PJ matches Java's object-oriented nature.

In fact, I recommend that all serious, scientifically oriented Java programmers and students read this book as soon as possible. Given the already ubiquitous presence of desktop computers with multiple processors — and the growing availability of clusters and supercomputing facilities — this book will no doubt aid these programmers and students in building successful, multifaceted careers.

Francisco (Paco) Esquembre is an associate professor and Dean of the Faculty of Mathematics at the University of Murcia, Spain. Contact him at fem@um.es. This review originally appeared in the Books department of Computing in Science & Engineering (July/Aug. 2011, pp. 6–8).