Multiprocessor Support: There are two classes of software support for MP environments—shared memory and message passing. In shared-memory systems, the application can execute on multiple processors. Every processor is able to directly access the same data, too. In the case of multiple executable images, system primitives are attached to the same data. A single executable also can start multiple threads, each running on a separate processor. The threads all access the same data without special system calls.
These shared-memory systems often provide compiler support to generate parallel executables with relatively minor enhancements to a uniprocessor application. The enhancements are in the form of compiler directives. They help the compiler identify sections of the application which can be executed in parallel on the available processors. Like automatic vectorization, this technology is well understood and mature. The performance achieved depends on the architecture and application. Additionally, there's better support for Fortran applications than for other languages.
If shared memory isn't available, message passing must be used to exchange and coordinate multiple executables. It also provides better performance for a number of applications on shared-memory architectures. Message passing has been available as long as there have been networks. But in practice, the software support isn't as mature as that provided by the parallel compilers.
A number of libraries have been invented over the last 10 years or so to simplify building processor applications. Parallel virtual machine (PVM), message-passage interface (MPI), and, more recently, MPI/RT are all attempts to produce standards that are available on multiple platforms. Also, vendors have produced proprietary libraries that provide optimal performance for their systems. These libraries, though, aren't portable. MPI and MPI/RT should be portable, but they often don't provide the desired performance. Using the libraries is generally likened to employing "assembly language for communications." The user must program the communications in much more detail than is necessary.
Communications Management: A few libraries are trying to simplify the communications problems. They typically layer on top of packages like MPI. Other packages, such as SPE, KHOROS, GEDAE, and RTExpress, provide a way to describe pipelined data flow.
An application is built by creating executables that take their inputs and outputs through the library. These executables are then configured to run on a number of processors forming a parallel module. The output of one module is fed to the input of the next. A reorganization of the data provided by the library partitions the data as needed, and possibly performs a transpose operation as well (see the figure).
Along with organizing the data flow, these libraries simplify configuration of a potentially large number of processors into processing modules. To start the application, the communications paths must be defined, buffers must be allocated, and a number of other bookkeeping chores must be managed.
In MPI, these chores involve a large number of calls. When managed by a data-flow library, the call total drops to just a few. Another benefit is that hardware failures can be managed more easily. Available processors are allocated to the various modules according to a simple algorithm. If a processor fails, rather than requiring a manual reconfiguration, the failed processor is simply removed from the available set. The remaining processors are then assigned according to the algorithm.
Automatic Communications: Some vital tools are starting to appear on the market. They can take a uniprocessor application and convert it to run on a distributed-memory architecture using a communications library like MPI. These tools are being developed by universities and national research labs with access to very large, high-performance computer systems. The scientists performing simulations on these systems face the same issues as the programmers creating radar applications. They want to work on their algorithm, not on the communications problem.
The state of the art here isn't very mature yet. These tools have had some limited success. But it still takes some effort to tune the application by inserting directives into the code to coerce the tool into producing an efficient application. When the tool succeeds, the effort expended is significantly less than what would have been required if the programmer had to add the communications calls manually. Unfortunately, for the present there are many applications in which a conversion tool doesn't help.