[Technology Report]
Parallel Programming Is Here To Stay
One size does not fit all, and it never will. Parallel programming looks to level the playing field by leveraging multicore hardware.
In scatter-gather, a typical parallel-programming pattern, data is distributed for processing. Then the results are gathered together, often with additional processing, to combine the results. This dataflow control can be a challenge for conventional control flow languages, but it’s second nature for National Instruments’ LabView.
The LabView graphical programming language also is a dataflow language with which programmers specify how data moves through the system (here, sequencing is a secondary issue). Not to say that sequential programming isn’t part of LabView. In fact, loop and conditional constructs will be part of any LabView program.
Many designers will be interested in how LabView works under the hood on a conventional processor. In the simplest case, pending operations are placed in a job queue. A thread reads the queue and performs the operation, potentially posting new jobs in the queue.
This scenario is the same used by Intel’s TBB. As with TBB, there may be multiple worker threads. The number of worker threads tends to match the number of cores. Fewer of them will avoid the hardware. More tend to result in idle threads.
Asynchronous I/O doesn’t delay the working threads. Instead, an entry is added to the queue when a background operation is complete.
In theory, job distribution and processing can be handled by a large number of cores, potentially using other hardware like GPUs. National Instruments is researching these areas now—dataflow semantics allow LabView to target more than conventional single-core and SMP platforms.
FPGA application design is naturally parallel. It also works well with graphical design tools, so it’s no surprise that Lab- View applications target FPGAs. LabView applications can be split across FPGAs and computing platforms.
Graphical dataflow languages like Lab- View aren’t common, though a few are available, such as the Mathworks’ Simulink and Microsoft’s Visual Programming Language (VPL).
ACTING PARALLEL The dataflow approach can be seen as a message-passing model. Implementations like LabView operate with a fine-grain resolution. Move to a coarser level of control, and the actor model emerges. Actors are objects that receive and send messages. They tend to be components rather than complete applications.
Ambric’s Am2045 Massively Parallel Processing Array 336-core chip is programmed using Java. Restrictions exist, primarily on size, because of the memory resources available within a core. Essentially, Ambric implements a messagebased actor model. Each core executes an active object/actor with messages being sent and received using a straightforward channel interface.
Actors and parallel programming are old friends. Programming languages like Erlang have been used to implement robust distributed applications. Erlang was originally designed by Ericsson with an eye toward fault tolerance.
Scala is a newer programming language that addresses the actor model (see “If Your Programming Language Doesn’t Work, Give Scala A Try”). It originally was designed to run atop a Java virtual machine. Scala also implements the functional programming model.
FUNCTIONALLY PARALLEL Functional programming is a bit more than just calling functions. It’s a programming model that avoids state and mutable data. This turns out to be good for parallel execution, but is at odds with most conventional programming languages where variables are designed to be changed at will.
Languages that incorporate functional programming aspects, such as Scala, are considered impure functional languages because variable values can change. Pure functional languages can’t modify the contents of a variable. At this point, functional programming is often more of an academic than commercial concern when it comes to implementation.
One advantage of a pure functional programming language is its referential transparency. Calling any function with a set of parameter values will always generate the same result. This is true for many of the functions implemented in conventional languages where it’s possible to use a functional programming style. However, that’s the case only if they don’t retain any state or access information outside of the function that can change.
Reprints
