3.3 Functional Parallel Algorithms
A growing trend in program development is the use of functional programming,
which emphasizes a mathematical formulation of a problem in terms of functions
and data structures as abstract notions, rather than in terms of their concrete
realization in memory.
One important application of functional programming is to parallelism. The
advantage is the declarative" nature of functional programs, which avoid over-
specifying the details of how data is represented or manipulated, in favor of a
more equational description of what the computation is to achieve. Functional
languages provide a useful example of an abstract cost measure, the cost graph,
which associates to a computation a representation of the dependencies among
subcomputations, exposing the implicit parallelism available for that program.
The cost graph determines both the sequential and parallel time and space
complexity of the program, and is a useful abstraction with which to analyze
the asymptotic complexity of a program without regard to platform-specic
parameters, which can be factored in later.
By stressing the tight connection with mathematics, functional languages
encourage a heoretical" mindset that is consonant with the aims of both the
algorithms and language communities. Behavioral verication of functional pro-
grams is far easier than for their imperative counterparts; it is interesting to ask
whether the same might be said for verication of their eciency. Certainly
the correctness arguments, on which any complexity analysis depends, are far
simpler, particularly in the parallel case, so this can only help.
The functional perspective has also proved eective in a distributed setting
in which there is no useful concept of shared state. The well-known Hadoop
and Map-Reduce systems exploit this aspect of parallelism to perform parallel
7
computations over large-scale distributed data sets. Such algorithms are highly
sensitive to issues of data locality, because of the enormous dierence in time to
access
emote" from local" data. This kind of problem was studied recently
by Blelloch and Harper [2013, 2014] from the perspective of a cost semantics,
showing that one can reason about the I/O complexity of an algorithm without
having to drop down to a low-level machine model.