We currently apply partial DAG execution at blocking “shuffle"
operator boundaries where data is exchanged and repartitioned,
since these are typically the most expensive operations in Shark. By
default, Spark materializes the output of each map task in memory
before a shuffle, spilling it to disk as necessary. Later, reduce tasks
fetch this output.