Volunteer Computing (VC) has been proving to be a way to
access large amounts of computational power, network bandwidth
and storage. With the recent developments of new programming
paradigms and their adaptation to run on the large
scale Internet, we believe that data distribution techniques
need to be re-thought in order to cope with the high volumes
of information handled by, for example, MapReduce.
Thus, we present a VC solution called freeCycles, that supports
MapReduce jobs. freeCycles presents two new contributions:
i) improves data distribution (among mappers and
reducers) by using the BitTorrent protocol to distribute (input,
intermediate and output) data, ii) improves intermediate
data availability by replicating it through volunteers in order
to avoid losing intermediate data and consequently preventing
big delays on the MapReduce execution time.
Volunteer Computing (VC) has been proving to be a way to
access large amounts of computational power, network bandwidth
and storage. With the recent developments of new programming
paradigms and their adaptation to run on the large
scale Internet, we believe that data distribution techniques
need to be re-thought in order to cope with the high volumes
of information handled by, for example, MapReduce.
Thus, we present a VC solution called freeCycles, that supports
MapReduce jobs. freeCycles presents two new contributions:
i) improves data distribution (among mappers and
reducers) by using the BitTorrent protocol to distribute (input,
intermediate and output) data, ii) improves intermediate
data availability by replicating it through volunteers in order
to avoid losing intermediate data and consequently preventing
big delays on the MapReduce execution time.
การแปล กรุณารอสักครู่..
