Type

Journal Article

Authors

Ge Yu
Claus Pahl
Li Zhang
Shu Xu
Jie Song

Subjects

Computer Science

Topics
big data mapreduce energy consumption self optimization algorithms terasort software engineering task resizing

Performance and energy optimization on terasort algorithm by task self-resizing (2014)

Abstract In applications of MapReduce, Terasort is one of the most successful ones, which has helped Hadoop to win the Sort Benchmark three times. While Terasort is known for its sorting speed on big data, its performance and energy consumption still can be optimized. We have analyzed the characteristics of Terasort and have identified the existence of idle notes, which does not only waste energy but also loses performance. Therefore, we optimize Terasort through a single-task distributed algorithm and a task self-resizing algorithm to save time and reduce the energy that is consumed by map nodes, which is caused by waiting for tasks and reduce nodes waiting for input. The algorithm proposed in this paper has proved to be effective in optimizing performance and energy consumption through a series of experiments. It can also be adapted to other applications in the MapReduce environment.
Collections Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing: School of Computing
Ireland -> Dublin City University -> Publication Type = Article
Ireland -> Dublin City University -> Subject = Computer Science
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools
Ireland -> Dublin City University -> Status = Published
Ireland -> Dublin City University -> Subject = Computer Science: Algorithms
Ireland -> Dublin City University -> Subject = Computer Science: Software engineering
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing

Full list of authors on original publication

Ge Yu, Claus Pahl, Li Zhang, Shu Xu, Jie Song

Experts in our system

1
Claus Pahl
Dublin City University
Total Publications: 191
 
2
Li Zhang
Dublin City University
Total Publications: 10