GridBench: A tool for Grid Benchmarking

Overview

The emergence of Grids as platforms of high performance and high throughput computing over the Internet raises many questions regarding the performance characterization of Grid infrastructures. Benchmarking metrics published on the Grid can provide a basis for users to assess the "quality of service" expected by a Virtual Organization providing computational services at a given cost. Grid benchmarking can also guide the selection of resources allocated to computationally intensive applications.

GridBench is a tool for evaluating the performance of Grids and Grid resources through benchmarking. It facilitates the easy definition of parameterized execution of benchmarks on the Grid, while at the same time allowing for archival and retrieval of results and the creation of customized charts from these results.

| Flexibility | Functionality | Easy of Use |

GridBench comprises a framework of tools and a suite of benchmarks. The tools provide a user-friendly graphical interface for defining, executing and administrating benchmarks upon the resources of a Virtual Organization (VO), as well as for browsing results. Additionally, it provides tools for archiving and analysing results through the easy construction of custom graphs.

Leverages new and existing benchmarks from the HPC community. New Benchmarks complement the tried and respected HPC benchmarks, that have been adapted to run the Grid.

| Features |

Benchmarks derived from CrossGrid and other Grid applications
MPI micro-benchmarks
Data-Transfer benchmarks
High-Performance Linpack
CPU micro-benchmarks
Memory micro-benchmarks
Local Disk-IO benchmarks

GridBench tool has been developed as part of the "Metrics and Benchmarks" task (Task 2.3) assigned to Work-Package 2 of the CrossGrid project funded by the European Union. CrossGrid aims to develop new Grid services and tools for interactive compute- and data-intensive applications like the biomedical simulation and visualization for vascular surgical procedures, the flooding-crisis team decision support system, distributed data analysis in high-energy physics, and air pollution combined with weather forecasting. As task 2.3 aimed to propose a set of performance metrics to describe concisely the performance capacity of the Grid configurations and application performance, and to develop and implement benchmarks that are representative of typical Grid workloads, GridBench belongs to the "supporting tools" layer of the CrossGrid project.

HPCL of University of Cyprus had fully undertaken this task.

Check out the new Gridbench Demo, the GridBench Tutorial and the GridBench brochure!

Technical Documentation

CrossGrid Developer Manual - GridBench, version 1.0a (in pdf), 1/2005
CrossGrid Installation Guide- GridBench , version 0.9 (in pdf), 12/2004.
CrossGrid User Manual - GridBench, version 1.0 (in pdf), 11/2004.
Prototype Documentation for GRIDBENCH, version 1.0 (in pdf), 2/2003.
Design Document for GRIDBENCH, version 1.0 (in pdf), 8/2002.
Software Requirements Specification, version 1.1 for GRIDBENCH (in pdf), 8/2002.
Software Requirements Specification, version 1.0 for GRIDBENCH (in pdf), 3/2002.

Publications

M. Dikaiakos, "Grid Benchmarking: Vision, Challenges, and Current Status." Concurrency and Computation: Practice and Experience, Volume 19, Issue 1, pages 89-105, Wiley, 2007 (available in pdf)
G. Tsouloupas, M. Dikaiakos, "Characterization of Computational Grid Resources Using Low-level Benchmarks." In e-Science 06, Proceedings of the 2nd IEEE International Conference on e-Science and Grid Computing, 8 pages, IEEE Computer Society (available through IEEE Xplore digital library and locally in pdf).
G. Tsouloupas, M. Dikaiakos, "Ranking and Performance Exploration of Grid Infrastructures: An Interactive Approach." Proceedings of the 7th IEEE/ACM International Conference on Grid Computing, pages 313-315, Barcelona, September 28-29, 2006, IEEE Computer Society.
M. Dikaiakos, "Grid Benchmarking: Vision, Challenges and Current Status." Technical Report TR-2005-11, Department of Computer Science, University of Cyprus, May 2005 (available in postscript).
A. Tiramo-Ramos, G. Tsouloupas, M. D. Dikaiakos, P. Sloot. "Grid Resource Selection by Application Benchmarking: a Computational Haemodynamics Case Study." Computational Science - ICCS 2005, 5th International Conference, Atlanta, GA, USA, May 22-25, 2005, Proceedings, Part I. Lecture Notes in Computer Science, vol. 3514, pages 534-543, Springer, 2005 (available in pdf through Springer's digital library).
Eamonn Kenny, Brian A. Coghlan, George Tsouloupas, Marios D. Dikaiakos, John Walsh, Stephen Childs, David O'Callaghan, Geoff Quigley: Heterogeneous Grid Computing: Issues and Early Benchmarks. Computational Science - ICCS 2005, 5th International Conference, Atlanta, GA, USA, May 22-25, 2005, Proceedings, Part III. Lecture Notes in Computer Science, vol. 3516, pages 870-874, Springer, May 2005 (available in pdf through Springer's digital library).
G. Tsouloupas, M. Dikaiakos, "Design and Implementation of GridBench", Advances in Grid Computing - EGC 2005. European Grid Conference, Amsterdam, The Netherlands, February 14-16, 2005, Revised Selected Papers, Lecture Notes in Computer Science, vol. 3470, pages 211-225, Springer, June 2005 (available in pdf).
G. Tsouloupas, M. Dikaiakos, "Characterization of Computational Grid Resources Using Low-level Measurements." Technical Report TR-2004-05, Department of Computer Science, University of Cyprus, October 2004 (available in pdf).
G. Tsouloupas, M. Dikaiakos, "GridBench: A Tool for Benchmarking Grids." In Proceedings of the 4th International Workshop on Grid Computing (Grid2003), pages 60-67, Phoenix, Arizona, 17 November 2003, IEEE Computer Society (available in pdf ).

Presentations

"Design and Implementation of GridBench", by George Tsouloupas, European Grid Conference, Amsterdam, February 14th, 2005. (available in ppt)
"Test driving the Grid," by M. D. Dikaiakos, Colloquium, Department of Computer Science, Rutgers University, Piscataway, NJ, USA, May 20, 2005.
"Test driving the Grid," by M. D. Dikaiakos, Institute of Computer Science, Foundation of Research and Technology Hellas (FORTH), Herakleion, Greece, October 22, 2004. (available in pdf)
"GridBench: A Tool for Benchmarking Grid Resources", by M. D. Dikaiakos, Grid Performance Workshop, University College London, London, UK, May 13, 2004. (available in ppt)
"GridBench: A Tool for Benchmarking Grids", by George Tsouloupas, 4th International Workshop on Grid Computing (Grid2003), Phoenix, Arizona, 17 November, 2003. (available in ppt)
"Grid Benchmarking", by Pedro Trancoso, APART-2 workshop on Grid Monitoring and Performance Analysis,Klagenfurt University, 16 August, 2003. (available in ppt)

Software

The GridBench software is available for distribution under the Open Source License of CrossGrid (also available in pdf). Please contact HPCL if you are interested to download GridBench.