An important factor that needs to be taken into account by end-users and systems (schedulers, resource brokers, policy brokers) when mapping applications to the Grid is the performance capacity of hardware resources attached to the Grid and made available through its Virtual Organizations. In this article, we examine the problem of characterizing the performance capacity of Grid resources using benchmarking. We examine the conditions under which such characterization experiments can be implemented in a Grid setting and present the challenges that arise in the Grid context. We specify a small number of performance metrics and propose a suite of microbenchmarks to estimate these metrics for sites that belong to large Virtual Organizations. We describe benchmarking experiments conducted with, and published through GridBench, a tool that we built to manage benchmarking experiments over the Grid and to publish and analyze performance metrics. Finally we show how results derived from GridBench can help end-users assess the performance capacity of resources belonging to a large Virtual Organization.