Whilst the performance gap between main memory and secondary storage of computer systems increased rapidly during the last years, especially the analysis of secondary storage performance becomes very important. For that purpose this paper introduces a plugin-based framework providing an integrated execution environment for new benchmarks and offering analysis tools for the complex result output of particularly I/O measurements. As an example of this environment we describe the PRIOmark, an I/O benchmark that characterizes file system performance of single and distributed systems by means of two different file system interfaces. Additionally, the I/O performance of three file systems, two file system interfaces as well as different I/O workloads are compared to show the flexibility of the presented PRIOmark.