In this paper, we address an issue of design in online rating systems: how many items should be elicited from the ratings provider. Recommender and reputation systems have traditionally relied on single-dimension ratings to reduce user burden, but for some types of information this amount of feedback may be insufficient. We presented users of an online news rating service with different numbers of items in a news rating exercise. We find that users show the highest satisfaction and greatest rating accuracy with a multi-item reviewing instrument.