Open source J2EE Application Servers (J2EE ASs) have already attracted industrial attentions in recent years, but the reliability is still an obstacle to their wide acceptance. The detail of software failures in J2EE AS was seldom discussed in the past, although such information is valuable to evaluate and improve its reliability. In this paper, we present a measurement- based failure classification and analysis. Presented results indicate open source J2EE AS's reliability needs improvement. Some notable results include: (1) the implementation of a widely used fault-tolerant mechanism, clustering, needs improvement; (2) only 15% of reported failures are removed within a week, and about 10% still open by the time we finish the study; (3) different J2EE ASs have different unreliable services so reliability improvement activities should be specific to both AS and applications.