The use of Java Message Service (JMS) for enterprise applications communication and integration is increasing very quickly. However, although JMS is frequently used in business-critical environments, applications are typically developed with the assumption that the middleware being used is robust, which is not always the case. Robustness failures in such environments are particularly dangerous, as they may originate vulnerabilities that can be maliciously exploited with severe consequences for the systems subject of attack. This paper proposes an approach for the evaluation of the robustness of JMS middleware. Our approach is presented through a concrete example of evaluating the robustness of three well-known JMS solutions (JBoss MQ 3.2.8.SP1, JBoss MQ 4.2.1.GA, and Active MQ 4.1.1), in which several robustness and critical security related problems have been disclosed (including specification conformance disparities).