The Web Services framework, since it leverages the Service Oriented Architecture model, enables the development of applications that are loosely coupled and easier to manage. A Web Service is typically hosted within a Web Service Container. There are several choices for these containers depending on the platform and the language in which the applications would be developed. In this paper our focus is on applications built using the Java platform. Here, the most dominant Web Service container is the opensource Apache Axis Web Service container. In this paper we describe our experiences in deploying Web Services, specifically WS-ReliableMessaging, within this container. We enumerate the problems and limitations that we encountered within Axis, and our solutions to get around this problem. We also have a set of recommendations that would make this a more flexible container for sophisticated Web Service applications.