In an integration system of RFID and wireless sensor network (WSN), RFID is used to identify objects while WSN can provide context environment information of these objects. Thus, it increases system intelligent in pervasive computing. We propose the EPC Sensor Network (ESN) architecture as an integration system of RFID and WSN. This ESN architecture is based on EPCglobal architecture, the de facto international standard for RFID. The core of ESN is the middleware part which is also implemented in our work. In this paper, complex event processing (CEP) technology is used in our ESN middleware which can handle large volume of events from distributed RFID and sensor readers in real time. Through filtering, grouping, aggregating and constructing complex event, ESN middleware provides a more meaningful report for the clients and increases system automation.