Pervasive computing is to enhance the environment by embedding many computers that are gracefully integrated with human users. To achieve this, the key research thrust is to create a smart context-awareness environment which should enclose various users and satisfy different needs of the users. Building such smart environments is still difficult and complex due to lacking a uniform infrastructure that can adapt to diverse smart domains. To address this problem, we propose a context-aware computing infrastructure, called FollowMe. Our infrastructure integrates an ontology based context model and a workflow based application model with the OSGi framework. By plugging different domain contexts and applications, FollowMe can be customized to various domains.