Emergent wide-area distributed systems like computational grids present opportunities for large scientific applications. On these systems, communication mechanisms have to deal with dynamic resource availability and occurrence of network failures. In this paper, we present the design and implementation of an asynchronous and persistent pipe mechanism, called π-channels. These communication issues are addressed by combining adaptive caching with data streaming for efficient and fault-tolerant communication. We present the underlying distributed algorithm that implements (a) caching of pipe data segments; (b) asynchronous operation; and (c) re-establishment of connections when a peer leaves and rejoins the computation - part of a communication state transfer mechanism. This makes it possible for different segments (from cache and from writer) of the pipe data to be concurrently streamed to the migrated reader, reducing the retrieval time. Finally, we present some performance results showing the benefits of asynchronous operation.