In this paper, we propose a simple and efficient mechanism to self-configure semi-structured overlay networks for heterogeneous peer-to-peer systems with no global knowledge. The overlay network is constructed as a hierarchical graph based on peer power which is an aggregate measure of available resources. We use index based, random walk JOIN and LEAVE protocols to build the overlay network in which node degree and workload correspond to its peer power. We also propose an index based query scheme for peers to search documents in the network. Performance studies demonstrate that the proposed self-configuration protocol and search scheme are efficient and viable. The self-configured P2P network has significantly lower query cost than traditional Gnutella-like P2P systems. We further propose a caching assisted query mechanism where query results are cached by intermediate nodes along the search path, thereby exploiting the surplus storage power of the peers and further improving query efficiency. Caching assisted query permits a quicker response to subsequent queries for popular data without incurring the excessive overhead of globally replicated directory information. Our experimental study reveals that the caching assisted query provides an additional reduction of up to 40% in messages needed per query.