Load balancing is a key technique in parallel computer supported collaborative work (CSCW) systems, parallel database system and P2P system for instance, to boost performance and improve scalability. In order to reduce total cost of ownership (TCO), adaptive/self-tuning administration techniques are gradually and extensively expected in the cyberspace. In parallel database systems, adaptive load balancing techniques are proposed to face the change in data storage patterns and access patterns in a dynamic real environment. The techniques utilized in both shared-nothing and shared-disk parallel database systems are discussed, and a general flexible framework based on collaborative agents is studied to support these techniques in both architectures. The framework supports two kinds of load balancing - one is passively executing query statements balancedly, and the other one is proactively adjusting data placement and task execution scheme, by means of data and task migration, whenever load unbalance is detected. Three categories of agents, scheduling agents, monitoring agents and task agents, are identified in the framework. The collaboration protocols and scheduling algorithms to support adaptive load balancing are described. The framework also applies to other parallel systems such as P2P systems and shared file processing systems due to their underlying commonness.