The general framework tackled in this paper is the automatic generation of intelligent collective behaviors using genetic programming and reinforcement learning. We define a behavior-based system relying on automatic design process using artificial evolution to synthesize high level behaviors for autonomous agents. Behavioral strategies are described by tree-based structures, and manipulated by genetic evolving processes. Each strategy is dynamically evaluated during simulation, and is weighted by an adaptation function as a quality factor that reflects its relevance as good solution for the learning task. It is computed using heterogeneous reinforcement techniques associating immediate reinforcements and delayed reinforcements as dynamic progress estimators.