Evolvable Hardware (EHW) has been proposed as a new technique to design complex systems. Often, complex systems turn out to be very difficult to evolve. The problem is that a general strategy is too difficult for the evolution process to discover directly. This paper proposes a new approach that performs incremental evolution in two directions: from complex system to sub-systems and from subsystems back to complex system. In this approach, incremental evolution gradually decomposes a complex problem into some sub-tasks. In a second step, we gradually make the tasks more challenging and general. Our approach automatically discovers the sub-tasks, their sequence as well as circuit layout dimensions. Our method is tested in a digital circuit domain and compared to direct evolution. We show that our bidirectional incremental approach can handle more complex, harder tasks and evolve them more effectively, then direct evolution.