Gradient based approaches for motion estimation (Optical-Flow) estimate the motion of an image sequence based on local changes in the image intensities. In order to best evaluate local changes in the intensities, specific filters are applied to the image sequence. These filters are typically composed of spatio-temporal derivatives. The design of these filters plays an important role in the estimation accuracy. This paper proposes a method for the design of these filters in an optimal manner. Unlike previous approaches that design optimal derivative filters in some sense, the proposed technique defines the optimality directly with respect to the motion estimation goal; The suggested approach takes into account prior knowledge on the motion distribution, the image characteristics, and the allocated filter length. Simulations demonstrate the advantage of the new design approach.