Geometric Constraint Problem can be transformed to an optimization problem which the objective function and constraints are non-convex functions. In this paper an evolutionary algorithm based on Ant Colony Optimization algorithm and the immune system model is proposed to provide solution to the Geometric Constraints Problem. In the new algorithm, affinity calculation process and pheromone trail lying is embedded to maintain diversity and carry out the global search and the local search in many directions rather than one direction around the same individual simultaneously. This new algorithm different with current optimization methods in that it gets the good solution by excluding bad solutions. The experimental results reported here will shed more light into how affects the hybrid algorithm's search power in solving Geometric constraint problem.