Curve detection is one of the fundamental steps in computer vision applications. Conventional edge detectors provide only an output of edge pixels; curve matching is then needed to fit edge pixels into curves. Despite having achieved some success, it suffers constraints for applications that require real-time and robust image analysis, such as robot vision and video surveillance. Gao and Wang developed a curve tracking algorithm for detecting perceptually sound curves by choosing initial edge pixel first and then do perceptual contour following based on gradient properties. It combines selective edge detection and curve extraction into one process for fast curve detection. In this paper, we extend this approach by integrating color and gradient properties for enhancing decision making when choosing the most relevant edge pixels for curve tracking. We provide experiments to demonstrate the performance improvement by comparing the new curve tracker with other edge detection techniques and the previous curve tracking algorithm.