This paper proposes a new technique of binarization and recognition of characters in color with a wide variety of image degradations and complex backgrounds. The key ideas are twofold. One is to automatically select one axis in the RGB color space that maximizes the between-class separability by a suitably chosen threshold for segmentation of character and background or binarization. The other is affine-invariant or distortiontolerant grayscale character recognition using global affine transformation (GAT) correlation that yields the maximum correlation value between input and template images. In experiments, we use a total of 698 test images extracted from the public ICDAR 2003 robust OCR dataset containing a variety of single-character images in natural scenes. In advance, we classify those images into seven groups according to the degree of image degradations and/or background complexity. On the other hand, we only prepare a single-font set of 62 alphanumerics for templates. Experimental results show an average recognition rate of 81.4%, ranging from 94.5% for clear images to 39.3% for seriously distorted images.