We propose a structural character recognition method using simulated annealing, where the dissimilarity between an input and a reference pattern is evaluated by matching stroke and loop objects extracted from these patterns. This method can compute the matching at a high-speed. It can also avoid an incorrect matching problem when a object is divided into several objects in a broken character, by generating a hypothesis for an object's integration in the matching process. We have applied this method to handwritten digits recognition. The experimental results reveal that this method has the same degree of recognition rate and about 1/10 of the computing time compared with a conventional matching method using probabilistic relaxation. Moreover, generating a hypothesis for an object's integration is effective for dealing with broken characters.