The defect inspection that used image sensing such as automated pattern inspection is a useful solution to automatize the visual check, not limit to factory automation field. Mostly such defect inspection is using the models of defect that described by primitive features. This paper proposes a new defect detection method that is the non-model based approach. In this approach, the method extracts the image description rule from local regions.It is useful for the defect inspection problems that cannot prepare a defect model such as scratch or superimpose detection, texture image analysis, etc. In the experiment, I tried the defect detection to the landscape picture which several types of superimpose were added. From these results, it was confirmed that the proposed method has high ability to detect the defected regions independently with the texture type.Furthermore, I attempted the application to a scene image.Therefrom, the possibility to apply the figure-ground separation of the image understanding basic problem was confirmed.