Cell morphology is an often-utilized feature in biology and medical science because changes in the shape of a cell indicate that some abnormal alterations may have occurred within the cell. However, there is little knowledge about the relationship between phenotype and genes. In this study, for determining the genetic effect on phenotypes,we designed a new algorithm called phenotype analysis with layered conjecture (PALACE); this algorithm which infers a gene network that represents the dependency of morphological features using a comprehensive yeast phenotype dataset resulting from the deletion of one gene. PALACE first creates gene groups in which the abnormal phenotypes of its members are mutually close; then, it generates a network with groups based on inclusion relations between abnormal phenotypes. A network inferred from 172 transcriptional genes comprises 63 gene groups and 183 edges. The inferred network has biologically reasonable features; however, it is independent of known metabolic networks. Therefore, the network is expected to include information leading to new insights into cell biology.