Issue No. 08 - August (1993 vol. 15)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/34.236251
<p>A deterministic annealing approach to clustering is derived on the basis of the principle of maximum entropy. This approach is independent of the initial state and produces natural hierarchical clustering solutions by going through a sequence of phase transitions. It is modified for a larger class of optimization problems by adding constraints to the free energy. The concept of constrained clustering is explained, and three examples are are given in which it is used to introduce deterministic annealing. The previous clustering method is improved by adding cluster mass variables and a total mass constraint. The traveling salesman problem is reformulated as constrained clustering, yielding the elastic net (EN) approach to the problem. More insight is gained by identifying a second Lagrange multiplier that is related to the tour length and can also be used to control the annealing process. The open path constraint formulation is shown to relate to dimensionality reduction by self-organization in unsupervised learning. A similar annealing procedure is applicable in this case as well.</p>
neural nets; pattern recognition; optimization; deterministic annealing; maximum entropy; phase transitions; constrained clustering; cluster mass variables; total mass constraint; traveling salesman problem; elastic net; second Lagrange multiplier; open path constraint; dimensionality reduction; unsupervised learning; constraint theory; information theory; neural nets; pattern recognition; simulated annealing
E. Gurewitz, G. Fox and K. Rose, "Constrained Clustering as an Optimization Method," in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 15, no. , pp. 785-794, 1993.