Gibbs Random Fields (GRFs), which produce elegant models, but which have very poor computational speed have been widely applied to image segmentation. In contrast to block-based hierarchies usually constructed for GRFs, the irregular region-based approach is a more natural model in segmenting real images. In this paper, we show that the fineto- coarse region-based hierarchical regions framework for the well-known Potts model can be extended to non-edge based interactions. By deliberately oversegmenting at the finer scale, the method proceeds conservatively by avoiding the construction of regions which straddle a region boundary by computing region mean differences. This demonstrates the hierarchical method is able to model region interactions through new generalizations at higher levels in the hierarchy which represent regions. Promising results are presented.