This paper aims at providing insights onto fundamental properties of two of the earlier models of matrix memories: nonlinear or Willshaw's matrix model, and the linear matrix memory. These are traditional models of artificial neural networks (ANNs) on which little has been done in recent years on themes such as retrieval and storage properties, which are studied in this paper. Equations for predicting network connectivity and storage capacity as a function of the activity levels at the input and output fields and network dimension are presented for Willshaw's model. For the linear model, an equation for relating input and output net work errors is presented. The theoretical results presented for both models fit very well with computer simulations.