The problem of non-parametric probability density function (PDF) estimation using Radial Basis Function (RBF) Neural Networks is addressed here. We investigate two criteria, based on a modified Kullback-Leibler distance, that lead to an appropriate choice of the network architecture complexity. In the first criterion the modification consists in the addition of a term that penalizes complex architectures (MPL criterion). The second strategy involves the regularization of the network through the imposition of lower bounds on the standard deviation derived from conditions of existence of rejection tests (LBSD criterion). Experimental results indicate that the MPL criterion outperforms the LBSD method.