The central theme of this paper is to overcome the inability of feedforward neural networks with hard limiting units to provide confidence evaluation. We consider a Madaline architecture for a $2$-group classification problem and concentrate on the probability density function for the neural activation of the first-layer units. As the following layers perform a Boolean table, the expectation value of the output is determined, utilizing the probability of a pattern to perform a definite binary input for the Boolean table. The Madaline architecture can be modified to the introduced Sigma-Pi-Sigma network, which evaluates the expectation value. Several assumptions on the distribution of the neural activation lead to a clear and simple architecture, which is applied to an OCR problem.