Connectionist networks are interesting computational models that have been proved to be useful for a range of applications. Knowledge in connectionist networks is encoded in distributed internal weights. Learning algorithms based on numerical optimization techniques can adapt these weights for a specific task (e.g., pattern classification). One of the major criticisms against the connectionist approach, however, is that knowledge generated by neural networks is not explicitly represented in the form of rules suitable for verification or interpretation, i.e., they are often regarded as black boxes. In this paper, we will discuss the ability of information exchange between connectionist and symbolic representations in a novel Hybrid Knowledge-Based Connectionist Network, called HKBCN. With our newly developed hybrid approach, domain knowledge can be encoded into the network, revised over time, and decoded into symbolic forms.