With learning-based natural language processing (NLP) becoming the main-stream of NLP research, neural networks (NNs), which are powerful parallel distributed learning/processing machines, should attract more attention from both NN and NLP researchers and can play more important roles in many areas of NLP. This paper tries to reveal the true power of NNs for NLP applications as supervised of unsupervised learning devices by concretely introducing three practical applications: part of speech (POS) tagging, error detection in annotated corpora, and sel-organization of semantic maps.