Most network anomaly detection research is based on packet header fields, while the payload is usually discarded. Preventing unknown attacks and Internet worms has led to a need for application level network anomaly detection. Payload based detection schemes in experiments are often misleading. In this paper, we discuss the problems associated with the experimental results. In the first section, a brief review will be given for application level anomaly detection research. Introduction to several major payload based approaches will be given in section 2. Then we use the DARPA?99 dataset to evaluate the ALAD mechanism, and discuss the problems by using original DARPA?99 datasets for evaluation. In the fourth section, an improved method will be proposed with a focus on detecting payload related attacks. In section 5, we demonstrate how to justify the payload based detection mechanism using the DARPA?99 dataset, and compare with ALAD to demonstrate its advantages.