There are many commercial software security assurance tools that claim to detect and prevent vulnerabilities in application software. However, a closer look at the tools often leaves one wondering which tools find what vulnerabilities. This paper identifies a taxonomy of software security assurance tools and defines one type of tool: web application scanner, i.e., an automated program that examines web applications for security vulnerabilities. We describe the types of functions that are generally found in a web application scanner and how to test it.