Markov chains have been used to model web usages and served as the basis for statistical testing, performance evaluation, and reliability analysis. However, most of such applications of Markov chains were carried out without answering the question: "Can web usage be accurately modeled by Markov chains?" In this paper, we propose a set of tests, which are easy to perform based on information about web link usage frequencies gathered from web server logs, to answer this question. We applied this approach to our university's web site, and our results indicate that Markov chains can provide fairly accurate models of web usages.