In our previous experiments of a supply chain using Beer Game, we found a counterexample for the bullwhip effect such that inventory level of the upstream firm is not always larger than that of the downstream firm even in the environment that the number of firms is many and the length of delay in shipping is longer. In this paper, we propose several quantitative definitions of the bullwhip effect and re-estimate our previous findings with these definitions. To the best of our knowledge, no systematic attempt has been made to define quantitatively the bullwhip effect while it is de- fined qualitatively so far. This study is the first attempt to define the bullwhip effect quantitatively. We show that a frequency based statistical measurement such as stochastic dominance is not appropriate to capture the bullwhip effect quantitatively because it cannot distinguish between a case of the bullwhip effect and a counterexample. On the other hand, descriptive statistics such as mean and standard deviation works well.