35th Annual Hawaii International Conference on System Sciences (HICSS'02)-Volume 3 Big Island, Hawaii January 07-January 10 ISBN: 0-7695-1435-9
System dynamics-based simulation models are useful for analyzing complex systems characterized by both large parameter spaces and pervasive nonlinearity. Unfortunately, these characteristics also make confidence intervals for the model's outcomes difficult to assess. Standard Monte Carlo testing with a priori realistic parameter variations produces simulated behavior that is a posteriori improbable, rendering simple approaches inappropriate for establishing confidence intervals. This paper gives a case study of a model used to forecast completion of design and construction of a defense platform. For the first time for this class of model, a confidence interval for outcome is computed, using Monte Carlo trials and discarding combinations that do not achieve an acceptable fit of simulated behavior to historical data. The vast majority of simulation trials were markedly dissimilar to actual program history. More than 50,000 trials were required, and with parameter variations far smaller than the a priori bounds, to create about 70 trials that met the fit criteria. For this case, the experiment confirmed the intuitive view that a well-formulated closed loop model calibrated against sparse but widespread data and an appropriate statistical fit criterion can create tight confidence intervals on some model outcomes.
Index Terms:
Monte Carlo, fit-constrained parameters, historical data, a posteriori, system dynamics, confidence interval, outcome, calibration, project, program
Citation:
A. Graham, C. Choi, T. Mullen, "Using Fit-Constrained Monte Carlo Trials to Quantify Confidence in Simulation Model Outcomes," hicss, vol. 3, pp.96, 35th Annual Hawaii International Conference on System Sciences (HICSS'02)-Volume 3, 2002 Usage of this product signifies your acceptance of the Terms of Use. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||