Validation of Multivariate Response Trace-driven Simulation Models

Abstract

A procedure is developed by using Rotelling's one-sample T^2 test to test the validity of a multivariate response trace-driven simulation model that represents an observable system. The vali- dity of the simulation model is tested with respect to the mean behavior under a given experimental frame. A procedure for cost-risk analysis for the one-sample T^2 test is developed. By using this procedure, a trade-off analysis can be performed and judgement decisions can be made as to what data collection budget to allocate, what data collection method to use, how many paired observations to collect on the model and system re- sponse variables, and what model builder's risk to choose for test- ing the validity under a satisfactory model user's risk. The procedure for validation and the cost-risk analysis are illustrated for a trace-driven simulation model that represents a time-sharing computer system with two performance measures of interest.