Simulation models use numerous Random Seeds to demonstrate natural variations in driving behaviors and vehicle entry into the network. To confirm validity of a model, it is recommended that the number of simulation runs is large enough where an atypical run has minimal effect on the sample average. Simulation runs are typically determined for every model being tested. Per FHWA guidance, the following formula is used to calculate the necessary minimum number of simulation runs with a 95% confidence level. A 95th-percent confidence interval is typically used, but may not be the best value for every analysis:

where:

N: Minimum number of simulation runs

x̄

: Mean of observations

𝑠: Standard deviation of observations

n: Number of observations

tn

−1,95% : t statistic for n-1 degrees of freedom and 95% confidence level

e: tolerance error percentage

To use this equation, first select a preliminary number of model runs for which the initial mean and standard deviation are derived. These initial values are then used in the equation to determine the minimum number of simulation runs. If the calculated number of simulation runs is unreasonably high, first confirm there are no significant deviations in any of the simulation runs that would skew the calculation. After that, if the calculated number of simulation runs is still high, the cap is 25 runs per project. Generally, the minimum number of simulation runs is 10.