Model Verification and Validation
After developing the simulation model,
we performed several different verification
and validation steps to ensure the model
results were reliable [19]. This included
verifying the model architecture with other
researchers and engineers; comparing the
simulation output to analytical models for
simple cases; and, showing the results to
managers actively working on supply chain
issues at the inkjet printer division of HP, who
could compare the results to their previous
experiences. For steady state cases with no
lost sales and stationary demand, we also
compared the simulation results for fill rate to
those from a well known analytical model
[20]. Appendix Table A1 contains the
simulation and analytical estimates of fill rate
for many different parameter settings (forecast
error and safety stock levels). The simulation
results include both the average fill rate and
associated 95% confidence intervals based
on 1000 independent replications of 16
simulated months starting at steady state. As
can be seen from the table the simulation
results for cases with low forecast error, as
defined by the coefficient of variation (CV =
standard deviation of forecast error/forecast)
less than 0.2, match those of the analytical
model (analytical results fall within 95%
confidence intervals from the simulation).
The analytical model assumes that the
forecast error is small enough to ensure
nonnegative demand. In the simulation,
negative demand was not allowed (truncated
at zero). As expected [21] when the forecast
error grows (CV > .3), the analytical model
slightly underestimates the true fill rate as
shown by the simulation results.
As part of the verification and validation
process, we also used the simulation to
examine earlier findings on benefits of
postponement for stationary demand under
steady state. We examined a set of scenarios
where eight derivative products were either:
• Customized at the factory and then
shipped to the distribution centers; or
• Manufactured as a generic product that
was later differentiated at the distribution
centers.
Table 1 shows the parameter settings for
those experiments.
Figure 4 shows that for the same
inventory policy, the average fill rate achieved
using postponement are substantially higher
than differentiating the products at the factory.
Figure 5 shows that postponement has the
greatest impact when the forecast error is
high, however the postponement strategy
performs well over a range of forecast errors.
These results are consistent with earlier
findings. Detailed results from the simulation
including confidence intervals for all
estimates and comparisons to analytical
model results are included in the Appendix
Table A2.