Return to Index
Operations Research Models and Methods
Computation Section
Subunit Discrete-Time Markov Chain
 - Simulation

The Simulate worksheet performs a discrete event simulation of the Markov Chain. Starting from a specified state, the Markov process is simulated for 20 or more steps using a Monte Carlo simulation. The number of steps in the display is set when the worksheet is created. Additional steps can be simulated by clicking on the More button. Charts can be made of the state, state value, cumulative value or discounted value. While all the other worksheets show probability distributions or expected values over collections of realizations, the simulation shows a single realization of the process. The presentation helps illustrate the dynamic nature of a Markov process and the statistical affects of observing the process for a finite number of steps.

The figure shows the first 20 steps of the simulation.

  The add-in computes statistics based on the simulation. The Cumulative Count row accumulates the numbers of observations in the several states, while the frequency row shows the relative frequencies observed. For Ergodic DTMCs this vector should converge to the steady-state probability vector.
  Clicking the button above the state column presents a graph of the states over the 20 steps.
  Clicking the More button several times, simulates the first 100 steps.
  The State Frequency row is beginning to resembled the steady-state vector.


Return to Top

tree roots

Operations Research Models and Methods
by Paul A. Jensen
Copyright 2004 - All rights reserved