Discrete-Time Markov Chain - Probability Analysis
 The Probabilities worksheet computes and displays the n-step transition matrix. The Start button sets the matrix to equal the transition matrix defined on the Matrix worksheet. The More button, provides subsequent transition matrices. The example below shows the transition matrix for the first three months of the light bulb case. The 1-step matrix is the transition matrix given in the original model. The 2-step matrix is the square of the 1-step matrix. The k+1 step matrix is produced by multiplying the k-step transition matrix by the 1-step transition matrix. The information is colored green because it is computed by a VBA algorithm. To illustrate the meaning of the matrix contents, consider the 3-step matrix. An entry in row i column j gives the probability of a transition from state i to state j in three steps. For example, the row for 2-mo. gives the probability distribution for the age of the bulb at some location, given that the bulb was 2 months old at the location at the beginning of a three month period. Clicking the More button produces more steps. At the 10th step, the rows of the matrix are the same as the steady-state probabilities to three decimal places. The two vectors at the right show the average step cost and the present worth of the step costs conditioned on the initial state.

Operations Research Models and Methods
Internet
by Paul A. Jensen
Copyright 2004 - All rights reserved