Discrete-Time Markov Chain - First Passage Probabilities
 The First Pass worksheet computes first passage probabilities from a given initial state to some other state. In the case shown, the first passage probabilities from state 0 to state 1 are shown in the last column of the display. The Start button allows entry of the initial and final states. The More button computes more first passage probabilities in sets of 20 (the number is set when the page is first created). This display is showing the probability distribution for the number of months required to pass from a new bulb to a bulb one month old for the first time. For the bulb example, the probability of first passage in one month is 0.5, the probability that the bulb does not fail during the first month. To have a first passage time of two months, the bulb must fail in the first month, but not fail in the second. This result has a probability of 0.25. For this simple case, the first passage probabilities are simply the geometric distribution with p = 0.5. The worksheet also shows the expected first passage times from each state into the specified final state. This vector is shown in the display at the right.

Operations Research Models and Methods
Internet
by Paul A. Jensen