Discrete-Time Markov Chain - Transient Analysis
 The Transient worksheet is constructed by clicking the appropriate button on the Matrix page. Given an initial starting state, the worksheet shows the transient probabilities for 20 or more steps. The START button provides the opportunity to specify the initial state. Pressing the MORE button reveals probabilities for additional steps. In addition to state probabilities, columns provide the economic value for each step, the cumulative value and the present worth using the discount factor. The transient analysis for the example problem starting from the state of a new bulb is shown below. For example after 12 months of operation, approximately 60% of the bulbs in the sign will be new (after maintenance), while only about 1% will be entering their 4th month of life. The expected cost of maintaining one bulb in the sign is \$0.71, the cumulative cost for the twelve months is \$9.14, and the present worth of costs using a 10% discount rate per month is \$4.96. These figures can be used to estimate budgets or make decisions regarding alternative design or maintenance policies. The CHART button creates a chart of the transient probabilities as shown below. This chart is typical for transient probabilities for many systems. The state probability for the initial state starts at 1, while the others are zero. The probabilities pass through a transient period and then reach steady values. The values to which they converge are called the steady state probabilities and are computed on the next worksheet.

Operations Research Models and Methods
Internet
by Paul A. Jensen