Discrete-Time Markov Chain - Economic Analysis
 The Economics worksheet is created at the same time as the Matrix worksheet and allows entry of economic data to evaluate the Markov Chain. The cell labeled Measure holds the word describing economic measures. Here we use cost, but it could be revenue or profit. The "discount rate" is used to compute the present value of the cash flows. It is the discount rate per period of the Markov Chain. There are two data areas: State Cost and Transition Cost. The "state cost" is expended whenever the chain enters a state. The matrix for "transition cost" specifies the expenditure caused by a transition from one state to another. For the example problem we assume that it costs \$0.1 to inspect a bulb and \$1 to replace it with a new one. Thus we have a cost of 0.1 no matter what state the system is in and assign a cost of \$1 for a transition into the next state. The Calculate button computes the "Expected State Value", this is the state cost plus the expected transition cost as determined by the transition probabilities. The Matrix button returns to the Matrix worksheet for this problem.

Operations Research Models and Methods
Internet
by Paul A. Jensen