Continuous-Time Markov Chain - The Rate Matrix
 After the OK button is pressed on the state number dialog, the add-in constructs the worksheet that holds the rate matrix. We have entered names for each state in the State Name array, where S0 refers to the state when the foyer is empty, S1 is the state when one person is being served, and so on. Transition rates are entered in the rate matrix according the problem definition. For example, the mean time between arrivals is 0.5 minutes, so using minutes as the time interval, we determine an arrival rate of 2 per minute. When there are n persons using or waiting for the ATM (n < 5), an arrival adds one more to that number, so we place the number 2 as the rate of transition from state n to state n + 1 (when n is not equal to 5). We assume that arrivals balk when the foyer is full, so the row for state 5 does not show the arrival rate. Service transitions are handled similarly. With a single ATM, the rate of departures is 2.5 per minute, so we use this rate for transitions from state n to state n -1 (when n is not equal to zero). When the foyer is empty, no departures can occur so this rate does not appear in row 0. All other entries in the rate matrix are 0, because no other single event causes a transition in the ATM example. The figure shows the Matrix worksheet for the ATM example. The buttons at the top control actions within this sheet. The Change button brings forth the state definition dialog to allow the number of states to be changed. If the number is changed, rows and columns are added or deleted according to whether the new number is more or less than the original. Data already entered into the matrix is not affected unless rows and columns are deleted. The Calculate button initiates calculations on the worksheet. The Analyze initiates a program to identify the classes of states for the model. States are identified as being in one class or another, and as being transient or recurrent. The buttons along the left margin control the analyses performed on the rate matrix. Some are discussed on these pages. Whenever one of these buttons is pressed, a new worksheet is created to perform the specified analysis. The Embedded Markov Chain The embedded Markov Chain is constructed to the right of the rate matrix on the worksheet and is shown for the example below. This matrix is used for some computations and can used as the transition matrix for a Markov Chain analysis.

Operations Research Models and Methods
Internet
by Paul A. Jensen