Main Index Computation Index
Operations Research Models and Methods / Computation

Unit Markov Process Add-in


Rate Matrix

Embedded Markov Chain


Steady State


The Markov Process Add-in performs computations for continuous time Markov Processes. When the add-in is installed, the menu items on the left are added to the OR_MS menu.

When one of these options is chosen, the worksheet associated with the specified computation is constructed automatically. The MPMatrix entry must be selected before any of the others are available. Some results associated with a Markov Process can be had by analyzing the associated embedded Markov Chain. When the Markov Chain Add-in is installed, this is easily accomplished. The MPMatrix worksheet constructs the transition matrix for the embedded Markov Chain for use by the Markov Chain Add-in.

The MPRelink Buttons command is useful when opening a workbook containing a Markov Process model in a different computer than the one in which the model was created. The command deletes the old buttons are replaces them with buttons linked to the add-in in the new computer.

Example: Providing ATM Service

To illustrate the elements of the stochastic process model, we use the example of a single Automated Teller Machine (ATM) located in foyer of a bank. The ATM performs banking operations for people arriving for service. The machine is used by only one person at a time, and that person said to be in service. Others arriving when the machine is busy must wait in a single queue, and these people are said to be in the queue. Following the rule of first-come-first-served, a person in the queue will eventually enter service and will ultimately leave the system. The number in the system is the total of the number in service plus the number in the queue. The foyer is limited in size so that it can hold only five people. Since the weather is generally bad in this part of the country, when the foyer is full, arriving people do not enter. We have gathered statistics on ATM usage that show the time between arrivals averages 30 seconds (or 0.5 minutes). The time for service averages 24 seconds (or 0.4 minutes). Although the ATM has sufficient capacity to meet all demand, we frequently observe queues at the machine and occasionally customers are lost.

We want to perform an analysis to determine statistical measures that describe the number of people in the system, the waiting time for customers, the efficiency of the ATM machine, and the number of customers not served because there is no room in the foyer. We intend to use these statistics to guide managers in design questions such as whether another ATM should be installed, or whether the size of the foyer should be expanded.

The Rate Matrix

Main Index Computation Index

Operations Research Models and Methods
by Paul A. Jensen and Jon Bard, University of Texas, Copyright by the Authors