|
|
 |
|
Markov
Collection
|
|
|
 |
Markov
Collection |
 |
 |
The Markov Collection is a series of add-ins associated
with Markov Processes. The Markov Process is a very general
representation of time varying stochastic process. Additional
discussion of this type of model are in the Models Section
where articles discuss: Stochastic
Processes, Discrete
Time Markov Chains, and Continuous
Time Markov Chains. Queuing is
a class of Markov Models that allow closed form solutions. Simulation is
a way to model and analyze more complicated stochastic
systems. Although we have not included the Queuing or Simulation add-ins
in this collection they are certainly relevant to the
student/practitioner of stochastic processes. The Random
Variables add-in may play a role in constructing
models. The Inventory add-in
describes an important application area.
The picture shows the ORMM menu when the add-in collection
is installed. Each add-in has an individual role and
can be operated independently from the rest. The Markov
Data add-in calls the Markov Models add-in to construct
models, and the Markov Models add-in calls the Markov
Analysis or Markov Decision Process add-ins to obtain
solutions.
This page provides brief introductions to the collection
add-ins, Start and Finish commands that are common to
the collection, and links to install the individual
add-ins. Be sure to read the general
instructions for installing and using add-ins before
attempting to use them.
The first column on the table below has links to download
the individual add-ins. The second column provides brief
introductions. The titles link to more lengthy descriptions
appearing elsewhere on this site. Click the Start/Finish
link in the left margin to learn about adding and deleting
buttons on pages. If you use any of the example worksheet
pages you must learn to use this command. |
|
|
|
|
Markov Data
This add-in builds
constructs data forms and Markov models
for specific problem classes. Models are
provided for birth death processes and
finite queues. Changing model parameters
automatically adjust the model to reflect
the changes. Large CTMC, DTMC and MDP models
can be constructed with simple changes
in parameter data. The Markov Data add-in
calls the Markov Models add-in to construct
the model forms.
|
|
|
Markov
Models
This add-in builds models
for Continuous Time and Discrete Time Markov
processes, and also Markov Decision Process
Models. The models are symbolic in that
formulas are entered for transitions, transition
rates, transition probabilities and other relevant
components of a stochastic process model. The
DTMC or CTMC models are constructed
automatically. The models constructed are analyzed
using the Markov Analysis add-in. Chapter 11,
12 and 14 in the ORMM book support this topic.
|
|
|
Markov
Analysis
This add-in performs
a variety of computations associated with DTMC
(Markov Chains) and CTMC (Markov Processes)
including: economic analysis, steady state
analysis, computation of n-step probabilities,
simulation, computation of first passage probabilities
and computation of absorbing state probabilities. Chapters
13 and 15 in the ORMM book support this topic.
|
|
|
Markov
Decision Process
A Markov Decision Process (MDP) adds decisions
to Markov analysis . At each state there are two
or more decision options that affect the costs and
transition probabilities of the Markov chain. The
system will operate the system in the least costly
way if the operator chooses decisions indicated by
the optimum policy. This add-in determines the optimum
policy
|
|
Queuing
Although not included in the
collection, the subject of queuing theory is important
because it provides closed form equations for evaluating
important special cases. The Queuing Add-in computes
steady-state measures associated with Poisson queuing
models, non Markovian queues and networks of queues.
Both open and closed Markovian queues are modeled.
The program also simulates multiple channel queues
using two methods, a discrete next-event simulation
and an entity simulation. An optimization feature
is also included.
|
|
Simulation
Although not included in the
collection the subject of simulation is necessary
for complicated systems and when random variables
are not governed by the Markov assumption. The Simulation add-in
creates multiline simulations useful for analyzing
a variety of systems that don't fit the model types
handled by the other add-ins. The add-in builds and
maintains worksheets on which simulations are easily
built. Packaged models for time series simulation
and inventory simulation are included. For most simulations,
the RV add-in should also be installed.
|
|
|
