The Dynamic Programming Collection
is a series of add-ins associated with processes that
involve *states*, *actions* and *events*.
Many situations can be described by a collection
of mutually exclusive *states*. Say we have a
situation where the state is observed over time. The
system will begin in some initial state. A decision maker
has a set of alternative *actions*. He will choose
one of these based on the current state. Some reward
may be earned by the current state and the selected action.
We call the combination of the state and action a *decision*.
The
decision will cause the system to change from the initial
state to a new state. The change
in state is called a *transition*.
When the state reached via the transition is assuredly
determined, the model is called a *deterministic
dynamic program*.
When the resultant state is uncertain,
we say that the state is determined by some *event*,
where the probability of the event is given. Each decision
determines a collection of mutually exclusive events
with the total probability of the collection equal to
1. When the results of decisions are uncertain the model
is called a *stochastic dynamic program*.
Dynamic programming is considered in three add-ins of
the ORMM collection. They can be used together or in
some cases separately to construct and solve significant
dynamic programming problems.
The picture shows the ORMM menu when the DP add-in collection
is installed. Each add-in has an individual role. The
*DP Data* add-in accepts data concerning general
situations and calls the *DP
Models* add-in
to construct models. The *DP Models* add-in calls the
*DP Solver* add-in to obtain numerical solutions.
The DP Models add-in also models *Markov Chain* problems.
These are described by states and events alone. No actions
are involved. Models constructed can either be analyzed
with the *DP Solver *add-in or the *Markov
Analysis* add-in.
*Markov Decision Processes* are subsumed under
this discussion as they are similar to stochastic dynamic
programming models.
This page provides brief introductions to the three
add-ins. The first column on the table below has links
to download the individual add-ins. The second column
provides brief introductions. The titles link to more
lengthy descriptions appearing elsewhere on this site.
Click the Start/Finish link in the left margin to learn
about adding and deleting buttons on pages. If you use
any of the example worksheet pages you must learn to
use this command. Be sure to read the general
instructions for
installing add-ins before attempting to use them. |