The Dynamic Programming Collection
is a series of add-ins associated with processes that
involve states, actions and events.
Many situations can be described by a collection
of mutually exclusive states. Say we have a
situation where the state is observed over time. The
system will begin in some initial state. A decision maker
has a set of alternative actions. He will choose
one of these based on the current state. Some reward
may be earned by the current state and the selected action.
We call the combination of the state and action a decision.
decision will cause the system to change from the initial
state to a new state. The change
in state is called a transition.
When the state reached via the transition is assuredly
determined, the model is called a deterministic
When the resultant state is uncertain,
we say that the state is determined by some event,
where the probability of the event is given. Each decision
determines a collection of mutually exclusive events
with the total probability of the collection equal to
1. When the results of decisions are uncertain the model
is called a stochastic dynamic program.
Dynamic programming is considered in three add-ins of
the ORMM collection. They can be used together or in
some cases separately to construct and solve significant
dynamic programming problems.
The picture shows the ORMM menu when the DP add-in collection
is installed. Each add-in has an individual role. The
DP Data add-in accepts data concerning general
situations and calls the DP
to construct models. The DP Models add-in calls the
DP Solver add-in to obtain numerical solutions.
The DP Models add-in also models Markov Chain problems.
These are described by states and events alone. No actions
are involved. Models constructed can either be analyzed
with the DP Solver add-in or the Markov
Markov Decision Processes are subsumed under
this discussion as they are similar to stochastic dynamic
This page provides brief introductions to the three
add-ins. The first column on the table below has links
to download the individual add-ins. The second column
provides brief introductions. The titles link to more
lengthy descriptions appearing elsewhere on this site.
Click the Start/Finish link in the left margin to learn
about adding and deleting buttons on pages. If you use
any of the example worksheet pages you must learn to
use this command. Be sure to read the general
installing add-ins before attempting to use them.