Absorbing Markov Chain — Definition, Formula & Example
An absorbing Markov chain is a Markov chain with finitely many states and at least one trap state that, once entered, is never left, and from every other state it is possible to reach a trap eventually. Over time the process is certain to end up stuck in one of these trap states.
A state is absorbing if . A finite Markov chain is absorbing if it has at least one absorbing state and every state can reach some absorbing state in a finite number of steps. Non-absorbing states are called transient. Listing transient states first puts the transition matrix in canonical form . Here holds transient-to-transient probabilities and holds transient-to-absorbing probabilities.
Key Formula
Where:
- = Square matrix of transition probabilities between transient states
- = Matrix of probabilities of moving from a transient state directly into an absorbing state
- = Fundamental matrix; entry is the expected number of visits to transient state when starting from transient state
- = Column vector of expected number of steps before absorption from each transient state
- = Column vector of all ones
- = Absorption probability matrix; entry is the probability of ending in absorbing state when starting from transient state
Worked Example
Problem: You start with $1 and bet $1 on each flip of a fair coin, winning or losing $1 each round. You stop as soon as you have $0 (broke) or $3 (your goal). What is the probability you reach $3, and how many rounds does the game last on average?
Step 1: The states are $0, $1, $2, $3. States $0 and $3 are absorbing. List the transient states ($1, $2) first. Then read off Q (rows and columns ordered $1, $2) and R (rows $1, $2; columns $0, $3).
Step 2: Compute the fundamental matrix. The determinant of I − Q is 1 − 1/4 = 3/4.
Step 3: Row sums of N give the expected number of rounds. The product NR gives the absorption probabilities. The first row of each result corresponds to starting with $1.
Answer: Starting with $1, you reach $3 with probability 1/3 and go broke with probability 2/3. The game lasts 2 rounds on average.
Why It Matters
Absorbing chains answer "where does it end, and how long does it take" questions in a stochastic processes or finite math course. Examples include gambler's ruin, whether a gene becomes fixed in a population, and how long a loan stays current before it is paid off or defaults.
Common Mistakes
Mistake: Calling a chain absorbing just because it has one absorbing state.
Correction: Every transient state must also be able to reach an absorbing state. If some group of states loops among itself with no path to a trap, the chain is not absorbing, and I − Q may fail to be invertible.
