# Continuous Time Markov Chain

##### Posted on

This note is based on Karl Sigman’s IEOR 6711: Continuous-Time Markov Chains.

For each $t\ge 0$, there is a transition matrix $P(t)=(P_{ij}(t))$, and $P(0)=I$.

Unlike the discrete-time case, there is no smallest “next time” until the next transition, there is a continuum of such possible times $t$.

Suppose now that whenever a chain enters state $i\in\cS$, independent of the past, the length of time spent in state $i$ is a continuous, strictly positive (and proper) random variable $H_i$ called the **holding time** in state $i$. The holding times must have the memoryless property and thus are exponentially distributed.

A CTMC can simply be described by a transition matrix $P=(P_{ij})$, describing how the chain changes state step-by-step at transition epochs, together with a set of rates ${a_i:i\in \cS}$, the holding time rates. Each time state $i$ is visited, the chain spends, on average, $\E(H_i)=1/a_i$ units of time there before moving on.