Tom's Kopfbahnhof

Search

Search IconIcon to open search

Deterministic Markov process

Last updated Dec 10, 2022

A deterministic Markov process defines the evolution of the sample paths $x(t)\in \Reals^d$ with a system of ODEs, e.g. $$\frac{\text{d}x(t)}{\text{d}t} = g(x(t)).$$ If $g$ does not explicitly depend on the time, the resulting process is homogenous.

An ODE corresponds to a phase flow $\Phi: \Reals \times \Reals^d \to \Reals^d$, which defines the trajectory $x(t)$ for every starting point $x_0$. We therefore have $\Phi(0,x_0) = x_0$ and generally $\Phi(t, x_0) = x(t)$ is the solution of the ODE for the given initial condition $x(0) = x_0$.

Since the sample paths are given by the phase flow, the propagator $T$ just sets $x$ equal to the sample path obtained from the time interval $t - t^’$ $$T(x,t|x^’,t^’) = \delta(x - \Phi(t - t^’, x’)).$$ With the group property of the phase flow $\Phi(t,\Phi(s,x)) = \Phi(t+s,x)$ this satisfies the Chapman-Kolmogorov equation.

This $\delta$ function expresses the deterministic nature of the process: The propagator is only non-zero if the Phase flow carries $x^’$ exactly to $x$, that is if $x = \Phi(t-t’, x^’)$

The generator $\mathcal{A}$ becomes $$\mathcal{A}\rho(x) = -\sum_i \frac{\partial}{\partial x_i}\left(g_i(x)\rho(x)\right)$$

Definition: Liouville equation The Liouville equation is the differential Chapman-Kolmogorov equation for the deterministic process. Inserting $\mathcal{A}$ we obtain it as $$\frac{\partial}{\partial t}T(x,t|x^’,t^’) = -\sum_i \frac{\partial}{\partial x_i}\left(g_i(x) T(x,t|x^’,t^’)\right).$$ The density $p(x,t)$ fulfills the same equation