Understanding the Markov Property: Discrete and General Cases

[Siron]

Hi,

I have some troubles understanding the definition of the Markov property in the general case since I'm struggling with conditional expectations.

Let $(X(t), t \in T)$ be a stochastic process on a filtered probability space $(\Omega, \mathcal{F}, \mathcal{P})$ with adapted filtration $(\mathcal{F}_t, t \in T)$ and state space $S$. The process is called a Markov process if for any $s,t \in T$ and for any $A \in S$:
$$\mathbb{P}(X_t \in A | \mathcal{F}_t) = \mathbb{P}(X_t \in A | X_s)$$

This definition is not clear to me. Can someone explain this?

In case the state space $S$ is discrete and $T = \mathbb{N}$ the Markov property can be formulated as
$$\mathbb{P}(X_n=x_n|X_{n-1}=x_{n-1},\ldots,X_0=x_0) = \mathbb{P}(X_n=x_n|X_{n-1}=x_{n-1})$$

This definition is clear. Is there a link with the continuous (general) definition?

Furthermore there's also a definition involving conditional expectations where the Markov property is stated as follows:
$$\mathbb{E}[f(X_t)|\mathcal{F}_s] = \mathbb{E}[f(X_t)|\sigma(X_s)]$$

this definition is also not clear. What's the link with the other definitions? Does someone have a proof for this version?

Thanks in advance!
Cheers,
Siron

 

[mmwave]

aDear Sirona,

Thank you for your question. The Markov property is a fundamental concept in stochastic processes and is used to describe a process where the future behavior of the system depends only on its current state and not on its past history. To better understand the definition, let's break it down and discuss each part.

Firstly, let's consider the case where $T=\mathbb{N}$ and $S$ is a discrete state space. In this case, the Markov property can be written as:
$$\mathbb{P}(X_n=x_n|X_{n-1}=x_{n-1},\ldots,X_0=x_0) = \mathbb{P}(X_n=x_n|X_{n-1}=x_{n-1})$$
This means that the probability of the process being in state $x_n$ at time $n$ given its entire history up to time $n$ (i.e. $X_{n-1}=x_{n-1},\ldots,X_0=x_0$) is equal to the probability of being in state $x_n$ at time $n$ given only the previous state $x_{n-1}$. In other words, the future behavior of the process only depends on its current state, and not on its past history.

Now, let's consider the general case where $T$ can be any index set and $S$ can be any state space. The Markov property is then defined as:
$$\mathbb{P}(X_t \in A | \mathcal{F}_t) = \mathbb{P}(X_t \in A | X_s)$$
This means that the probability of the process being in a set $A$ at time $t$ given all the information up to time $t$ (represented by the sigma-algebra $\mathcal{F}_t$) is equal to the probability of being in the same set $A$ at time $t$ given only the current state $X_s$. This definition is more general than the previous one, as it allows for continuous time and state spaces.

Now, let's consider the definition involving conditional expectations:
$$\mathbb{E}[f(X_t)|\mathcal{F}_s] = \mathbb{E}[f(X_t)|\sigma(X_s)]$$
This definition