Understanding Markov Processes & Stationary Distributions

[senan]

Homework Statement

Consider a Markov Process X(t) over discrete set omega = {-1,0,1} with transition probabitities

W(-1 | 0,t)=W(0 | -1,t)=W(1 | 0,t)=W(0 | 1,t)=d

a) What is master equation

b) Find the stationary distrubtion P_s(x) for x element of {-1,0,1}

The Attempt at a Solution

a)
the master equation is d p(z,t)/dt = Integrate[W(z | y,t)p(y,t) - W(y | z,t)p(z,t)]dy

so how do you relate the transition probabilites to the master equation?

b) if its stationary then the LHS of the master eqn is zero and you get

0= Integrate[W(z | y)p_s(y) - W(y | z)p_s(z)]dy

again I'm not exactly sure how to relate the transition probabities to the equation

 

[mighty2000]

.

Hello,

The master equation is a fundamental equation in the study of Markov processes. It describes the time evolution of the probability distribution of the system over a discrete state space. In this case, the state space is omega = {-1,0,1} and the master equation is given by:

dP(x,t)/dt = sum over y [W(x|y,t)P(y,t) - W(y|x,t)P(x,t)]

where P(x,t) is the probability of the system being in state x at time t and W(x|y,t) is the transition probability from state y to state x at time t.

To relate the transition probabilities to the master equation, you can think of them as the rates at which the system transitions from one state to another. So, for example, W(-1|0,t) would represent the rate at which the system transitions from state 0 to state -1 at time t.

For part b), as you correctly mentioned, the stationary distribution Ps(x) is the solution to the master equation when the left-hand side is set to zero. So, you can solve the equation by setting dP(x,t)/dt = 0 and solving for Ps(x). This will give you the stationary distribution for each state x in the state space {-1,0,1}.

I hope this helps clarify things for you. Let me know if you have any further questions.