Transience of MC: Proving b>=1

  • Thread starter alphabeta1989
  • Start date
In summary, We are given a model where X_{n+1} given X_n, X_{n-1},...,X_0 has a Poisson distribution with mean \lambda=a+bX_n where a>0,b\geq{0}. We are asked to show that X=(X_n)_{n\in\mathrm{N_0}} is a transient M.C if b\geq 1 by using the theorem that states that a necessary and sufficient condition for a Markov chain to be transient is the existence of a non-constant, non-negative super-harmonic function \phi. The definition of a superharmonic function is a function \phi: S \rightarrow R that satisfies
  • #1
alphabeta1989
3
0
1. Homework Statement
Consider the following model.

[itex]X_{n+1}[/itex] given [itex]X_n, X_{n-1},...,X_0[/itex] has a Poisson distribution with mean [itex]\lambda=a+bX_n[/itex] where [itex]a>0,b\geq{0}[/itex]. Show that [itex]X=(X_n)_{n\in\mathrm{N_0}}[/itex] is a transient M.C if [itex]b\geq 1[/itex].2. Homework Equations

How do we approach this question? I was thinking of using the theorem below.

Let [itex]X[/itex] be an irreducible Markov chain with countable state space [itex]S[/itex]. A necessary and sufficient condition for [itex]X[/itex] to be transient is the existence of a non-constant, non-negative super-harmonic function [itex]\phi[/itex].
3. The Attempt at a Solution
I was thinking of using an exponential function as a superharmonic function, but failed terribly. What superharmonic function can we use to prove transcience for [itex]b\geq 1[/itex] Thanks in advance.
 
Last edited:
Physics news on Phys.org
  • #2
alphabeta1989 said:
1. Homework Statement
Consider the following model.

[itex]X_{n+1}[/itex] given [itex]X_n, X_{n-1},...,X_0[/itex] has a Poisson distribution with mean [itex]\lambda=a+bX_n[/itex] where [itex]a>0,b\geq{0}[/itex]. Show that [itex]X=(X_n)_{n\in\mathrm{N_0}}[/itex] is a transient M.C if [itex]b\geq 1[/itex].


2. Homework Equations

How do we approach this question? I was thinking of using the theorem below.

Let [itex]X[/itex] be an irreducible Markov chain with countable state space [itex]S[/itex]. A necessary and sufficient condition for [itex]X[/itex] to be transient is the existence of a non-constant, non-negative super-harmonic function [itex]\phi[/itex].



3. The Attempt at a Solution
I was thinking of using an exponential function as a superharmonic function, but failed terribly. What superharmonic function can we use to prove transcience for [itex]b\geq 1[/itex] Thanks in advance.

Something is missing: you have included no statement about what happens to/with the super-harmonic function ##\phi## in the context of ##X##.
 
  • #3
Ray Vickson said:
Something is missing: you have included no statement about what happens to/with the super-harmonic function ##\phi## in the context of ##X##.

I attempted to use functions such as [itex]\phi(x) = e^{bx}, e^{(b-1)x}[/itex], but all of them are not superharmonic w.r.t [itex]X[/itex]. What type of functions should I attempt?
 
  • #4
alphabeta1989 said:
I attempted to use functions such as [itex]\phi(x) = e^{bx}, e^{(b-1)x}[/itex], but all of them are not superharmonic w.r.t [itex]X[/itex]. What type of functions should I attempt?

You are missing the whole point: WHAT is supposed to happen if I give you a superharmonic function? You quoted only half of a theorem; the other half is vital!
 
  • #5
Ray Vickson said:
You are missing the whole point: WHAT is supposed to happen if I give you a superharmonic function? You quoted only half of a theorem; the other half is vital!

I am sorry about that! This is the definition of a superharmonic function!

Let [itex]X[/itex] be a time-homogeneous irreducible Markov chain with countable state space [itex]S[/itex] and one-step transition probability matrix [itex]P(x, y)[/itex]. A function [itex]\phi: S \rightarrow R[/itex] is said to be superharmonic for X at [itex]x \in S[/itex] if [itex]\sum_{y\in S} P(x,y)\phi(y)\leq\phi(x)[/itex]
 
  • #6
alphabeta1989 said:
I am sorry about that! This is the definition of a superharmonic function!

Let [itex]X[/itex] be a time-homogeneous irreducible Markov chain with countable state space [itex]S[/itex] and one-step transition probability matrix [itex]P(x, y)[/itex]. A function [itex]\phi: S \rightarrow R[/itex] is said to be superharmonic for X at [itex]x \in S[/itex] if [itex]\sum_{y\in S} P(x,y)\phi(y)\leq\phi(x)[/itex]

If you don't care about signs, just getting a superharmonic ##\phi## is easy: in this case, ##\phi(x) = -x## is superharmonic if ##a>0,\: b \geq 1##. However, if you want a non-negative ##\phi## it is harder. You can follow the construction in
http://math.stackexchange.com/questions/165913/markov-chains-recurrence-and-transcience
 

FAQ: Transience of MC: Proving b>=1

What is the concept of transience in Markov chains?

In Markov chain theory, transience refers to a state in which the probability of eventually returning to a particular state is less than 1. This means that there is a possibility that the chain may never return to that state, or it may take an infinite amount of time to return.

How is transience measured in Markov chains?

The transience of a Markov chain is measured by the expected number of steps it takes for the chain to return to a particular state, also known as the expected hitting time. If this value is infinite, then the state is considered transient.

What is the significance of proving b>=1 in terms of transience of MC?

Proving that b (the expected number of steps to return to a state) is greater than or equal to 1, indicates that the state is transient. This means that the state may never be reached again or may take an infinite amount of time to return to.

How is b>=1 proved in Markov chains?

To prove that b>=1 in Markov chains, one must use mathematical techniques such as recurrence relations and limit theorems. These techniques involve analyzing the transition probabilities and expected hitting times of the chain to determine if the state is transient.

What are the practical applications of understanding the transience of Markov chains?

Understanding the transience of Markov chains has various practical applications, such as predicting the long-term behavior of a system, analyzing the stability of a system, and making decisions based on the expected time to return to a state. It is also used in fields such as finance, biology, and computer science.

Back
Top