Question about sample spaces (in probability)

[AxiomOfChoice]

So a stochastic process (e.g., the simple random walk) is defined as a sequence of random variables. And random variables are defined as real-valued (measurable) functions on the sample space $\Omega$ of some probability triple $(\Omega, \mathcal M, \mathbb P)$.

Question: In the case of the simple random walk - which is a sequence $X_1, X_2, \ldots$ of i.i.d. random variables such that $\mathbb P (X_i = \pm 1) = 1/2$, what on Earth is the sample space on which these random variables are defined? My best guess is that it's the set of all sequences of the form $(\pm 1, \pm 1, \pm 1, \ldots)$, such that $X_i$ just spits out the $i$th entry.

 

[AxiomOfChoice]

Hmmm...now that I think about it, I guess I have the same basic question about simple Brownian motion $B(t)$ in one dimension, which has a continuous index and continuous state space. What's its sample space?

 

[micromass]

You are asking an extremely good question. You'll need to find an $$\Omega$$ such that there exists functions $$X_i:\Omega\rightarrow \mathbb{R}$$. This is not trivial at all!

In fact, constructing suitable sample spaces can be quite a difficult task. Let me construct the sample space of the random variables in your OP. You will see right away that this is not a trivial task.

For $$n\geq 1$$ and $$(\epsilon_1,...,\epsilon_n)\in \{0,1\}^n$$, we put

$$I^{(n)}_{\epsilon_1,...,\epsilon_n}=]\sum_{i=1}^n{\epsilon_i2^{-i}},2^{-n}+\sum_{i=1}^n{\epsilon_i2^{-i}}]$$

Put $$\Omega=[0,1]$$ with the ordinary sigma-algebra and Lebesgue-measure. Then we set $$B_1=I_1^{(1)}$$ and $$B_n=\bigcup_{(\epsilon_1,...,\epsilon_{n-1})\in \{0,1\}^{n-1}}{I_{\epsilon_1,...,\epsilon_{n-1},1}^{(n)}}$$ for n>1.

Then $$P(B_n)=1/2$$. We put

$$X_n(x)=\left\{\begin{array}{c}1~\text{if}~x\in B_n\\ -1~\text{if}~x\notin B_n\end{array}\right.$$

This will be the random variables we're looking for. They are identically distributed, and also independent because

$$P(\{X_1=\epsilon_1,...,X_n=\epsilon_n\})=2^{-n}$$ for $$(\epsilon_1,...,\epsilon_n)\in \{0,1\}^n$$.

Finding the sample space of the Brownian motion is also possible, but even more complicated!
This is why probability theorists will simply not care about the sample space. They will show that the sample space exists, and that's all they care about. The only thing important for probability is the functions $$X_n$$...

 

[AxiomOfChoice]

You are asking an extremely good question. You'll need to find an $$\Omega$$ such that there exists functions $$X_i:\Omega\rightarrow \mathbb{R}$$. This is not trivial at all!

In fact, constructing suitable sample spaces can be quite a difficult task. Let me construct the sample space of the random variables in your OP. You will see right away that this is not a trivial task.



Finding the sample space of the Brownian motion is also possible, but even more complicated!
This is why probability theorists will simply not care about the sample space. They will show that the sample space exists, and that's all they care about. The only thing important for probability is the functions $$X_n$$...

Wow! Interesting. Thanks. I've got a few more questions, though:

(1) What is wrong with my suggestion about the sample space for the simple random walk?

(2) Is Kolmogorov's Existence Theorem concerned with proving the sample space exists (for, say, Brownian motion)?

 

[micromass]

(1) What is wrong with my suggestion about the sample space for the simple random walk?

Ah, I'm sorry, I forgot that you had your own suggestion. Well, your suggestion is a very good $$\Omega$$, but that's not enough for a sample space. You'll also need a sigma-algebra on it, but I guess you'll take $$\mathbb{M}=\mathcal{P}(\Omega)$$. But you will also need a probability measure on $$\Omega$$, and I don't really see how you can define a good probability measure on that space...

(2) Is Kolmogorov's Existence Theorem concerned with proving the sample space exists (for, say, Brownian motion)?

Indeed, the Kolmogorov's Existence Theorem states something similar. It provides a suitable probability function on $$(\mathbb{R}^T,\mathcal{R}^T)$$ such that certain distributions are coming from this sample space. So yes, it provides the existence of a suitable sample space... And the Kolmogorov's Existence theorem is indeed used in the proof that the Brownian motion exists. A quick look at the proof reveals that the sample space of the Brownian motion is again the unit interval with the Borel sigma algebra. But I guess that the probability measures on that interval will be more complicated...

 

[AxiomOfChoice]

A quick look at the proof reveals that the sample space of the Brownian motion is again the unit interval with the Borel sigma algebra. But I guess that the probability measures on that interval will be more complicated...

I'd love to see the proof. Where are you looking? Is it in a book? I might be able to get it at my school's library...of course, if it's online, that's even better, since you could just link me to it Thanks, again, for your help!

 

[micromass]

I'd love to see the proof. Where are you looking? Is it in a book? I might be able to get it at my school's library...of course, if it's online, that's even better, since you could just link me to it Thanks, again, for your help!

Check out "Probability and Measure" by Billingsley. Your school's library should have it, since it is considered by many to be the best book on probability there is (although it's probably not suitable for very beginners).

In theorem 5.2, Billingsley constructs sample spaces for several random variables.
In theorem 8.1, Billingsley constructs sample spaces for Markov chains.
In theorem 37.1, Billingsley constructs the Brownian motion, the proof is quite long however.

You can find Billingsley's book on the internet. But I guess that the mentors won't like me posting the link since it's an illegal download...

 

[AxiomOfChoice]

They actually *do* have Billingsley, so I'll head over and check it out. Thanks!

Here's another quick question, just to clarify: If we consider Brownian motion $B(t)$ as a continuously indexed stochastic process on $(\Omega,\mathcal M, \mathbb P)$ (where, as you said, $\Omega = [0,1]$ and $\mathcal M$ is the Borel sets), and we look at it from the vantage point of [itex]t \mapsto B(\omega,t)[/tex] for fixed $\omega \in \Omega$, we then have something deterministic, right (at least in principle)? I mean, for any fixed $\omega$, we know how $B(t)$ acts on it, right?

 

[micromass]

Yes, if we know $$\omega$$, then we know $$B(t,\omega)$$ for any t. But in Brownian motion research, we don't know $$\omega$$. It's the same with any random variable X, really. If we know $$\omega$$, then we of course know $$X(\omega)$$, but can we infer information about X without knowing $$\omega$$ explicitly, that's the question.

I hope I didn't misunderstood your question...

 

[AxiomOfChoice]

Yes, if we know $$\omega$$, then we know $$B(t,\omega)$$ for any t. But in Brownian motion research, we don't know $$\omega$$. It's the same with any random variable X, really. If we know $$\omega$$, then we of course know $$X(\omega)$$, but can we infer information about X without knowing $$\omega$$ explicitly, that's the question.

I hope I didn't misunderstood your question...

No, I don't think you did. It's just that when one sees "plots" of Brownian motion...one is immediately led to ask, "How did they get the plot?" There has to be something deterministic there. And from what I can tell, this is done by fixing an $\omega$ and then plotting $t$ on the horizontal axis, $B(t|\omega)$ on the vertical axis. (I understand that it's not strictly speaking possible to plot a true Brownian motion; what I'm assuming is done is that one of the discrete approximations to it is plotted.) I think this is what is meant by associating with Brownian motion the notion of a "random function".

 

[micromass]

I think that the easiest way to get plots of Brownian motion is by some kind of Monte Carlo method. That is, instead of calculating the Brownian motion, we approximate the rules of the Brownian motion and plot a function that looks like it.

Of course, the Brownian motion is deterministic in the sense that if we know $$\omega$$, then we can graph $$B(\cdot,\omega)$$. But I can see a number of possible objections:
1) You don't know $$\omega$$ in the usual applications, you can only approximate it
2) The formula for the Brownian motion is probably very difficult or impossible to write down (if the proof of existence is nonconstructive). You should check the proof of the existence, but I've never seen an easy formula for the Brownian motion...

I'm also by far no expert on Brownian motions, so don't simply believe everything I say...

 

[AxiomOfChoice]

I think that the easiest way to get plots of Brownian motion is by some kind of Monte Carlo method. That is, instead of calculating the Brownian motion, we approximate the rules of the Brownian motion and plot a function that looks like it.

Of course, the Brownian motion is deterministic in the sense that if we know $$\omega$$, then we can graph $$B(\cdot,\omega)$$. But I can see a number of possible objections:
1) You don't know $$\omega$$ in the usual applications, you can only approximate it
2) The formula for the Brownian motion is probably very difficult or impossible to write down (if the proof of existence is nonconstructive). You should check the proof of the existence, but I've never seen an easy formula for the Brownian motion...

I'm also by far no expert on Brownian motions, so don't simply believe everything I say...

Okay, fair enough. But when you hear someone talking about the interesting properties of Brownian motion - i.e., that the paths are (almost surely) nowhere differentiable, or that they are locally Hoelder continuous for any Hoelder constant less than 1/2 - they're implicitly fixing an $\omega \in \Omega$ and talking about the behavior of $B(t|\omega)$ as a function on $[0,\infty)$, right? I mean...when you say "almost surely," aren't you talking about what happens for all $\omega\in \Omega$ except for possibly $\omega \in A$ where $\mathbb P(A) = 0$?

 

[micromass]

Yes, indeed, in that cases, they fix an omega, and they regard $$B(t,\omega)$$. For allmost all $$\omega$$, this function $$B(\cdot,\omega)$$ has weird properties, like nowhere differentiable and stuff.

 

[Stephen Tashi]

one is immediately led to ask, "How did they get the plot?" There has to be something deterministic there. And from what I can tell, this is done by fixing an $\omega$ and then plotting $t$ on the horizontal axis, $B(t|\omega)$ on the vertical axis. (I understand that it's not strictly speaking possible to plot a true Brownian motion; what I'm assuming is done is that one of the discrete approximations to it is plotted.) I think this is what is meant by associating with Brownian motion the notion of a "random function".

Writing a computer program to plot Brownian motion to a time resolution input by the user is an excellent way to get an intuitive understanding of it. The first idea that would pop into the average computer progammer's head is to have a step size (say deltaY = 1) for the time interval deltaT = 1. Pick a random jump of plus or minus deltaY at intervals of deltaT. Plot them and plot another graph of the sum of all the current jumps at time T (I think this sum would be "The Wiener Process" associated with the Brownian motion.

If the user wants to see Brownian motion at increments of deltaT = 1/2 the natural mistake would be to make deltaY = 1/2 and do the same type of graph. However, once the user inputs a very small deltaT, say 1/1000, the error in using the proportional deltaY = 1/1000 will be obvious since the graphs of the sums of the deltaY's at time T won't look right. (I.e. they won't look as variable as they did when deltaY = 1/2 and deltaT = 1/2, so the new process won't be a plausible "interpolation" of the coarser process.

The computer programmer who has made this mistake will start down the path toward discovering a formula for how deltaY ought to be reduced when deltaT is reduced. Once that formula is found, Brownian motion can be thought about as a limit of a sequence of stochastic processes, each using a smaller and smaller deltaT and each being a plausible interpolation of its predecessor.