In general, a probability model is a pair \((\Omega,P)\text{,}\) where \(\Omega\) is a set, and where \(P\) is a function on a subset of the power set of \(\Omega\text{.}\) What all probability models have in common are the vocabulary (Definition 2.5) and properties (Proposition 2.7) that you have already seen for finite probability models in the previous section. We begin with a generalization of the finite probability model.
The definitions (Definition 2.1, Definition 2.2, Definition 2.3) for a finite probability model generalize naturally for countably infinite probability spaces. The only necessary change is to replace the word “finite” with the phrase “finite or countably infinite”. Together, finite probability models and their countably infinite generalizations are called discrete probability models.
A probability function on a finite or countably infinite set \(\Omega\) is a function \(p\colon \Omega\to
[0,1]\) that satisfies \(\sum_{\omega\in\Omega}p(\omega)=1\text{.}\) A discrete probability measure on a finite or countably infinite space \(\Omega\) is a function \(P\colon \mathcal{P}(\Omega)\to
[0,1]\) given by \(P(E)=\sum_{e\in E}p(e)\) for \(E\subseteq
\Omega\text{,}\) where \(p\) is a probability function on \(\Omega\text{.}\) A discrete probability model is a pair \((\Omega,P)\text{,}\) where \(\Omega\) is a finite or countably infinite set, and where \(P\) is a discrete probability measure on \(\Omega\text{.}\)
Show that the definition of the probability space \(\Omega^n\) (see Subsection 2.2) works for countably infinite descrete models \((\Omega,P)\text{.}\)
Samples from probability models. For a discrete probability model \((\Omega,P)\) with probability function \(p\text{,}\) we define the probability space of samples of size \(n\) from \(\Omega\) to be the space \(\Omega^n\) with probability function \(p_{\Omega^n}\) defined by (2.4).
For an uncountable set \(\Omega\text{,}\) it is impossible to have a probability function. That is, there does not exist any function \(p\colon \Omega \to [0,1]\) that satisfies \(\sum_{\omega\in \Omega}p(\omega)=1\text{.}\) Nevertheless, it is possible to define a probability measure in a way that satisfies all of the properties in Proposition 2.7 by restricting the set of events to a proper subset of the power set \(\mathcal{P}(\Omega)\text{.}\) In these notes, we will omit the technical details for how this is done. Instead we will illustrate with two examples.
Uniform probability measures. Let \(\Omega\) be the set of points inside of a geometric figure with a finite area or volume, say, a two-dimensional figure with area equal to \(A\) square units. Given any rectangle \(R\subset\Omega\text{,}\) define \(P(R)\) to be
\begin{equation}
P(R)=\frac{\text{area of }R}{A}.\tag{3.1}
\end{equation}
This probability measure is uniform in the sense that it doesn’t matter where \(R\) is located with \(\Omega\text{.}\) The probability of \(R\) is completely determined by its area. Similarly, the uniform probability measure on an interval \(\Omega=[a,b]\) of real numbers is given by \(P([c,d]) = \frac{d-c}{b-a}\) for any interval \([c,d]\subseteq \Omega\text{.}\)
Infinite sequences of coin tosses Let \(\Omega\) be the set of infinite sequences of tosses of a fair coin. For \((\omega_1,\omega_2,\omega_3,\ldots)\in \Omega\text{,}\) each entry \(\omega_k\) is one of the two outcomes in the probability space \(C=\{H,T\}\) (\(C\) is for “coin”, \(H\) is for “heads”, and \(T\) is for “tails”) with probability function \(p(H)=p(T)=1/2\text{.}\) Now suppose that \(E\subseteq
C^n\text{,}\) so that \(P_{C^n}(E)=|E|/2^n\) (see (2.4)). Let \(E'\) is the subset of \(\Omega\) defined by
In fact it turns out that such a probability measure exists. We will not give the full details for how it is constructed. Happily, it turns out that (3.2) give us what we need to work with the probability space \((\Omega,P_{\Omega})\text{.}\)
Explain why \(\Omega\) is uncountable. Explain why there can be no probability function on \(\Omega^\infty\text{.}\) Find the probability that the first head in an infinite sequence of tosses occurs on toss number 5.
Samples from uncountable probability models. For an uncountable probability model \((\Omega,P)\text{,}\) we define the probability space of samples of size \(n\) from \(\Omega\) to be the space \(\Omega^n\) with probability measure \(P_{\Omega^n}\) defined by
