Definition 2.1.
A finite probability function is a function \(p\colon \Omega \to [0,1]\text{,}\) where \(\Omega\) is a finite set, and where \(p\) satisfies
\begin{equation}
\sum_{\omega\in\Omega} p(\omega)=1.\tag{2.1}
\end{equation}
Section 2 Finite Probability Models
Probability models are mathematical frameworks that help us think about chance processes whose outcomes cannot be predicted with certainty. We begin with one of the simplest types of probability models, in which the set of possible outcomes of a chance process is finite.
Subsection 2.1 Vocabulary and Properties
The next definition extends the assignments of probabilities to subsets of \(\Omega\text{.}\) Recall that the power set \(\mathcal{P}(S)\) of a set \(S\) is the set of all subsets of \(S\text{.}\)
Definition 2.2.
A finite probability measure is a function \(P\colon \mathcal{P}(\Omega)\to [0,1]\) be given by
\begin{equation}
P(E)=\sum_{\omega \in E} p(\omega)\tag{2.2}
\end{equation}
for \(E\subseteq \Omega\text{,}\) where \(p\colon \Omega\to [0,1]\) is a finite probability function.
Comment on notation: For a single element \(\omega\) in \(\Omega\text{,}\) it is common practice to write \(P(\omega)\) as a shorthand for \(P(\{\omega\})\text{.}\)
Definition 2.3. Finite probability model.
A finite probability model is a pair \((\Omega,P)\text{,}\) where \(\Omega\) is a finite set, and where \(P\) is a finite probability measure on \(\Omega\text{.}\) The set \(\Omega\) is called the probability space of the model. Elements of \(\Omega\) are called outcomes, and elements of \(\mathcal{P}(\Omega)\) are called events.
Checkpoint 2.4.
Write the probability function for the probability space \(\Omega=\{a,b,c\}\) where outcome \(b\) is twice as likely as outcome \(a\text{,}\) and outcome \(c\) is three times as likely as outcome \(b\text{.}\)
Definition 2.5. More probability terminology.
Let \((\Omega,P)\) be a finite probability model. The complement \(A^c=\Omega\setminus A\) of an event \(A\) is also called the opposite of \(A\text{.}\) Disjoint events \(A,B\) (that is, \(A\cap B=\emptyset)\) are also called mutually exclusive. Given events \(U,V\) with \(P(V)\neq 0\text{,}\) the conditional probability of \(U\) given \(V\), denoted \(P(U|V)\text{,}\) is defined to be
\begin{equation}
P(U|V)=\frac{P(U\cap V)}{P(V)}.\tag{2.3}
\end{equation}
Events \(U,V\) are called independent if \(P(U\cap
V)=P(U)P(V)\text{.}\) Otherwise, \(U,V\) are called dependent.
Checkpoint 2.6.
Let \(\Omega=\{a,b,c\}\text{,}\) let \(U=\{a,b\}\text{,}\) and let \(V=\{b,c\}\text{.}\) Give an example of a probability function on \(\Omega\) for which \(U,V\) are independent. Give an example of a probability function on \(\Omega\) for which \(U,V\) are dependent.
Proposition 2.7. Properties of probability.
Let \((\Omega,P)\) be a finite probability model. The following properties hold.
-
\(P(\Omega)=1\text{.}\)
-
If \(A\subseteq B\) for some events \(A,B\text{,}\) then \(P(A)\leq P(B)\text{.}\)
-
(addition rule) If events \(A,B\) are disjoint, then \(P(A\cup B)=P(A)+P(B)\text{.}\)
-
(opposite rule) For any event \(A\text{,}\) we have \(P(A^c)=1-P(A)\text{.}\)
-
(multiplication rule) For any events \(U,V\) such that \(P(U)\neq 0\text{,}\) we have \(P(U\cap V) = P(U)P(V|U)\text{.}\)
-
Let \(U,V\) be events with \(P(U)\neq 0\text{.}\) Events \(U,V\) are independent if and only if \(P(V)=P(V|U)\text{.}\)
Checkpoint 2.8.
Prove all the parts of Proposition 2.7.
Subsection 2.2 Random Samples
Let \((\Omega,P)\) be a finite probability model with probability function \(p\text{,}\) and let \(n\) be a positive integer. Let \(p_{\Omega^n}\colon
\Omega^n\to [0,1]\) be defined by
\begin{equation}
p_{\Omega^n}(\omega_1,\omega_2,\ldots,\omega_n)=p(\omega_1)p(\omega_2)\cdots
p(\omega_n).\tag{2.4}
\end{equation}
It is easy to check (see Checkpoint 2.9 below) that \(p_{\Omega^n}\) is a probability function. Let \(P_{\Omega^n}\) denote the corresponding probability measure. The probability model \((\Omega^n,P_{\Omega^n})\) is called the space of (random) samples of size \(n\) taken from the space \(\Omega\text{.}\) The space \(\Omega^n\) models the outcomes that are obtained by \(n\) repetitions of the chance process that produces outcomes in \(\Omega\text{.}\)
Checkpoint 2.9.
Show that \(p_{\Omega^n}\) is a probability function. Let \(E,F\) be events in \(\Omega\text{,}\) and let \(E',F'\) be the events \(E'=\{\vec{\omega}\colon\omega_j\in E\}\) and \(F'=\{\vec{\omega}\colon\omega_k\in F\}\text{.}\) Show that \(E',F'\) are independent in \(\Omega^n\) if \(j\neq k\text{.}\)
Subsection 2.3 Simple Random Samples
Let \((\Omega,P)\) be a finite probability space with \(N=|\Omega|\text{,}\) and with the constant probability function \(p(\omega)=\frac{1}{N}\) for all \(\omega\in \Omega\text{.}\) Let \(\Omega^{n\ast}\) denote the set of all one-to-one sequences in \(\Omega\) of size \(n\text{.}\) An element \((\omega_1,\omega_2,\ldots,\omega_n)\) of \(\Omega^{n\ast}\) is called a simple random sample of size \(n\) taken from a probability space \(\Omega\text{.}\) Let \(p_{\Omega^{n\ast}}\) be given by the constant function
1
We use the nonstandard notation \(\Omega^{n\ast}\text{,}\) rather than the standard notation \(P_n(\Omega)\text{,}\) to denote the set of permutations of \(n\) elements of \(\Omega\text{,}\) in order to avoid confusion with probability measures, which are also denoted using capital letter P.
\begin{equation}
p_{\Omega^{n\ast}}(\omega_1,\omega_2,\ldots,\omega_n)=\frac{1}{N(N-1)\cdots
(N-n+1)}\tag{2.5}
\end{equation}
for all \((\omega_1,\omega_2,\ldots,\omega_n)\in
\Omega^{n\ast}\text{.}\) It is easy to check (see Checkpoint 2.10 below) that \(p_{\Omega^n}\) is a probability function. Let \(P_{\Omega^{n\ast}}\) denote the corresponding probability measure. The probability model \((\Omega^n,P_{\Omega^{n\ast}})\) is called the space of simple random samples of size \(n\) taken from the space \(\Omega\) (or the space of samples of size \(n\) taken from \(\Omega\) without replacement).
Checkpoint 2.10.
Show that \(p_{\Omega^{n\ast}}\) is a probability function. Let \(E\) be an event in \(\Omega\text{,}\) and let \(E',F'\) be the events \(E'=\{\vec{\omega}\colon\omega_j\in E\}\text{,}\) \(F'=\{\vec{\omega}\colon\omega_k\in E\}\text{,}\) Show that \(E',F'\) are dependent in \(\Omega^{n\ast}\) if \(j\neq k\text{.}\)
Subsection 2.4 Bayes’ Rule
Bayes’ Rule can be viewed as a practical method for finding conditional probabilities. For events \(A,B\) in a probability space \(\Omega\text{,}\) where both probabilities \(P(A),P(B)\) are nonzero, the definition of conditional probability (2.3) gives us two ways to write \(P(A\cap B)\text{,}\) namely
\begin{equation*}
P(A\cap B)= P(A)P(B|A) = P(B)P(A|B).
\end{equation*}
Dividing the last two terms by \(P(A)\) leads to the following, which is a basic form of Bayes’ Rule.
\begin{equation}
P(B|A) =\frac{P(B)P(A|B)}{P(A)}\tag{2.6}
\end{equation}
A use case for this version of Bayes’ Rule is a problem in which you know the probabilities in the expression on the right, and you wish to find the probability on the left. In effect, this allows you to use \(P(A|B)\) to find \(P(B|A)\text{.}\)
In a more nuanced form of Bayes’ Rule, the event \(B\) is one of of a collection of events \(B_1,B_2,\ldots,
B_r\) that form a partition of \(\Omega\text{,}\) say, \(B=B_k\text{.}\) In this case, we have
\begin{align}
A \amp = \bigcup_{i=1}^r (A\cap B_i)\tag{2.7}\\
P(A) \amp = \sum_{i=1}^r P(A\cap B_i)\tag{2.8}\\
\amp = \sum_{i=1}^r P(B_i)P(A|B_i)\tag{2.9}
\end{align}
Substituting \(B=B_k\text{,}\) and substituting the last expression (2.9) for \(P(A)\) in (2.6), we have the following form of Bayes’ Rule.
\begin{equation}
P(B_k|A) = \frac{P(B_k)P(A|B_k)}{\sum_i P(B_i)P(A|B_i)}\tag{2.10}
\end{equation}
Checkpoint 2.11.
-
Justify all the steps in the derivation for Bayes’ Rule.
-
Find the sets \(A_i\) and \(B_j\) for medical testing scenario (what is the chance of having the disease given a positive test result?)
Exercises 2.5 Probability Exercises
1.
2.
3.
Work the “Probability Exercises” set posted on Canvas.
