2. Review of elementary probability

2. Review of elementary probability.
Let’s begin by recalling some of the definitions and basic concepts of elementary
probability. We will only work with discrete models at first.
We start with an arbitrary set, called the probability space, which we will denote
by Ω, the capital Greek letter “omega.” We are given a class F of subsets of Ω. These are
called events. We require F to be a σ-field.
Definition 2.1. A collection F of subsets of Ω is called a σ-field if
(1) ∅ ∈ F,
(2) Ω ∈ F,
(3) A ∈ F implies Ac ∈ F, and
(4) A1, A2, . . . ∈ F implies both ∪
∞
i=1Ai ∈ F and ∩
∞
i=1Ai ∈ F.
Here Ac = {ω ∈ Ω : ω /∈ A} denotes the complement of A. ∅ denotes the empty set, that
is, the set with no elements. We will use without special comment the usual notations of
∪ (union), ∩ (intersection), ⊂ (contained in), ∈ (is an element of).
Typically, in an elementary probability course, F will consist of all subsets of
Ω, but we will later need to distinguish between various σ-fields. Here is an example.
Suppose one tosses a coin two times and lets Ω denote all possible outcomes. So
Ω = {HH, HT, T H, T T}. A typical σ-field F would be the collection of all subsets of Ω.
In this case it is trivial to show that F is a σ-field, since every subset is in F. But if
we let G = {∅, Ω, {HH, HT}, {T H, T T}}, then G is also a σ-field. One has to check the
definition, but to illustrate, the event {HH, HT} is in G, so we require the complement of
that set to be in G as well. But the complement is {T H, T T} and that event is indeed in
G.
One point of view which we will explore much more fully later on is that the σ-field
tells you what events you “know.” In this example, F is the σ-field where you “know”
everything, while G is the σ-field where you “know” only the result of the first toss but not
the second. We won’t try to be precise here, but to try to add to the intuition, suppose
one knows whether an event in F has happened or not for a particular outcome. We
would then know which of the events {HH}, {HT}, {T H}, or {T T} has happened and so
would know what the two tosses of the coin showed. On the other hand, if we know which
events in G happened, we would only know whether the event {HH, HT} happened, which
means we would know that the first toss was a heads, or we would know whether the event
{T H, T T} happened, in which case we would know that the first toss was a tails. But
there is no way to tell what happened on the second toss from knowing which events in G
happened. Much more on this later.
The third basic ingredient is a probability.
4
Definition 2.2. A function P on F is a probability if it satisfies
(1) if A ∈ F, then 0 ≤ P(A) ≤ 1,
(2) P(Ω) = 1, and
(3) P(∅) = 0, and
(4) if A1, A2, . . . ∈ F are pairwise disjoint, then P(∪
∞
i=1Ai) = P∞
i=1 P(Ai).
A collection of sets Ai
is pairwise disjoint if Ai ∩ Aj = ∅ unless i = j.
There are a number of conclusions one can draw from this definition. As one
example, if A ⊂ B, then P(A) ≤ P(B) and P(Ac
) = 1 − P(A). See Note 1 at the end of
this section for a proof.
Someone who has had measure theory will realize that a σ-field is the same thing
as a σ-algebra and a probability is a measure of total mass one.
A random variable (abbreviated r.v.) is a function X from Ω to R, the reals. To
be more precise, to be a r.v. X must also be measurable, which means that {ω : X(ω) ≥
a} ∈ F for all reals a.
The notion of measurability has a simple definition but is a bit subtle. If we take
the point of view that we know all the events in G, then if Y is G-measurable, then we
know Y . Phrased another way, suppose we know whether or not the event has occurred
for each event in G. Then if Y is G-measurable, we can compute the value of Y .
Here is an example. In the example above where we tossed a coin two times, let X
be the number of heads in the two tosses. Then X is F measurable but not G measurable.
To see this, let us consider Aa = {ω ∈ Ω : X(ω) ≥ a}. This event will equal



Ω if a ≤ 0;
{HH, HT, T H} if 0 < a ≤ 1;
{HH} if 1 < a ≤ 2;
∅ if 2 < a.
For example, if a =
3
2
, then the event where the number of heads is 3
2
or greater is the
event where we had two heads, namely, {HH}. Now observe that for each a the event Aa
is in F because F contains all subsets of Ω. Therefore X is measurable with respect to F.
However it is not true that Aa is in G for every value of a – take a =
3
2
as just one example
– the subset {HH} is not in G. So X is not measurable with respect to the σ-field G.
A discrete r.v. is one where P(ω : X(ω) = a) = 0 for all but countably many a’s,
say, a1, a2, . . ., and P
i
P(ω : X(ω) = ai) = 1. In defining sets one usually omits the ω;
thus (X = x) means the same as {ω : X(ω) = x}.
In the discrete case, to check measurability with respect to a σ-field F, it is enough
that (X = a) ∈ F for all reals a. The reason for this is that if x1, x2, . . . are the values of