Math 525: Probability Homeworks

Homework 1

Problem 1

Let $𝑛 \in ℕ$ .

Show that $2^{𝑛} = \sum_{𝑘 = 0}^{𝑛} (\binom{𝑛}{𝑘})$ . Given that a set of $𝑛$ elements has $2^{𝑛}$ subsets, what is the combinatorial interpretation of this equality?
Show that
$\sum_{𝑘 odd, 0 \leq 𝑘 \leq 𝑛} (\binom{𝑛}{𝑘}) = \sum_{𝑘 even, 0 \leq 𝑘 \leq 𝑛} (\binom{𝑛}{𝑘})$
Show that
$(\binom{2 𝑛}{𝑛}) = \sum_{𝑘 = 0}^{𝑛} {(\binom{𝑛}{𝑘})}^{2}$

Hint: You may use ${(\binom{𝑛}{𝑘})}^{2} = (\binom{𝑛}{𝑘}) (\binom{𝑛}{𝑛 - 𝑘})$ .

Proof

By the binomial theorem, we have:
$(1 + 1)^{𝑛} = \sum_{𝑘 = 0}^{𝑛} (\binom{𝑛}{𝑘}) 1^{𝑘} 1^{𝑛 - 𝑘} = \sum_{𝑘 = 0}^{𝑛} (\binom{𝑛}{𝑘})$
The combinatorial interpretation of this equality is that: Let $𝑆_{𝑘}$ be the collection of all subsets that have size $𝑘$ . For collection $𝑆_{𝑘}$ , its size is $(\binom{𝑛}{𝑘})$ since it represents choosing $𝑘$ elements from $𝑛$ elements without regard to order.
Therefore the total number of subsets of $𝑆$ is:
$| 𝒫︀ (𝑆) | = \sum_{𝑘 = 0}^{𝑛} | 𝑆_{𝑘} | = \sum_{𝑘 = 0}^{𝑛} (\binom{𝑛}{𝑘}) = 2^{𝑛}$
Using the binomial theorem, we have:
$0 = (1 - 1)^{𝑛} = \sum_{𝑘 = 0}^{𝑛} (\binom{𝑛}{𝑘}) (- 1)^{𝑘} 1^{𝑛 - 𝑘} = \sum_{𝑘 = 0}^{𝑛} (\binom{𝑛}{𝑘}) (- 1)^{𝑘}$
Thus:
$\begin{aligned} \sum_{𝑘 = 0}^{𝑛} (\binom{𝑛}{𝑘}) (- 1)^{𝑘} = 0 \\ ⟹ \sum_{0 \leq 𝑘 \leq 𝑛, 𝑘 odd} (\binom{𝑛}{𝑘}) (- 1) + \sum_{0 \leq 𝑘 \leq 𝑛, 𝑘 even} (\binom{𝑛}{𝑘}) (1) = 0 \\ ⟹ \sum_{0 \leq 𝑘 \leq 𝑛, 𝑘 odd} (\binom{𝑛}{𝑘}) = \sum_{0 \leq 𝑘 \leq 𝑛, 𝑘 even} (\binom{𝑛}{𝑘}) \end{aligned}$
We prove by combinatorial argument.
Let $𝑆$ be a setwith $2 𝑛$ distinct elements. The number of ways to choose a subset $𝑃$ containing $𝑛$ elements is $(\binom{2 𝑛}{𝑛})$ .
In another way: We can first arbitrarily divide the $2 𝑛$ distinct elements into two groups: group $𝐴$ and group $𝐵$ , each containing $𝑛$ elements:
$𝑆 = 𝐴 ⊔ 𝐵$
And fix the two groups.
For any subset $𝑃$ of the $2 𝑛$ elements with size $𝑛$ , some of them are from group $𝐴$ , and the rest of them are from group $𝐵$ .
Let $𝑘$ be the number of elements of $𝑃$ that are chosen from $𝐴$ , then the number of elements chosen from $𝐵$ must be $𝑛 - 𝑘$ .
Note the number of ways to choose $𝑘$ elements from $𝐴$ is $(\binom{𝑛}{𝑘})$ , and the number of ways to choose $𝑛 - 𝑘$ elements from $𝐵$ is $(\binom{𝑛}{𝑛 - 𝑘})$ .
Therefore, the total number of ways to get $𝑃$ from $𝑆 = 𝐴 ⊔ 𝐵$ with $𝑘$ elements from $𝐴$ is $(\binom{𝑛}{𝑘}) (\binom{𝑛}{𝑛 - 𝑘}) = {(\binom{𝑛}{𝑘})}^{2}$ .
Thus, summing over all possible values of $𝑘 = 0, 1, \dots, 𝑛$ , the number of ways to choose $𝑛$ elements from $𝑆$ i.e. the number of ways to get $𝑃$ from $𝑆$ , is
$\sum_{𝑘 = 0}^{𝑛} {(\binom{𝑛}{𝑘})}^{2}$
Thus, we obtain
$(\binom{2 𝑛}{𝑛}) = \sum_{𝑘 = 0}^{𝑛} {(\binom{𝑛}{𝑘})}^{2}$
as desired.

□

Problem 2

We roll a fair die three times and record the outcomes $𝑎, 𝑏, 𝑐 \in {1, 2, 3, 4, 5, 6}$ . What is the probability that the equation $𝑎 𝑥^{2} + 𝑏 𝑥 + 𝑐 = 0$ does not have solutions in the real numbers?

Solution

The equation $𝑎 𝑥^{2} + 𝑏 𝑥 + 𝑐 = 0$ does not have solutions in the real numbers iff the the discriminant is negative, i.e. $Δ = 𝑏^{2} - 4 𝑎 𝑐 < 0$ .
Total possible equations is $6^{3} = 216$ . For each $𝑏$ , the total possible $(𝑎, 𝑐)$ pairs are $36$ . We can calculate the number of $(𝑎, 𝑐)$ pairs that satisfy the condition case by case.

For $𝑏 = 1$ : $4 𝑎 𝑐 > 1$ holds for all $(𝑎, 𝑐)$ .
For $𝑏 = 2$ : $4 𝑎 𝑐 > 4 ⟹ 𝑎 𝑐 > 1$ , which excludes only $(1, 1)$ .
For $𝑏 = 3$ : $4 𝑎 𝑐 > 9 ⟹ 𝑎 𝑐 \geq 3$ since they are integers, so excluding $(1, 1), (1, 2), (2, 1)$ ( $3$ cases).
For $𝑏 = 4$ : $4 𝑎 𝑐 > 16 ⟹ 𝑎 𝑐 \geq 5$ , excluding: $(1, 3), (1, 4), (2, 2), (3, 1), (4, 1)$ besides the previous case, thus $8$ cases excluded.
For $𝑏 = 5$ : $4 𝑎 𝑐 > 25 ⟹ 𝑎 𝑐 \geq 7$ , excluding: $(1, 5), (1, 6), (2, 3), (3, 2), (5, 1), (6, 1)$ besides the previous case, thus $14$ cases excluded.
For $𝑏 = 6$ : $4 𝑎 𝑐 > 36 ⟹ 𝑎 𝑐 \geq 10$ , excluding: $(2, 4), (3, 3), (4, 2)$ besides the previous case, thus $17$ cases excluded.

Thus, the total number of triples for which the discriminant is not negative (exlcuded) is

1 + 3 + 8 + 14 + 17 = 43

Therefore, the desired probability is

1 - ℙ (the equation has solutions in the real numbers) = 1 - \frac{43}{216} = \frac{173}{216}

Problem 3

An ant starts at the origin $(0, 0)$ on the integer lattice. At each step it moves either one unit to the right or one unit upward, each with probability $\frac{1}{2}$ . The ant continues moving until it reaches the point $(205, 200)$ .
What is the probability that the ant visits the point $(105, 100)$ at some time during its journey?
Hint: Start by counting the number of paths from $(0, 0)$ to $(205, 200)$ .

Solution

Any path from $(0, 0)$ to $(205, 200)$ must consist of $205$ steps to the right and $200$ steps upward, for a total of $405$ steps. So a path is uniquely determined by the choice of 205 steps to the right (which is equivalent to the choice of 200 steps upward).
Thus total number of paths from $(0, 0)$ to $(205, 200)$ is

𝑁 = (\binom{405}{205})

A path passes through the point $(105, 100)$ if and only if it first goes from $(0, 0)$ to $(105, 100)$ and then from $(105, 100)$ to $(205, 200)$ .
Thus the number of such paths is the product of the number of paths from $(0, 0)$ to $(105, 100)$ and the number of paths from $(105, 100)$ to $(205, 200)$ , by the fundamental counting principle. For the same reason as deciding the number of total paths from $(0, 0)$ to $(205, 200)$ , the number of paths from $(0, 0)$ to $(105, 100)$ is

𝑁_{1} = (\binom{205}{105})

And similarly, the number of paths from $(105, 100)$ to $(205, 200)$ is

𝑁_{2} = (\binom{200}{100})

Note that from a point to another point, all such paths are equally likely to be chosen. Therefore, the desired probability is

ℙ (path passes through (105, 100)) = \frac{(\binom{205}{105}) (\binom{200}{100})}{(\binom{405}{205})}

Problem 4

From a lottery containing $𝑛$ tickets numbered $1, 2, \dots, 𝑛$ , a ticket is drawn, its number is recorded, and then it is returned to the lottery. This process is repeated $𝑘 \geq 3$ times. Find the probabilities of the following events:

Ticket 1 is selected at least once.
Tickets 1, 2, and 3 are each selected at least once.

Solution

Let $𝐸$ be the event that ticket $1$ is selected at least once. $$\begin{align*} \mathbb{P}(E) &= 1 - \mathbb{P}(\text{ticket $1$ is never selected in $k$ draws}) \\ &= 1 - \left(\frac{n-1}{n}\right)^k \tag*{\text{(by independence of each draw)}} \end{align*}$$
Let $𝐹$ be the event that tickets $1, 2, 3$ are each selected at least once.
For $𝑖 = 1, 2, 3$ , let
$𝐴_{𝑖} ≔ {ticket 𝑖 is never selected in the 𝑘 draws}$
Thus
$𝑃 (𝐹) = 1 - 𝑃 (𝐴_{1} \cup 𝐴_{2} \cup 𝐴_{3})$
By the principle of inclusion-exclusion,
$𝑃 (𝐴_{1} \cup 𝐴_{2} \cup 𝐴_{3}) = 𝑃 (𝐴_{1}) + 𝑃 (𝐴_{2}) + 𝑃 (𝐴_{3}) - 𝑃 (𝐴_{1} \cap 𝐴_{2}) - 𝑃 (𝐴_{1} \cap 𝐴_{3}) - 𝑃 (𝐴_{2} \cap 𝐴_{3}) + 𝑃 (𝐴_{1} \cap 𝐴_{2} \cap 𝐴_{3})$
Since similar to part (a), we have: $ℙ (𝐴_{𝑖}) = {(\frac{𝑛 - 1}{𝑛})}^{𝑘}$ , $ℙ (𝐴_{𝑖} \cap 𝐴_{𝑗}) = {(\frac{𝑛 - 2}{𝑛})}^{𝑘}$ , $ℙ (𝐴_{1} \cap 𝐴_{2} \cap 𝐴_{3}) = {(\frac{𝑛 - 3}{𝑛})}^{𝑘}$ , we then calculate:
$\begin{aligned} ℙ (𝐹) & = 1 - (\binom{3}{1}) {(\frac{𝑛 - 1}{𝑛})}^{𝑘} + (\binom{3}{2}) {(\frac{𝑛 - 2}{𝑛})}^{𝑘} - (\binom{3}{3}) {(\frac{𝑛 - 3}{𝑛})}^{𝑘} \\ = 1 - 3 {(\frac{𝑛 - 1}{𝑛})}^{𝑘} + 3 {(\frac{𝑛 - 2}{𝑛})}^{𝑘} - {(\frac{𝑛 - 3}{𝑛})}^{𝑘} \end{aligned}$

Problem 5

In a house, drawer $𝑆_{1}$ contains 3 gold coins and 3 silver coins, while drawer $𝑆_{2}$ contains 3 gold coins and 6 silver coins. A thief (in the dark) randomly opens one drawer and then randomly takes two coins from it.

What is the probability that both coins are gold?
If it is discovered (upon his arrest) that he has stolen two gold coins, what is the probability that he opened drawer $𝑆_{1}$ ?

Solution

The thief chooses a drawer uniformly at random, so for each pick, $ℙ (𝑆_{1} is chosen) = ℙ (𝑆_{2} is chosen) = \frac{1}{2}$ . Given a drawer, he draws two coins without replacement.

Using the law of total probability,
$\begin{aligned} ℙ (two gold) & = ℙ (two gold ∣ 𝑆_{1} is chosen) ℙ (drawer 𝑆_{1}) + ℙ (two gold ∣ 𝑆_{2} is chosen) ℙ (drawer 𝑆_{2}) \\ = \frac{1}{2} \cdot \frac{(\binom{3}{2})}{(\binom{6}{2})} + \frac{1}{2} \cdot \frac{(\binom{3}{2})}{(\binom{9}{2})} \\ = \frac{1}{2} (\frac{3}{15} + \frac{3}{36}) \\ = \frac{36 + 15}{360} \\ = \frac{17}{120} \end{aligned}$
Let $𝐺$ be the event that the thief stole two gold coins. By Bayes’ rule,
$ℙ (𝑆_{1} ∣ 𝐺) = \frac{ℙ (𝐺 ∣ 𝑆_{1}) ℙ (𝑆_{1})}{ℙ (𝐺)}$
Since we have $ℙ (𝐺 ∣ 𝑆_{1}) = \frac{(\binom{3}{2})}{(\binom{6}{2})} = \frac{1}{5}$ , $ℙ (𝑆_{1}) = \frac{1}{2}$ , and $ℙ (𝐺) = \frac{17}{120}$ from part (a), we get:
$ℙ (𝑆_{1} ∣ 𝐺) = \frac{\frac{1}{5} \cdot \frac{1}{2}}{\frac{17}{120}} = \frac{12}{17}$

Problem 6

Let $𝐴$ and $𝐵$ be events of a probability space with $ℙ (𝐴) > 0$ . Show that:

$ℙ (𝐴 \cup 𝐵) > 0$ and $ℙ (𝐴 \cap 𝐵 ∣ 𝐴 \cup 𝐵) \leq ℙ (𝐴 \cap 𝐵 ∣ 𝐴)$ .
$ℙ (𝐵 ∣ 𝐵 \cup 𝐴) \geq ℙ (𝐵 ∣ 𝐴)$ .

Proof

Since $𝐴 \subseteq 𝐴 \cup 𝐵$ , we have by monotonicity of probability measure:
$ℙ (𝐴 \cup 𝐵) \geq ℙ (𝐴) > 0$
Also, since $𝐴 \cap 𝐵 \subseteq 𝐴$ and $ℙ (𝐴 \cup 𝐵) \geq ℙ (𝐴) > 0$ , both conditional probabilities below are well-defined. Then
$\begin{aligned} ℙ (𝐴 \cap 𝐵 ∣ 𝐴 \cup 𝐵) & = \frac{ℙ ((𝐴 \cap 𝐵) \cap (𝐴 \cup 𝐵))}{ℙ (𝐴 \cup 𝐵)} \\ = \frac{ℙ (𝐴 \cap 𝐵)}{ℙ (𝐴 \cup 𝐵)} \\ \leq \frac{ℙ (𝐴 \cap 𝐵)}{ℙ (𝐴)} \\ = ℙ (𝐴 \cap 𝐵 ∣ 𝐴) \end{aligned}$
This finishes the proof.
Let $𝑥 ≔ ℙ (𝐴 \cap 𝐵)$ , $𝑦 ≔ ℙ (𝐴 \ 𝐵)$ , $𝑧 ≔ ℙ (𝐵 \ 𝐴)$ .
so $𝑥, 𝑦, 𝑧 \geq 0$ by non-negativity of probability measure.
And since
$𝐴 = (𝐴 \cap 𝐵) ⊔ (𝐴 \ 𝐵)$
Thus, we have:
$ℙ (𝐴) = ℙ (𝐴 \cap 𝐵) + ℙ (𝐴 \ 𝐵) = 𝑥 + 𝑦$
By similar reason, we have:
$ℙ (𝐴 \cup 𝐵) = 𝑥 + 𝑦 + 𝑧, ℙ (𝐵) = 𝑥 + 𝑧$
Thus we have:
$ℙ (𝐵 ∣ 𝐴 \cup 𝐵) = \frac{ℙ (𝐵 \cap (𝐴 \cup 𝐵))}{ℙ (𝐴 \cup 𝐵)} = \frac{ℙ (𝐵)}{ℙ (𝐴 \cup 𝐵)} = \frac{𝑥 + 𝑧}{𝑥 + 𝑦 + 𝑧}$
and
$ℙ (𝐵 ∣ 𝐴) = \frac{ℙ (𝐴 \cap 𝐵)}{ℙ (𝐴)} = \frac{𝑥}{𝑥 + 𝑦}$
Note the two probabilities are well-defined since $𝑥 + 𝑦 = ℙ (𝐴) > 0$ (and so $𝑥 + 𝑦 + 𝑧 > 0$ ).
Now it remains to show that:
$\frac{𝑥 + 𝑧}{𝑥 + 𝑦 + 𝑧} \geq \frac{𝑥}{𝑥 + 𝑦}$
i.e.
$(𝑥 + 𝑧) (𝑥 + 𝑦) \geq 𝑥 (𝑥 + 𝑦 + 𝑧)$
which is equivalent to:
$𝑥 (𝑥 + 𝑦) + 𝑧 (𝑥 + 𝑦) \geq 𝑥 (𝑥 + 𝑦) + 𝑥 𝑧$
Eliminating common terms, this is equivalent to:
$𝑧 𝑦 \geq 0$
which is true by non-negativity of $𝑧$ and $𝑦$ . This finishes the proof that:
$ℙ (𝐵 ∣ 𝐴 \cup 𝐵) \geq ℙ (𝐵 ∣ 𝐴)$

□

Homework 2

Problem 1

Suppose that the cumulative distribution function (CDF) of a random variable $𝐹 : ℝ \to ℝ$ is strictly increasing and continuous. Let $𝑈$ be a random variable with the uniform distribution on $(0, 1)$ and define

𝑋 ≔ 𝐹^{- 1} (𝑈)

Show that $𝑋$ has CDF equal to $𝐹$ . This exercise shows us how to construct a random variable with given distribution, assuming that we have a uniform random variable.

Proof

Since $𝐹$ is strictly increasing and continuous, it has an inverse function $𝐹^{- 1}$ on its range, and $𝐹^{- 1}$ is also strictly increasing. Thus for any $𝑥, 𝑦 \in ℝ$ ,

𝐹^{- 1} (𝑦) \leq 𝑥 \Leftrightarrow 𝑦 \leq (𝐹^{- 1})^{- 1} (𝑥) = 𝐹 (𝑥)

Therefore for any $𝑥 \in ℝ$ , we have

{𝑥 ∣ 𝑋 (𝑥) \leq 𝑥} = {𝑥 ∣ 𝐹^{- 1} (𝑈 (𝑥)) \leq 𝑥} = {𝑥 ∣ 𝑈 (𝑥) \leq 𝐹 (𝑥)}

Therefore

ℙ (𝑋 \leq 𝑥) = ℙ (𝑈 (𝑥) \leq 𝐹 (𝑥))

Since $𝑈 \sim U n i f (0, 1)$ and for a CDF we have $𝐹 (𝑥) \in [0, 1]$ , we get

ℙ (𝑈 (𝑥) \leq 𝐹 (𝑥)) = 𝐹 (𝑥)

Thus for all $𝑥$ $ℙ (𝑋 \leq 𝑥) = 𝐹 (𝑥)$ , i.e., the CDF of $𝑋$ equals $𝐹$ .

□

Problem 2

A gas station fills its tank completely once a week. Let the weekly sales volume (in thousands of liters) be a random variable with density

𝑓 (𝑥) = {\begin{cases} 𝑎 (1 - 𝑥)^{4}, & 𝑥 \in (0, 1), \\ 0, & otherwise \end{cases}

Find the constant $𝑎$ . What should be the tank capacity so that the probability of running out of fuel during a given week is $1 / 100$ ?

Solution

Since the density integrates to 1,

1 = \int_{- \infty}^{\infty} 𝑓 (𝑥) 𝑑 𝑥 = \int_{0}^{1} 𝑎 (1 - 𝑥)^{4} 𝑑 𝑥 = 𝑎 \int_{0}^{1} (1 - 𝑥)^{4} 𝑑 𝑥

Let $𝑢 = 1 - 𝑥$ , then

\int_{0}^{1} (1 - 𝑥)^{4} 𝑑 𝑥 = \int_{0}^{1} 𝑢^{4} 𝑑 𝑢 = \frac{1}{5}

So $\frac{1}{5} 𝑎 = 1$ , which gives

𝑎 = 5

Now we look for the tank capacity $𝑐$ such that $ℙ (𝑋 > 𝑐) = \frac{1}{100}$ .
Let the tank capacity be $𝑐$ (in thousands of liters). Running out of fuel in a week occurs when sales exceed $𝑐$ , i.e., the event ${𝑋 > 𝑐}$ . We need

ℙ (𝑋 > 𝑐) = \frac{1}{100}

Since $𝑎 = 5$ ,

ℙ (𝑋 > 𝑐) = \int_{𝑐}^{1} 5 (1 - 𝑥)^{4} 𝑑 𝑥

Again let $𝑢 = 1 - 𝑥$ , then we have

\int_{𝑐}^{1} 5 (1 - 𝑥)^{4} 𝑑 𝑥 = 5 \int_{1 - 𝑐}^{0} 𝑢^{4} (- 𝑑 𝑢) = 5 \int_{0}^{1 - 𝑐} 𝑢^{4} 𝑑 𝑢 = 5 \cdot \frac{(1 - 𝑐)^{5}}{5} = (1 - 𝑐)^{5}

Therefore

(1 - 𝑐)^{5} = \frac{1}{100} ⟹ 1 - 𝑐 = 100^{- 1 / 5} = 10^{- 2 / 5} ⟹ 𝑐 = 1 - 10^{- 2 / 5}

So the tank capacity should be $1 - 10^{- 2 / 5}$ thousand liters.

Problem 3

Let the random variable $𝑋$ have density

𝑓_{𝑋} (𝑥) = {\begin{cases} \frac{1}{2 𝑥^{2}}, & | 𝑥 | \geq 1, \\ 0, & | 𝑥 | < 1 . \end{cases}

Find the probability density function of $𝑌 ≔ 𝑋^{2}$ and compute the probability $ℙ (2 𝑌 + 3 \leq 10)$ .

Solution

Since $𝑓_{𝑋} (𝑥) = 0$ for $| 𝑥 | < 1$ , we have $ℙ (| 𝑋 | \geq 1) = 1$ . Hence $𝑌 = 𝑋^{2} \geq 1$ almost surely, so $𝐹_{𝑌} (𝑦) = 0$ for $𝑦 < 1$ and therefore $𝑓_{𝑌} (𝑦) = 0$ for $𝑦 < 1$ (a.e.).

For $𝑦 \geq 1$ ,

𝐹_{𝑌} (𝑦) = ℙ (𝑋^{2} \leq 𝑦) = ℙ (- \sqrt{𝑦} \leq 𝑋 \leq \sqrt{𝑦}) = \int_{- \sqrt{𝑦}}^{- 1} \frac{1}{2 𝑥^{2}} 𝑑 𝑥 + \int_{1}^{\sqrt{𝑦}} \frac{1}{2 𝑥^{2}} 𝑑 𝑥

Compute each integral:

\int_{1}^{\sqrt{𝑦}} \frac{1}{2 𝑥^{2}} 𝑑 𝑥 = \frac{1}{2} \int_{1}^{\sqrt{𝑦}} 𝑥^{- 2} 𝑑 𝑥 = \frac{1}{2} - 𝑥^{- 1} 1_{\sqrt{𝑦}} = \frac{1}{2} (1 - \frac{1}{\sqrt{𝑦}})

and similarly $\int_{- \sqrt{𝑦}}^{- 1} \frac{1}{2 𝑥^{2}} 𝑑 𝑥 = \frac{1}{2} (1 - \frac{1}{\sqrt{𝑦}})$ since the function is even. Therefore, for $𝑦 \geq 1$ ,

𝐹_{𝑌} (𝑦) = 1 - \frac{1}{\sqrt{𝑦}}

Combining both cases we have

𝐹_{𝑌} (𝑦) = {\begin{cases} 0, & 𝑦 < 1, \\ 1 - \frac{1}{\sqrt{𝑦}} & 𝑦 \geq 1 \end{cases}

Notice that on $𝑦 \geq 1$ , $𝐹_{𝑌} (𝑦)$ is differentiable (except on $𝑦 = 1$ ):

𝐹_{𝑌}^{'} (𝑦) = \frac{𝑑}{𝑑 𝑦} (1 - 𝑦^{- 1 / 2}) = \frac{1}{2} 𝑦^{- 3 / 2}

So consider the function

𝑔 (𝑦) = {\begin{cases} \frac{1}{2 𝑦^{3 / 2}}, & 𝑦 \geq 1, \\ 0, & 𝑦 < 1 . \end{cases}

Then for $𝑥 < 1$ ,

\int_{- \infty}^{𝑥} 𝑔 (𝑦) 𝑑 𝑦 = 0 = 𝐹_{𝑌} (𝑥)

and for $𝑥 \geq 1$

\int_{- \infty}^{𝑥} 𝑔 (𝑦) 𝑑 𝑦 = \int_{1}^{𝑥} \frac{1}{2 𝑦^{3 / 2}} 𝑑 𝑦 = {[- 𝑦^{- 1 / 2}]}_{1}^{𝑥} = 1 - \frac{1}{\sqrt{𝑥}} = 𝐹_{𝑌} (𝑥)

This shows that $𝑌$ is absolutely continuous and $𝑔$ is a probability density of $𝑌$ . Hence

𝑓_{𝑌} (𝑦) = {\begin{cases} \frac{1}{2 𝑦^{3 / 2}}, & 𝑦 \geq 1, \\ 0, & 𝑦 < 1 \end{cases}

Now we compute $ℙ (2 𝑌 + 3 \leq 10)$ .

We have $2 𝑌 + 3 \leq 10 \Leftrightarrow 𝑌 \leq \frac{7}{2}$ . Thus

ℙ (2 𝑌 + 3 \leq 10) = ℙ (𝑌 \leq \frac{7}{2}) = 𝐹_{𝑌} (\frac{7}{2}) = 1 - \frac{1}{\sqrt{7 / 2}} = 1 - \sqrt{\frac{2}{7}}

Thus,

ℙ (2 𝑌 + 3 \leq 10) = 1 - \sqrt{\frac{2}{7}}

Problem 4

Let the random variable $𝑋$ have density $𝑓$ , which is symmetric about $𝜇 \in ℝ$ , that is, $𝑓 (𝜇 + 𝑥) = 𝑓 (𝜇 - 𝑥)$ , for all $𝑥 \in ℝ$ . Show that $ℙ (𝑋 \leq 𝜇) = ℙ (𝑋 \geq 𝜇)$ . If in addition $𝔼 | 𝑋 | < \infty$ , show that $𝔼 (𝑋) = 𝜇$ . Can you use this observation if $𝑋 \sim 𝑁 (0, 1)$ ?

Proof

Since $𝑋$ has density $𝑓$ ,

ℙ (𝑋 \leq 𝜇) = \int_{- \infty}^{𝜇} 𝑓 (𝑡) 𝑑 𝑡

Let $𝑡 = 𝜇 - 𝑥$ so that $𝑑 𝑡 = - 𝑑 𝑥$ . Then

\int_{- \infty}^{𝜇} 𝑓 (𝑡) 𝑑 𝑡 = \int_{\infty}^{0} 𝑓 (𝜇 - 𝑥) (- 𝑑 𝑥) = \int_{0}^{\infty} 𝑓 (𝜇 - 𝑥) 𝑑 𝑥

Similarly,

ℙ (𝑋 \geq 𝜇) = \int_{𝜇}^{\infty} 𝑓 (𝑡) 𝑑 𝑡 = \int_{0}^{\infty} 𝑓 (𝜇 + 𝑥) 𝑑 𝑥

By symmetry $𝑓 (𝜇 - 𝑥) = 𝑓 (𝜇 + 𝑥)$ for all $𝑥$ , hence the two integrals are equal, i.e. proved

ℙ (𝑋 \leq 𝜇) = ℙ (𝑋 \geq 𝜇)

If $𝔼 | 𝑋 | < \infty$ , then $𝔼 [𝑋] = 𝜇$ for some $𝜇 \in ℝ$ , We want to show that this $𝜇$ is the same as the one in the symmetry condition. Consider $𝔼 [𝑋 - 𝜇]$ . Since $𝔼 | 𝑋 | < \infty$ , we also have $𝔼 | 𝑋 - 𝜇 | < \infty$ , so the following integral is well-defined:

𝔼 [𝑋 - 𝜇] = \int_{- \infty}^{\infty} (𝑡 - 𝜇) 𝑓 (𝑡) 𝑑 𝑡

Let $𝑡 = 𝜇 + 𝑥$ ; then

𝔼 [𝑋 - 𝜇] = \int_{- \infty}^{\infty} 𝑥 𝑓 (𝜇 + 𝑥) 𝑑 𝑥

Define $𝑔 (𝑥) ≔ 𝑓 (𝜇 + 𝑥)$ . The symmetry condition $𝑓 (𝜇 + 𝑥) = 𝑓 (𝜇 - 𝑥)$ implies that $𝑔$ is an even function, thus $𝑥 𝑔 (𝑥)$ is an odd function. Since $\int | 𝑥 | 𝑔 (𝑥) 𝑑 𝑥 < \infty$ , we may integrate over symmetric limits to get

\int_{- \infty}^{\infty} 𝑥 𝑔 (𝑥) 𝑑 𝑥 = 0

Therefore $𝔼 [𝑋 - 𝜇] = 0$ , thus

𝔼 [𝑋] = 𝜇

Application to $𝑋 \sim 𝑁 (0, 1)$ : Since the standard normal density $𝜑 (𝑥) = \frac{1}{\sqrt{2 𝜋}} 𝑒^{- 𝑥^{2} / 2}$ satisfies $𝜑 (0 + 𝑥) = 𝜑 (0 - 𝑥)$ , so it is symmetric about $𝜇 = 0$ . Hence

ℙ (𝑋 \leq 0) = ℙ (𝑋 \geq 0) = \frac{1}{2} and 𝔼 [𝑋] = 0

□

Problem 5

An airline has observed that $5 %$ of ticket holders do not show up for their flight. Today’s flight has an airplane with 200 seats, and the airline has sold 203 tickets. What is the probability that the airline will not be able to accommodate a ticketed passenger? Assume that, for each passenger $𝑖$ , the event $𝐴_{𝑖}$ that passenger $𝑖$ shows up is independent of all others, for $1 \leq 𝑖 \leq 203$ .

Solution

Let $𝑆$ be the number of passengers who show up. The condition indicates that $𝑆$ is a binomial random variable with parameters $𝑛 = 203$ and $𝑝 = 0.95$ :

𝑆 \sim B i n o m i a l (𝑛 = 203, 𝑝 = 0.95)

The airline cannot accommodate everyone exactly when more than 200 passengers show up, i.e.

ℙ (cannot accommodate) = ℙ (𝑆 \geq 201) = \sum_{𝑘 = 201}^{203} (\binom{203}{𝑘}) (0.95)^{𝑘} (0.05)^{203 - 𝑘}

Equivalently, letting $𝑁 ≔ 203 - 𝑆$ be the number of no-shows, we have $𝑁 \sim B i n o m i a l (203, 0.05)$ and

ℙ (𝑆 \geq 201) = ℙ (𝑁 \leq 2) = \sum_{𝑗 = 0}^{2} (\binom{203}{𝑗}) (0.05)^{𝑗} (0.95)^{203 - 𝑗}

Numerically we can calculate

ℙ (cannot accommodate) \approx 0.206 %

Problem 6

Consider a sequence of tosses of a fair die. We continue tossing until both outcomes 3 and 4 have appeared at least once. For example, one possible sequence of results is

5, 1, 1, 4, 6, 5, 4, 2, 6, 3,

and we then stop. Let $𝑋$ be the number of tosses required (in this example, $𝑋 = 10$ ). What is the expected value of the random variable $𝑋$ ?

Solution

We can decompose the waiting time into two stages.

Stage 1: wait until the first time we see either 3 or 4: On each toss, the probability to get a 3 or 4 is $2 / 6 = 1 / 3$ . Hence the number of tosses $𝑇_{1}$ until the first occurrence of ${3, 4}$ is geometric with success probability $1 / 3$ , so

𝔼 [𝑇_{1}] = \frac{1}{1 / 3} = 3

Stage 2: after seeing one of them, wait until we see the other: Once 3 has appeared, each subsequent toss produces a 4 with probability $1 / 6$ ; otherwise we are still missing a 4. Thus the additional waiting time $𝑇_{2}$ is geometric with success probability $1 / 6$ , so

𝔼 [𝑇_{2}] = \frac{1}{1 / 6} = 6

Since $𝑋 = 𝑇_{1} + 𝑇_{2}$ , by linearity of expectation we get

𝔼 [𝑋] = 𝔼 [𝑇_{1}] + 𝔼 [𝑇_{2}] = 3 + 6 = 9

Homework 3

Problem 1

Let $𝑍$ be a standard normal random variable $𝑍 \sim 𝑁 (0, 1)$ . We denote by $Φ$ its distribution function. Answer the questions below

If $𝑎, 𝑏 \in ℝ$ with $𝑎 > 0$ , show that the random variable $𝑎 𝑍 + 𝑏$ is also normal and find its mean and variance.
Show that $Φ (0) = 1 / 2$ .
Show that $Φ (- 𝑥) = 1 - Φ (𝑥)$ for any $𝑥 \in ℝ$ .

Solution

$𝑍 \sim 𝑁 (0, 1)$ has density
$𝑓_{𝑍} (𝑧) = \frac{1}{\sqrt{2 𝜋}} 𝑒^{- 𝑧^{2} / 2}$ $\begin{aligned} 𝐹_{𝑋} (𝑥) = ℙ (𝑋 \leq 𝑥) & = ℙ (𝑎 𝑍 + 𝑏 \leq 𝑥) \\ = ℙ (𝑍 \leq \frac{𝑥 - 𝑏}{𝑎}) \\ = Φ (\frac{𝑥 - 𝑏}{𝑎}) \end{aligned}$
Thus
$𝑓_{𝑋} (𝑥) = \frac{𝑑}{𝑑 𝑥} Φ (\frac{𝑥 - 𝑏}{𝑎}) = \frac{1}{𝑎} 𝜑 (\frac{𝑥 - 𝑏}{𝑎}) = \frac{1}{\sqrt{𝑎^{22} 𝜋}} 𝑒^{- \frac{(𝑥 - 𝑏)^{2}}{2 𝑎^{2}}}$
Note this is the density of a normal distribution with mean $𝑏$ and variance $𝑎^{2}$ . Therefore
$𝑎 𝑍 + 𝑏 \sim 𝑁 (𝑏, 𝑎^{2})$
Since $𝑍$ has mean $0$ and variance $1$ , use linearity we have
$𝔼 [𝑎 𝑍 + 𝑏] = 𝑎 𝔼 [𝑍] + 𝑏 = 𝑏$
and
$V a r (𝑎 𝑍 + 𝑏) = 𝑎^{2} V a r (𝑍) = 𝑎^{2}$
Note the standard normal density is an even function:
$𝜑 (𝑥) = \frac{1}{\sqrt{2 𝜋}} 𝑒^{- 𝑥^{2} / 2} = 𝜑 (- 𝑥)$
Thus
$Φ (0) = \int_{- \infty}^{0} 𝜑 (𝑥) 𝑑 𝑥 = \int_{0}^{\infty} 𝜑 (𝑥) 𝑑 𝑥$
Since $\int_{- \infty}^{\infty} 𝜑 (𝑥) 𝑑 𝑥 = 1$ , the two equal halves are each $1 / 2$ , so $Φ (0) = 1 / 2$ .
For any $𝑥 \in ℝ$ ,
$Φ (- 𝑥) = \int_{- \infty}^{- 𝑥} 𝜑 (𝑡) 𝑑 𝑡$
Let $𝑢 = - 𝑡$ , using $𝜑 (- 𝑢) = 𝜑 (𝑢)$ we have
$Φ (- 𝑥) = \int_{\infty}^{𝑥} 𝜑 (- 𝑢) (- 𝑑 𝑢) = \int_{𝑥}^{\infty} 𝜑 (𝑢) 𝑑 𝑢 = 1 - \int_{- \infty}^{𝑥} 𝜑 (𝑢) 𝑑 𝑢 = 1 - Φ (𝑥)$

Problem 2

Let $𝑋$ and $𝑌$ be random variables with joint density

𝑓 (𝑥, 𝑦) = {\begin{cases} - 𝑥 𝑦, & (𝑥, 𝑦) \in (- 1, 0) \times (0, 1) \cup (1, 2) \times (- 1, 0), \\ 0, & otherwise \end{cases}

Compute the probability $ℙ (𝑋 + 𝑌 < 0)$ .
Compute the expected value $𝔼 [𝑋 𝑌]$ .
Are $𝑋$ and $𝑌$ independent?

Solution

On $(1, 2) \times (- 1, 0)$ we have $𝑥 + 𝑦 > 0$ since $𝑥 > 1$ and $𝑦 > - 1$ , hence this region contributes nothing to ${𝑋 + 𝑌 < 0}$ .
On $(- 1, 0) \times (0, 1)$ , the ineq $𝑥 + 𝑦 < 0$ is equivalent to $0 < 𝑦 < - 𝑥$ . Therefore,
$ℙ (𝑋 + 𝑌 < 0) = \int_{- 1}^{0} \int_{0}^{- 𝑥} (- 𝑥 𝑦) 𝑑 𝑦 𝑑 𝑥$
Compute the inner integral:
$\int_{0}^{- 𝑥} (- 𝑥 𝑦) 𝑑 𝑦 = - 𝑥 \cdot \frac{(- 𝑥)^{2}}{2} = - \frac{𝑥^{3}}{2}$
Hence,
$ℙ (𝑋 + 𝑌 < 0) = \int_{- 1}^{0} (- \frac{𝑥^{3}}{2}) 𝑑 𝑥 = - \frac{1}{2} \cdot \frac{𝑥^{4}}{4} {- 1}_{0} = \frac{1}{8}$
By def,
$\begin{aligned} 𝔼 [𝑋 𝑌] & = \int_{ℝ^{2}} 𝑥 𝑦 𝑓 (𝑥, 𝑦) 𝑑 𝑥 𝑑 𝑦 \\ = - \int_{(- 1, 0) \times (0, 1) \cup (1, 2) \times (- 1, 0)} 𝑥^{2} 𝑦^{2} 𝑑 𝑥 𝑑 𝑦 \\ = - \int_{- 1}^{0} \int_{0}^{1} 𝑥^{2} 𝑦^{2} 𝑑 𝑦 𝑑 𝑥 - \int_{1}^{2} \int_{- 1}^{0} 𝑥^{2} 𝑦^{2} 𝑑 𝑦 𝑑 𝑥 \end{aligned}$
Split over the two rectangles. On $(- 1, 0) \times (0, 1)$ ,
$- \int_{- 1}^{0} \int_{0}^{1} 𝑥^{2} 𝑦^{2} 𝑑 𝑦 𝑑 𝑥 = - (\int_{- 1}^{0} 𝑥^{2} 𝑑 𝑥) (\int_{0}^{1} 𝑦^{2} 𝑑 𝑦) = - (\frac{1}{3}) (\frac{1}{3}) = - \frac{1}{9}$
On $(1, 2) \times (- 1, 0)$ ,
$- \int_{1}^{2} \int_{- 1}^{0} 𝑥^{2} 𝑦^{2} 𝑑 𝑦 𝑑 𝑥 = - (\int_{1}^{2} 𝑥^{2} 𝑑 𝑥) (\int_{- 1}^{0} 𝑦^{2} 𝑑 𝑦) = - (\frac{7}{3}) (\frac{1}{3}) = - \frac{7}{9}$
Thus,
$𝔼 [𝑋 𝑌] = - \frac{1}{9} - \frac{7}{9} = - \frac{8}{9}$
Consider: For $𝑥 \in (- 1, 0)$ ,
$𝑓_{𝑋} (𝑥) = \int_{0}^{1} (- 𝑥 𝑦) 𝑑 𝑦 = \frac{- 𝑥}{2}$
For $𝑦 \in (0, 1)$ ,
$𝑓_{𝑌} (𝑦) = \int_{- 1}^{0} (- 𝑥 𝑦) 𝑑 𝑥 = 𝑦 \int_{- 1}^{0} (- 𝑥) 𝑑 𝑥 = \frac{𝑦}{2}$
And for $(𝑥, 𝑦) \in (- 1, 0) \times (0, 1)$ ,
$𝑓_{𝑋} (𝑥) 𝑓_{𝑌} (𝑦) = (\frac{- 𝑥}{2}) (\frac{𝑦}{2}) = \frac{- 𝑥 𝑦}{4} \neq - 𝑥 𝑦 = 𝑓 (𝑥, 𝑦)$
Thus $𝑋$ and $𝑌$ are not independent.

Problem 3

Let $𝑋 \sim Exp (1)$ and $𝑌 = 𝑋 + \frac{1}{𝑋 + 1}$ . Find $ℙ ((𝑋 + 1) 𝑌 \leq 2)$ and $Cov (𝑋, 𝑌)$ .
Hint: You may leave your answer as a function of the integral $\int_{0}^{\infty} \frac{𝑒^{- 𝑥}}{1 + 𝑥} 𝑑 𝑥$ .

Solution

Note

(𝑋 + 1) 𝑌 = (𝑋 + 1) (𝑋 + \frac{1}{𝑋 + 1}) = 𝑋 (𝑋 + 1) + 1 = 𝑋^{2} + 𝑋 + 1

Thus,

(𝑋 + 1) 𝑌 \leq 2 \Leftrightarrow 𝑋^{2} + 𝑋 - 1 \leq 0

The roots of $𝑥^{2} + 𝑥 - 1 = 0$ are $\frac{- 1 \pm \sqrt{5}}{2}$ . Since $𝑋 \geq 0$ , the event is

0 \leq 𝑋 \leq \frac{\sqrt{5} - 1}{2}

Therefore, using the CDF of $E x p (1)$ ,

ℙ ((𝑋 + 1) 𝑌 \leq 2) = ℙ (𝑋 \leq \frac{\sqrt{5} - 1}{2}) = 1 - 𝑒^{- \frac{\sqrt{5} - 1}{2}} = 1 - exp (- \frac{\sqrt{5} - 1}{2})

Nowe we compute the covariance. By def,

C o v (𝑋, 𝑌) = 𝔼 [𝑋 𝑌] - 𝔼 [𝑋] 𝔼 [𝑌]

For $𝑋 \sim E x p (1)$ , $𝔼 [𝑋] = 1$ and $𝔼 [𝑋^{2}] = 2$ . Let

𝐼 ≔ \int_{0}^{\infty} \frac{𝑒^{- 𝑥}}{1 + 𝑥} 𝑑 𝑥 = 𝔼 [\frac{1}{1 + 𝑋}]

Then

𝔼 [𝑌] = 𝔼 [𝑋] + 𝔼 [\frac{1}{1 + 𝑋}] = 1 + 𝐼

Also,

𝑋 𝑌 = 𝑋 (𝑋 + \frac{1}{1 + 𝑋}) = 𝑋^{2} + \frac{𝑋}{1 + 𝑋} = 𝑋^{2} + (1 - \frac{1}{1 + 𝑋})

𝔼 [𝑋 𝑌] = 𝔼 [𝑋^{2}] + 1 - 𝔼 [\frac{1}{1 + 𝑋}] = 2 + 1 - 𝐼 = 3 - 𝐼

Hence,

C o v (𝑋, 𝑌) = (3 - 𝐼) - (1) (1 + 𝐼) = 2 - 2 𝐼 = 2 - 2 \int_{0}^{\infty} \frac{𝑒^{- 𝑥}}{1 + 𝑥} 𝑑 𝑥

Problem 4

Find the conditional density $𝑓_{𝑌 ∣ 𝑋} (𝑦 ∣ 𝑥)$ of $𝑌$ given that $𝑋 = 𝑥$ and the corresponding conditional expectation $𝔼 [𝑌 ∣ 𝑋 = 𝑥]$ if the pair of random variables $(𝑋, 𝑌)$ has absolutely continuous distribution with joint density: $𝑓_{𝑋, 𝑌} (𝑥, 𝑦) = 𝜆^{2} 𝑒^{- 𝜆 𝑦} 𝟏_{{0 \leq 𝑥 \leq 𝑦}}$ .

Solution

Given the joint density $𝑓_{𝑋, 𝑌} (𝑥, 𝑦) = 𝜆^{2} 𝑒^{- 𝜆 𝑦} 𝟏_{{0 \leq 𝑥 \leq 𝑦}}$ , we first compute the marginal density of $𝑋$ . For $𝑥 \geq 0$ ,

𝑓_{𝑋} (𝑥) = \int_{𝑦 = 𝑥}^{\infty} 𝜆^{2} 𝑒^{- 𝜆 𝑦} 𝑑 𝑦 = 𝜆^{2} \cdot \frac{𝑒^{- 𝜆 𝑥}}{𝜆} = 𝜆 𝑒^{- 𝜆 𝑥}

and $𝑓_{𝑋} (𝑥) = 0$ for $𝑥 < 0$ .

Therefore, for $𝑥 \geq 0$ ,

𝑓_{𝑌 | 𝑋} (𝑦 | 𝑥) = \frac{𝑓_{𝑋, 𝑌} (𝑥, 𝑦)}{𝑓_{𝑋} (𝑥)} = \frac{𝜆^{2} 𝑒^{- 𝜆 𝑦} 𝟏_{{𝑦 \geq 𝑥}}}{𝜆 𝑒^{- 𝜆 𝑥}} = 𝜆 𝑒^{- 𝜆 (𝑦 - 𝑥)} 𝟏_{{𝑦 \geq 𝑥}}

This shows that $𝑌 | 𝑋 = 𝑥$ has the same distribution as $𝑥 + 𝐸$ where $𝐸 \sim E x p (𝜆)$ , hence

𝔼 [𝑌 | 𝑋 = 𝑥] = 𝑥 + \frac{1}{𝜆}

Problem 5

A machine produces a coin that shows heads with a random probability $𝑝$ . The value of $𝑝$ is unknown to us, but from many observations of the coins produced by the machine we know that the distribution of the random parameter $𝑝$ is uniform on $(0, 1 / 2)$ . We start tossing the coin. Compute the following probabilities:

The coin shows heads on the first toss.
The expected number of tosses until tails show up.

Solution

The head probability $𝑝 \sim U n i f (0, 1 / 2)$ . Thus the density is:
$𝑓_{𝑃} (𝑝) = 2 𝟏_{(0, 1 / 2)} (𝑝)$
The unconditional probability of heads on the first toss is
$ℙ (H on first toss) = 𝔼 [𝑝] = \int_{0}^{1 / 2} 𝑝 \cdot 2 𝑑 𝑝 = 2 \cdot \frac{𝑝^{2}}{2} 0_{1 / 2} = \frac{1}{4}$
Let $𝑇$ be the number of tosses until the first tail occurs. Conditional on $𝑝$ , tails occurs with probability $1 - 𝑝$ each toss, so $𝑇$ is geometric with parameter $1 - 𝑝$ . Hence
$𝔼 [𝑇 | 𝑝] = \frac{1}{1 - 𝑝}$
Taking expectation over $𝑝$ ,
$𝔼 [𝑇] = 𝔼 [\frac{1}{1 - 𝑝}] = \int_{0}^{1 / 2} \frac{1}{1 - 𝑝} \cdot 2 𝑑 𝑝 = 2 - ln (1 - 𝑝) 0_{1 / 2} = 2 ln 2$

Problem 6

The joint probability density function of the random variables $𝑋$ and $𝑌$ is given by

𝑓_{𝑋, 𝑌} (𝑥, 𝑦) = {\begin{cases} 𝑐 (𝑥^{2} + \frac{𝑥 𝑦}{2}), & (𝑥, 𝑦) \in (0, 1) \times (0, 2) \\ 0, & otherwise \end{cases}

Find the constant $𝑐$ .
Find the marginal density of $𝑋$ and compute $𝔼 [𝑋]$ .
Compute $ℙ (𝑋 > 𝑌)$ .
Compute $ℙ (𝑌 > \frac{1}{2} | 𝑋 < \frac{1}{2})$ .

Solution

Determine $𝑐$ from normalization:
$1 = \int_{0}^{1} \int_{0}^{2} 𝑐 (𝑥^{2} + \frac{𝑥 𝑦}{2}) 𝑑 𝑦 𝑑 𝑥$
For fixed $𝑥$ ,
$\int_{0}^{2} (𝑥^{2} + \frac{𝑥 𝑦}{2}) 𝑑 𝑦 = 2 𝑥^{2} + \frac{𝑥}{2} \cdot \frac{𝑦^{2}}{2} 0_{2} = 2 𝑥^{2} + 𝑥$
Thus
$1 = 𝑐 \int_{0}^{1} (2 𝑥^{2} + 𝑥) 𝑑 𝑥 = 𝑐 (\frac{2}{3} + \frac{1}{2}) = 𝑐 \cdot \frac{7}{6}$
so $𝑐 = \frac{6}{7}$
The marginal density of $𝑋$ (for $0 < 𝑥 < 1$ ) is
$𝑓_{𝑋} (𝑥) = \int_{0}^{2} 𝑐 (𝑥^{2} + \frac{𝑥 𝑦}{2}) 𝑑 𝑦 = 𝑐 (2 𝑥^{2} + 𝑥) = \frac{6}{7} (2 𝑥^{2} + 𝑥)$
and $𝑓_{𝑋} (𝑥) = 0$ otherwise.
Thus
$𝔼 [𝑋] = \int_{0}^{1} 𝑥 𝑓_{𝑋} (𝑥) 𝑑 𝑥 = \frac{6}{7} \int_{0}^{1} (2 𝑥^{3} + 𝑥^{2}) 𝑑 𝑥 = \frac{6}{7} (\frac{1}{2} + \frac{1}{3}) = \frac{5}{7}$
The event ${𝑋 > 𝑌}$ corresponds to the region $0 < 𝑦 < 𝑥 < 1$ (since $𝑥 \in (0, 1)$ ). Hence
$ℙ (𝑋 > 𝑌) = \int_{0}^{1} \int_{0}^{𝑥} 𝑐 (𝑥^{2} + \frac{𝑥 𝑦}{2}) 𝑑 𝑦 𝑑 𝑥$
For fixed $𝑥$ ,
$\int_{0}^{𝑥} (𝑥^{2} + \frac{𝑥 𝑦}{2}) 𝑑 𝑦 = 𝑥^{3} + \frac{𝑥}{2} \cdot \frac{𝑦^{2}}{2} 0_{𝑥} = 𝑥^{3} + \frac{𝑥^{3}}{4} = \frac{5}{4} 𝑥^{3}$
Therefore
$ℙ (𝑋 > 𝑌) = 𝑐 \int_{0}^{1} \frac{5}{4} 𝑥^{3} 𝑑 𝑥 = 𝑐 \cdot \frac{5}{4} \cdot \frac{1}{4} = 𝑐 \cdot \frac{5}{16} = \frac{6}{7} \cdot \frac{5}{16} = \frac{15}{56}$
By definition,
$ℙ (𝑌 > \frac{1}{2} | 𝑋 < \frac{1}{2}) = \frac{ℙ (𝑌 > \frac{1}{2}, 𝑋 < \frac{1}{2})}{ℙ (𝑋 < \frac{1}{2})}$
Calculate each part. First the denominator:
$ℙ (𝑋 < \frac{1}{2}) = \int_{0}^{1 / 2} 𝑓_{𝑋} (𝑥) 𝑑 𝑥 = 𝑐 \int_{0}^{1 / 2} (2 𝑥^{2} + 𝑥) 𝑑 𝑥 = 𝑐 (\frac{1}{12} + \frac{1}{8}) = 𝑐 \cdot \frac{5}{24} = \frac{5}{28}$
And the numerator:
$ℙ (𝑌 > \frac{1}{2}, 𝑋 < \frac{1}{2}) = \int_{0}^{1 / 2} \int_{1 / 2}^{2} 𝑐 (𝑥^{2} + \frac{𝑥 𝑦}{2}) 𝑑 𝑦 𝑑 𝑥$
For fixed $𝑥$ ,
$\int_{1 / 2}^{2} (𝑥^{2} + \frac{𝑥 𝑦}{2}) 𝑑 𝑦 = 𝑥^{2} (2 - \frac{1}{2}) + \frac{𝑥}{2} \cdot \frac{𝑦^{2}}{2} {1 / 2}_{2} = \frac{3}{2} 𝑥^{2} + \frac{𝑥}{4} (4 - \frac{1}{4}) = \frac{3}{2} 𝑥^{2} + \frac{15}{16} 𝑥$
Thus
$ℙ (𝑌 > \frac{1}{2}, 𝑋 < \frac{1}{2}) = 𝑐 \int_{0}^{1 / 2} (\frac{3}{2} 𝑥^{2} + \frac{15}{16} 𝑥) 𝑑 𝑥 = 𝑐 (\frac{1}{16} + \frac{15}{128}) = 𝑐 \cdot \frac{23}{128} = \frac{69}{448}$
Therefore,
$ℙ (𝑌 > \frac{1}{2} | 𝑋 < \frac{1}{2}) = \frac{69 / 448}{5 / 28} = \frac{69}{80}$

Homework 4

Problem 1

Let $𝑋$ be a random variable with values in $[0, + \infty]$ such that $𝔼 [𝑋] = 0$ . Explain why $𝑋 < \infty$ almost surely and show that $𝑋 = 0$ almost surely.

Proof

Since $𝑋 \geq 0$ and $𝔼 [𝑋] = 0 < \infty$ , by Markov’s inequality, for any $𝑡 > 0$ ,

ℙ (𝑋 \geq 𝑡) \leq \frac{𝔼 [𝑋]}{𝑡} = 0

Hence $ℙ (𝑋 \geq 𝑡) = 0$ for all $𝑡 > 0$ . In particular,

ℙ (𝑋 = + \infty) = lim_{𝑡 \to + \infty} ℙ (𝑋 \geq 𝑡) = lim_{𝑡 \to + \infty} 0 = 0

that is, $𝑋 < \infty$ almost surely.

Also notice that

{𝑋 > 0} = ⋃_{𝑛 = 1}^{\infty} {𝑋 \geq \frac{1}{𝑛}}

and by countable subadditivity,

ℙ (𝑋 > 0) \leq \sum_{𝑛 = 1}^{\infty} ℙ (𝑋 \geq \frac{1}{𝑛}) = 0

Thus $ℙ (𝑋 > 0) = 0$ . And since $𝑋$ takes values in $[0, + \infty]$ , we have

ℙ ({𝑋 = 0}) = ℙ ({𝑋 \leq 0}) = 1 - ℙ ({𝑋 > 0}) = 1

, and

This finishes the proof that $𝑋 < \infty$ a.s. and $𝑋 = 0$ a.s.

□

Problem 2

Let $𝑋$ be a random variable with $𝔼 [𝑋] = 3$ and $𝔼 [𝑋^{2}] = 13$ . Show that:

ℙ (- 2 \leq 𝑋 \leq 8) \geq \frac{21}{25}

Solution

Compute the variance of $𝑋$ :

V a r (𝑋) = 𝔼 [𝑋^{2}] - (𝔼 [𝑋])^{2} = 13 - 3^{2} = 4

Observe that

ℙ (- 2 \leq 𝑋 \leq 8) = ℙ (| 𝑋 - 3 | \leq 5)

By Chebyshev’s inequality,

ℙ (| 𝑋 - 3 | \geq 5) \leq \frac{V a r (𝑋)}{5^{2}} = \frac{4}{25}

Therefore,

ℙ (| 𝑋 - 3 | \leq 5) = 1 - ℙ (| 𝑋 - 3 | \geq 5) \geq 1 - \frac{4}{25} = \frac{21}{25}

Thus,

ℙ (- 2 \leq 𝑋 \leq 8) \geq \frac{21}{25}

Problem 3

Let $𝑋, 𝑌$ be two random variables such that $ℙ (𝑌 = 1) = 1 / 5, ℙ (𝑌 = 2) = 3 / 5$ and $ℙ (𝑌 = 3) = 1 / 5$ . In addition

𝑋 | {𝑌 = 1} \sim Exp (2), 𝑋 | {𝑌 = 2} \sim Exp (3) and 𝑋 ∣ {𝑌 = 3} = 7 .

Compute the moment generating function of $𝑋$ .

Solution

Compute each conditional moment generating function:

For $𝑋 ∣ {𝑌 = 1} \sim Exp (2)$

𝔼 [𝑒^{𝑡 𝑋} ∣ 𝑌 = 1] = \frac{2}{2 - 𝑡}, 𝑡 < 2

For $𝑋 ∣ {𝑌 = 2} \sim Exp (3)$

𝔼 [𝑒^{𝑡 𝑋} ∣ 𝑌 = 2] = \frac{3}{3 - 𝑡}, 𝑡 < 3

For $𝑋 ∣ {𝑌 = 3} = 7$ ,

𝔼 [𝑒^{𝑡 𝑋} ∣ 𝑌 = 3] = 𝑒^{7 𝑡}

Then we use the law of total expectation conditioning on $𝑌$ : for any $𝑡$ such that the expectations below are finite, we have

\begin{aligned} 𝑀_{𝑋} (𝑡) = 𝔼 [𝑒^{𝑡 𝑋}] & = \sum_{𝑦 = 1}^{3} 𝔼 [𝑒^{𝑡 𝑋} ∣ 𝑌 = 𝑦] ℙ (𝑌 = 𝑦) \\ = \frac{1}{5} \cdot \frac{2}{2 - 𝑡} + \frac{3}{5} \cdot \frac{3}{3 - 𝑡} + \frac{1}{5} 𝑒^{7 𝑡} \\ = \frac{2}{5 (2 - 𝑡)} + \frac{9}{5 (3 - 𝑡)} + \frac{1}{5} 𝑒^{7 𝑡}, 𝑡 < 2 \end{aligned}

So the moment generating function of $𝑋$ is

𝑀_{𝑋} (𝑡) = \frac{2}{5 (2 - 𝑡)} + \frac{9}{5 (3 - 𝑡)} + \frac{1}{5} 𝑒^{7 𝑡}, 𝑡 < 2

Problem 4

For any $𝑛 \in ℕ$ with $𝑛 \geq 1$ we set $𝑎_{𝑛} = 1 / (2 𝑛^{2})$ . Consider the sequence of random variables ${(𝑋_{𝑛})}_{𝑛 \in ℕ}$ with

𝑋_{𝑛} = {\begin{cases} 0, & with probability 𝑎_{𝑛} \\ 1, & with probability 1 - 2 𝑎_{𝑛} \\ 𝑛^{2}, & with probability 𝑎_{𝑛} \end{cases}

Check if $(𝑋_{𝑛})$ converges in probability and if $(𝑋_{𝑛})$ converges in $𝐿^{1}$ .

Solution

We first show that $𝑋_{𝑛} \to 1$ in probability.

Let $𝜀 > 0$ .
For $𝑛$ large enough s.t. $𝑛^{2} - 1 > 𝜀$ , $| 𝑋_{𝑛} - 1 | > 𝜀$ iff $𝑋_{𝑛} = 0$ or $𝑋_{𝑛} = 𝑛^{2}$ . Therefore,

ℙ (| 𝑋_{𝑛} - 1 | > 𝜀) = ℙ (𝑋_{𝑛} = 0) + ℙ (𝑋_{𝑛} = 𝑛^{2}) = 𝑎_{𝑛} + 𝑎_{𝑛} = 2 𝑎_{𝑛} = \frac{1}{𝑛^{2}}

Since $\frac{1}{𝑛^{2}} \to 0$ , we have:

lim_{𝑛 \to \infty} ℙ (| 𝑋_{𝑛} - 1 | > 𝜀) = 0

Since $𝜀 > 0$ is arbitrary, we conclude that

𝑋_{𝑛} \overset{ℙ}{\to} 1

We then show that $𝑋_{𝑛}$ does not converge to $1$ in $𝐿^{1}$ .

We compute

\begin{aligned} 𝔼 [| 𝑋_{𝑛} - 1 |] & = | 0 - 1 | 𝑎_{𝑛} + | 1 - 1 | (1 - 2 𝑎_{𝑛}) + | 𝑛^{2} - 1 | 𝑎_{𝑛} \\ = 𝑎_{𝑛} + (𝑛^{2} - 1) 𝑎_{𝑛} \\ = 𝑛^{2} 𝑎_{𝑛} = \frac{1}{2} \to̸ 0 \end{aligned}

Hence $$X_n \not\xrightarrow{L^1} 1$$

Problem 5

Let $𝑋$ and $𝑌$ be independent random variables with densities

\begin{aligned} 𝑓_{𝑋} (𝑥) = {2 𝑥, & 0 \leq 𝑥 \leq 1, \\ 0, & otherwise 𝑓_{𝑌} (𝑦) = {\begin{cases} 1 / 2, & 0 \leq 𝑦 \leq 2 \\ 0, & otherwise \end{cases} \end{aligned}

Find the distribution function of the sum $𝑍 = 𝑋 + 𝑌$ .

Solution

Since $𝑋$ and $𝑌$ are independent, the density of $𝑍 = 𝑋 + 𝑌$ is given by convolution:

𝑓_{𝑍} (𝑧) = \int_{- \infty}^{\infty} 𝑓_{𝑋} (𝑥) 𝑓_{𝑌} (𝑧 - 𝑥) 𝑑 𝑥

where we know

𝑓_{𝑋} (𝑥) = 2 𝑥 𝟏_{[0, 1]} (𝑥), 𝑓_{𝑌} (𝑦) = \frac{1}{2} 𝟏_{[0, 2]} (𝑦)

Thus

\begin{aligned} 𝑓_{𝑍} (𝑧) & = \int_{- \infty}^{\infty} 2 𝑥 \cdot \frac{1}{2} 𝟏_{[0, 1]} (𝑥) 𝟏_{[0, 2]} (𝑧 - 𝑥) 𝑑 𝑥 \\ = \int_{- \infty}^{\infty} 𝑥 𝟏_{[0, 1]} (𝑥) 𝟏_{[0, 2]} (𝑧 - 𝑥) 𝑑 𝑥 \\ = \int_{max (0, 𝑧 - 2)}^{min (1, 𝑧)} 𝑥 𝑑 𝑥 \end{aligned}

Compute this piecewise: If $0 \leq 𝑧 \leq 1$ , then the interval is $[0, 𝑧]$ , so

𝑓_{𝑍} (𝑧) = \int_{0}^{𝑧} 𝑥 𝑑 𝑥 = \frac{𝑧^{2}}{2}

If $1 \leq 𝑧 \leq 2$ , then the interval is $[0, 1]$ , so

𝑓_{𝑍} (𝑧) = \int_{0}^{1} 𝑥 𝑑 𝑥 = \frac{1}{2}

If $2 \leq 𝑧 \leq 3$ , then the interval is $[𝑧 - 2, 1]$ , so

𝑓_{𝑍} (𝑧) = \int_{𝑧 - 2}^{1} 𝑥 𝑑 𝑥 = \frac{1 - (𝑧 - 2)^{2}}{2}

Outside $[0, 3]$ , clearly $𝑓_{𝑍} (𝑧) = 0$ . Therefore,

𝑓_{𝑍} (𝑧) = {\begin{cases} 0, & 𝑧 < 0, \\ \frac{𝑧^{2}}{2}, & 0 \leq 𝑧 \leq 1, \\ \frac{1}{2}, & 1 \leq 𝑧 \leq 2, \\ \frac{1 - (𝑧 - 2)^{2}}{2}, & 2 \leq 𝑧 \leq 3, \\ 0, & 𝑧 > 3 \end{cases}

Now we integrate to get the cdf. For $𝑧 < 0$ , $𝐹_{𝑍} (𝑧) = 0$ .

For $0 \leq 𝑧 \leq 1$ ,

𝐹_{𝑍} (𝑧) = \int_{0}^{𝑧} \frac{𝑡^{2}}{2} 𝑑 𝑡 = \frac{𝑧^{3}}{6}

For $1 \leq 𝑧 \leq 2$ ,

𝐹_{𝑍} (𝑧) = 𝐹_{𝑍} (1) + \int_{1}^{𝑧} \frac{1}{2} 𝑑 𝑡 = \frac{1}{6} + \frac{𝑧 - 1}{2} = \frac{𝑧}{2} - \frac{1}{3}

For $2 \leq 𝑧 \leq 3$ ,

𝐹_{𝑍} (𝑧) = 𝐹_{𝑍} (2) + \int_{2}^{𝑧} \frac{1 - (𝑡 - 2)^{2}}{2} 𝑑 𝑡 = \frac{2}{3} + \frac{𝑧 - 2}{2} - \frac{(𝑧 - 2)^{3}}{6}

And for $𝑧 \geq 3$ , $𝐹_{𝑍} (𝑧) = 1$ .

Hence the cdf of $𝑍 = 𝑋 + 𝑌$ is

𝐹_{𝑍} (𝑧) = {\begin{cases} 0, & 𝑧 < 0, \\ \frac{𝑧^{3}}{6}, & 0 \leq 𝑧 \leq 1, \\ \frac{𝑧}{2} - \frac{1}{3}, & 1 \leq 𝑧 \leq 2, \\ \frac{2}{3} + \frac{𝑧 - 2}{2} - \frac{(𝑧 - 2)^{3}}{6}, & 2 \leq 𝑧 \leq 3, \\ 1, & 𝑧 \geq 3 \end{cases}

Problem 6

Let $𝑎_{1}, 𝑎_{2}, \dots, 𝑎_{𝑛}$ and $𝜆$ be positive constants and let ${𝑋_{𝑖} : 1 \leq 𝑖 \leq 𝑛}$ be independent random variables with

𝑋_{𝑖} \sim Γ (𝑎_{𝑖}, 𝜆), 𝑖 = 1, 2, \dots, 𝑛

(i.e., with the same second parameter $𝜆$ ). Show that $𝑋_{1} + 𝑋_{2} + \dots + 𝑋_{𝑛} \sim Γ (𝑎, 𝜆)$ , with $𝑎 = \sum_{𝑖 = 1}^{𝑛} 𝑎_{𝑖}$ .

Proof

Let

𝑆_{𝑛} ≔ 𝑋_{1} + 𝑋_{2} + \dots + 𝑋_{𝑛}, 𝑎 = \sum_{𝑖 = 1}^{𝑛} 𝑎_{𝑖}

We need to show that $𝑆_{𝑛} \sim Γ (𝑎, 𝜆)$ .

Since $𝑋_{𝑖} \sim Γ (𝑎_{𝑖}, 𝜆)$ , its moment generating function is

𝑀_{𝑋_{𝑖}} (𝑡) = 𝔼 [𝑒^{𝑡 𝑋_{𝑖}}] = {(\frac{𝜆}{𝜆 - 𝑡})}^{𝑎_{𝑖}}, 𝑡 < 𝜆

Since $𝑋_{1}, \dots, 𝑋_{𝑛}$ are independent, the moment generating function of $𝑆_{𝑛}$ is

𝑀_{𝑆_{𝑛}} (𝑡) = 𝔼 [𝑒^{𝑡 (𝑋_{1} + \dots + 𝑋_{𝑛})}] = \prod_{𝑖 = 1}^{𝑛} 𝔼 [𝑒^{𝑡 𝑋_{𝑖}}] = \prod_{𝑖 = 1}^{𝑛} 𝑀_{𝑋_{𝑖}} (𝑡)

Therefore,

𝑀_{𝑆_{𝑛}} (𝑡) = \prod_{𝑖 = 1}^{𝑛} {(\frac{𝜆}{𝜆 - 𝑡})}^{𝑎_{𝑖}} = {(\frac{𝜆}{𝜆 - 𝑡})}^{\sum_{𝑖 = 1}^{𝑛} 𝑎_{𝑖}} = {(\frac{𝜆}{𝜆 - 𝑡})}^{𝑎}, 𝑡 < 𝜆

Note this is exactly the moment generating function of a $Γ (𝑎, 𝜆)$ random variable. Hence,

𝑆_{𝑛} = 𝑋_{1} + \dots + 𝑋_{𝑛} \sim Γ (𝑎, 𝜆), 𝑎 = \sum_{𝑖 = 1}^{𝑛} 𝑎_{𝑖}

□

Homework 5

Problem 1

Let ${(𝑈_{𝑖})}_{𝑖 \in ℕ}$ be an i.i.d sequence of random variables with $𝑈_{𝑖} \sim 𝑈 ([0, 1])$ . Show that

${lim}_{𝑛 \to \infty} {(𝑈_{1} 𝑈_{2} \dots 𝑈_{𝑛})}^{1 / 𝑛} = 𝑒^{- 1}$ almost surely.
${lim}_{𝑛 \to \infty} 𝑈_{1} 𝑈_{2} \dots 𝑈_{𝑛} = 0$ almost surely.

Proof

Let
$𝑋_{𝑖} ≔ - log 𝑈_{𝑖}, 𝑖 \in ℕ$
Since $𝑈_{𝑖} \sim 𝑈 ([0, 1])$ , for $𝑥 \geq 0$ ,
$ℙ (𝑋_{𝑖} \leq 𝑥) = ℙ (- log 𝑈_{𝑖} \leq 𝑥) = ℙ (𝑈_{𝑖} \geq 𝑒^{- 𝑥}) = 1 - 𝑒^{- 𝑥}$
Thus $𝑋_{𝑖} \sim E x p (1)$ , so $𝔼 [𝑋_{𝑖}] = 1$ . Also,
$log ((𝑈_{1} 𝑈_{2} \dots 𝑈_{𝑛})^{1 / 𝑛}) = \frac{1}{𝑛} \sum_{𝑖 = 1}^{𝑛} log 𝑈_{𝑖} = - \frac{1}{𝑛} \sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖}$
By the Strong Law of Large Numbers,
$\frac{1}{𝑛} \sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖} = \frac{1}{𝑛} \sum_{𝑖 = 1}^{𝑛} log 𝑈_{𝑖} \to 1 a.s.$
Since the exponential function is continuous,
$(𝑈_{1} 𝑈_{2} \dots 𝑈_{𝑛})^{1 / 𝑛} = exp (\frac{1}{𝑛} \sum_{𝑖 = 1}^{𝑛} log 𝑈_{𝑖}) \to 𝑒^{- 1} a.s.$
Let
$𝑃_{𝑛} ≔ 𝑈_{1} 𝑈_{2} \dots 𝑈_{𝑛}$
Then from the first part we instantly have
$𝑃_{𝑛}^{1 / 𝑛} \to 𝑒^{- 1} < 1 a.s.$
Then for any event $𝜔$ in the event of probability one where this convergence holds, choose $𝑟$ s.t. $𝑒^{- 1} < 𝑟 < 1$ , then for all sufficiently large $𝑛$ ,
$𝑃_{𝑛} (𝜔)^{1 / 𝑛} < 𝑟, i.e. 𝑃_{𝑛} (𝜔) < 𝑟^{𝑛}$
Since $0 < 𝑟 < 1$ , we have $𝑟^{𝑛} \to 0$ . Therefore $𝑃_{𝑛} (𝜔) \to 0$ . Therefore
$𝑈_{1} 𝑈_{2} \dots 𝑈_{𝑛} \to 0 a.s.$

□

Problem 2

A factory produces small resistors, and the resistance of each resistor is a random variable $𝑋_{𝑖}$ with unknown mean $𝜇$ and variance $𝜎^{2} = 0.25$ ohms ${\begin{matrix} \end{matrix}}^{2}$ . The quality control engineer wants to estimate the average resistance of a batch. She decides to measure $𝑛$ resistors and compute the sample average

{\bar{𝑋}}_{𝑛} ≔ \frac{𝑋_{1} + 𝑋_{2} + \dots + 𝑋_{𝑛}}{𝑛}

Determine approximately the number of resistors $𝑛$ she needs to measure so that the probability that the sample mean differs from the true mean by more than 0.005 ohms is less than $1 %$ , i.e.,

ℙ (| {\bar{𝑋}}_{𝑛} - 𝜇 | > 0.005) < 0.01

Proof

By linearity of expectation,

𝔼 [{\bar{𝑋}}_{𝑛}] = 𝜇

And (assmuming the $𝑋_{𝑖}$ are independent), we have

V a r (\sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖}) = \sum_{𝑖 = 1}^{𝑛} V a r (𝑋_{𝑖}) + \sum_{𝑖 \neq 𝑗} C o v (𝑋_{𝑖}, 𝑋_{𝑗}) = 𝑛 𝜎^{2} = 0.25 𝑛

Thus

V a r ({\bar{𝑋}}_{𝑛}) = \frac{𝜎^{2}}{𝑛} = \frac{0.25}{𝑛}

By Chebyshev’s inequality, for any $𝜀 > 0$ ,

ℙ (| {\bar{𝑋}}_{𝑛} - 𝜇 | > 𝜀) \leq \frac{V a r ({\bar{𝑋}}_{𝑛})}{𝜀^{2}}

Taking $𝜀 = 0.005$ , we get

ℙ (| {\bar{𝑋}}_{𝑛} - 𝜇 | > 0.005) \leq \frac{0.25 / 𝑛}{(0.005)^{2}} = \frac{0.25}{𝑛 \cdot 0.000025} = \frac{10000}{𝑛}

We want this upper bound to be less than $0.01$ , so it is enough to require

\frac{10000}{𝑛} < 0.01

Therefore, she needs to measure approximately $𝑛 \approx 1, 000, 000$ resistors.

□

Problem 3

Let ${(𝑋_{𝑛})}_{𝑛 \in ℕ}$ be a sequence of random variables with $ℙ (𝑋_{𝑛} \neq 0) = 1 / 𝑛^{2}$ for all $𝑛 \in ℕ$ . Show that with probability 1, there exists an $𝑛_{0} \in ℕ$ such that $𝑋_{𝑛} = 0$ for all $𝑛 \geq 𝑛_{0}$ .

Proof

Let

𝐴_{𝑛} ≔ {𝑋_{𝑛} \neq 0}, 𝑛 \in ℕ

Then $ℙ (𝐴_{𝑛}) = \frac{1}{𝑛^{2}}$ by assumption. Hence

\sum_{𝑛 = 1}^{\infty} ℙ (𝐴_{𝑛}) = \sum_{𝑛 = 1}^{\infty} \frac{1}{𝑛^{2}} < \infty

By Borel-Cantelli lemma,

ℙ (\underset{𝑛 \to \infty}{lim sup} 𝐴_{𝑛}) = 0

So with probability $1$ , only finitely many of the events $𝐴_{𝑛}$ occur. In other words, with probability $1$ , there exists $𝑛_{0} \in ℕ$ such that for all $𝑛 \geq 𝑛_{0}$ ,

𝐴_{𝑛}^{𝑐} = {𝑋_{𝑛} = 0}

occurs. Equivalently,

𝑋_{𝑛} = 0 for all 𝑛 \geq 𝑛_{0}

Therefore, with probability $1$ , there exists $𝑛_{0} \in ℕ$ such that $𝑋_{𝑛} = 0$ for all $𝑛 \geq 𝑛_{0}$ .

□

Problem 4

Assume that ${(𝑋_{𝑛})}_{𝑛 \in ℕ}$ be a sequence of i.i.d random variables with density

𝑓 (𝑥) = {\begin{cases} \frac{1}{2 \sqrt{𝑥}}, & if 𝑥 \in (0, 1), \\ 0, & otherwise \end{cases}

Find the distribution function of $𝑋_{1}$ .
Let $𝑌_{𝑛} = min {𝑋_{1}, \dots, 𝑋_{𝑛}}$ for any $𝑛 \in ℕ$ . Show that $𝑛^{2} 𝑌_{𝑛} \overset{𝑑}{\to} 𝑌$ , where $𝑌$ has distribution function
$𝐹_{𝑌} (𝑥) = {\begin{cases} 0, & if 𝑥 \leq 0 \\ 1 - 𝑒^{- \sqrt{𝑥}}, & otherwise \end{cases}$

Proof

For $𝑥 \in ℝ$ ,
$𝐹_{𝑋_{1}} (𝑥) = ℙ (𝑋_{1} \leq 𝑥) = \int_{- \infty}^{𝑥} 𝑓 (𝑡) 𝑑 𝑡 = {\begin{cases} 0, & 𝑥 \leq 0, \\ \int_{0}^{𝑥} \frac{1}{2 \sqrt{𝑡}} 𝑑 𝑡 = \sqrt{𝑥}, & 0 < 𝑥 < 1, \\ 1, & 𝑥 \geq 1 . \end{cases}$
If $𝑥 \leq 0$ , then $𝑛^{2} 𝑌_{𝑛} \geq 0$ , so
$ℙ (𝑛^{2} 𝑌_{𝑛} \leq 𝑥) = 0$
Now consider $𝑥 > 0$ . Then
$ℙ (𝑛^{2} 𝑌_{𝑛} > 𝑥) = ℙ (𝑌_{𝑛} > \frac{𝑥}{𝑛^{2}})$
Since
$𝑌_{𝑛} > \frac{𝑥}{𝑛^{2}} \Leftrightarrow 𝑋_{1} > \frac{𝑥}{𝑛^{2}}, \dots, 𝑋_{𝑛} > \frac{𝑥}{𝑛^{2}}$
and the $𝑋_{𝑖}$ are independent,
$ℙ (𝑌_{𝑛} > \frac{𝑥}{𝑛^{2}}) = {(ℙ (𝑋_{1} > \frac{𝑥}{𝑛^{2}}))}^{𝑛}$
For all sufficiently large $𝑛$ , we have $0 < \frac{𝑥}{𝑛^{2}} < 1$ , hence
$ℙ (𝑋_{1} > \frac{𝑥}{𝑛^{2}}) = 1 - 𝐹_{𝑋_{1}} (\frac{𝑥}{𝑛^{2}}) = 1 - \sqrt{\frac{𝑥}{𝑛^{2}}} = 1 - \frac{\sqrt{𝑥}}{𝑛}$
Therefore,
$ℙ (𝑛^{2} 𝑌_{𝑛} > 𝑥) = {(1 - \frac{\sqrt{𝑥}}{𝑛})}^{𝑛}$
so
$𝐹_{𝑛^{2} 𝑌_{𝑛}} (𝑥) = ℙ (𝑛^{2} 𝑌_{𝑛} \leq 𝑥) = 1 - {(1 - \frac{\sqrt{𝑥}}{𝑛})}^{𝑛}$
Taking $𝑛 \to \infty$ , we use the standard limit
${(1 - \frac{𝑎}{𝑛})}^{𝑛} \to 𝑒^{- 𝑎}$
with $𝑎 = \sqrt{𝑥}$ , we obtain
$𝐹_{𝑛^{2} 𝑌_{𝑛}} (𝑥) \to 1 - 𝑒^{- \sqrt{𝑥}}, 𝑥 > 0$
Thus,
$𝐹_{𝑛^{2} 𝑌_{𝑛}} (𝑥) \to {\begin{cases} 0, & 𝑥 \leq 0, \\ 1 - 𝑒^{- \sqrt{𝑥}}, & 𝑥 > 0 \end{cases}$
which is exactly $𝐹_{𝑌} (𝑥)$ . Hence
$𝑛^{2} 𝑌_{𝑛} \overset{𝑑}{\to} 𝑌$

□

Problem 5

Let ${(𝑋_{𝑛})}_{𝑛 \in ℕ}$ be a sequence of random variables with values in $ℝ$ . Show that there exists a sequence ${(𝑎_{𝑛})}_{𝑛 \in ℕ}$ with $𝑎_{𝑛} > 0$ such that

\frac{𝑋_{𝑛}}{𝑎_{𝑛}} \overset{a.s.}{\to} 0

For simplicity you may assume that $𝑋_{𝑛} \sim Exp (1 / 𝑛)$ .

Hint: For any $𝑛 \in ℕ$ construct $𝑏_{𝑛}$ such that $ℙ (| 𝑋_{𝑛} | \geq 𝑏_{𝑛}) \leq \frac{1}{2^{𝑛}}$ and use Borel-Cantelli for the events ${| 𝑋_{𝑛} | / 𝑏_{𝑛} \geq 𝑛}$ .

Proof

For each $𝑛 \in ℕ$ , choose $𝑏_{𝑛} > 0$ such that

ℙ (| 𝑋_{𝑛} | \geq 𝑏_{𝑛}) \leq \frac{1}{2^{𝑛}}

Notice such a choice is always possible, since $| 𝑋_{𝑛} |$ is a well-defined random variable, which implies $ℙ (| 𝑋_{𝑛} | \geq 𝑡) \to 0$ as $𝑡 \to \infty$ .

Now define

𝑎_{𝑛} ≔ 𝑛 𝑏_{𝑛} > 0

Consider the sequence of events

𝐸_{𝑛} ≔ {\frac{| 𝑋_{𝑛} |}{𝑎_{𝑛}} \geq \frac{1}{𝑛}}

By the definition of $𝑎_{𝑛}$ , we have

𝐸_{𝑛} = {\frac{| 𝑋_{𝑛} |}{𝑛 𝑏_{𝑛}} \geq \frac{1}{𝑛}} = {| 𝑋_{𝑛} | \geq 𝑏_{𝑛}}

It follows that

\sum_{𝑛 = 1}^{\infty} ℙ (𝐸_{𝑛}) = \sum_{𝑛 = 1}^{\infty} ℙ (| 𝑋_{𝑛} | \geq 𝑏_{𝑛}) \leq \sum_{𝑛 = 1}^{\infty} \frac{1}{2^{𝑛}} < \infty

By the first Borel-Cantelli lemma,

ℙ (𝐸_{𝑛} infinitely often) = 0

This means that for almost all $𝜔 \in Ω$ , there exists $𝑁 (𝜔) \in ℕ$ such that: for all $𝑛 \geq 𝑁 (𝜔)$ , the event $𝐸_{𝑛}$ does not occur, i.e.,

\frac{| 𝑋_{𝑛} (𝜔) |}{𝑎_{𝑛}} < \frac{1}{𝑛}

Since $1 / 𝑛 \to 0$ as $𝑛 \to \infty$ , it follows immediately that

\frac{𝑋_{𝑛}}{𝑎_{𝑛}} \to 0 a.s.

If we assume $𝑋_{𝑛} \sim E x p (1 / 𝑛)$ , we can provide an explicit sequence. Since

ℙ (𝑋_{𝑛} \geq 𝑡) = 𝑒^{- 𝑡 / 𝑛} for 𝑡 \geq 0

we can choose $𝑏_{𝑛} = 𝑛^{2} log 2$ . So that

ℙ (𝑋_{𝑛} \geq 𝑏_{𝑛}) = 𝑒^{- (𝑛^{2} log 2) / 𝑛} = 𝑒^{- 𝑛 log 2} = \frac{1}{2^{𝑛}}

Then $𝑎_{𝑛} = 𝑛 𝑏_{𝑛} = 𝑛^{3} log 2$ , the general argument above guarantees that

\frac{𝑋_{𝑛}}{𝑎_{𝑛}} = \frac{𝑋_{𝑛}}{𝑛^{3} log 2} \overset{a.s.}{\to} 0

This completes the proof.

□

Problem 6

Let ${𝑋_{𝑖}}_{𝑖 \geq 1}$ be i.i.d. positive integer-valued random variables with $0 < 𝔼 [𝑋_{1}] < \infty$ . Interpret $𝑋_{𝑖}$ as the number of children in family $𝑖$ . From the first $𝑛$ families, choose a child uniformly at random among all children. Let $𝑁_{𝑛}$ denote the number of children in the selected child’s family. Show that $𝑁_{𝑛} \overset{𝑑}{\to} 𝑋_{1}^{*}$ , where $𝑋_{1}^{*}$ has distribution

ℙ (𝑋_{1}^{*} = 𝑘) = \frac{𝑘 ℙ (𝑋_{1} = 𝑘)}{𝔼 [𝑋_{1}]}

Proof

For each $𝑛 \in ℕ$ , let

𝑆_{𝑛} ≔ 𝑋_{1} + \dots + 𝑋_{𝑛}

be the total number of children in the first $𝑛$ families.

Given $𝑋_{1}, \dots, 𝑋_{𝑛}$ , we choose one child uniformly at random among these $𝑆_{𝑛}$ children. Hence, conditionally on $𝑋_{1}, \dots, 𝑋_{𝑛}$ , the probability that the chosen child comes from a family with exactly $𝑘$ children is

ℙ (𝑁_{𝑛} = 𝑘 ∣ 𝑋_{1}, \dots, 𝑋_{𝑛}) = \frac{\sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖} 𝟏_{{𝑋_{𝑖} = 𝑘}}}{𝑆_{𝑛}}

Since $𝑋_{𝑖} 𝟏_{{𝑋_{𝑖} = 𝑘}} = 𝑘 𝟏_{{𝑋_{𝑖} = 𝑘}}$ , this becomes

ℙ (𝑁_{𝑛} = 𝑘 ∣ 𝑋_{1}, \dots, 𝑋_{𝑛}) = \frac{𝑘 \sum_{𝑖 = 1}^{𝑛} 𝟏_{{𝑋_{𝑖} = 𝑘}}}{𝑆_{𝑛}} = \frac{𝑘 \cdot \frac{1}{𝑛} \sum_{𝑖 = 1}^{𝑛} 𝟏_{{𝑋_{𝑖} = 𝑘}}}{\frac{1}{𝑛} \sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖}}

By the Strong Law of Large Numbers,

\frac{1}{𝑛} \sum_{𝑖 = 1}^{𝑛} 𝟏_{{𝑋_{𝑖} = 𝑘}} \to 𝔼 [𝟏_{{𝑋_{1} = 𝑘}}] = ℙ (𝑋_{1} = 𝑘) a.s.

and

\frac{1}{𝑛} \sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖} \to 𝔼 [𝑋_{1}] a.s.

Since $0 < 𝔼 [𝑋_{1}] < \infty$ , it follows that

ℙ (𝑁_{𝑛} = 𝑘 ∣ 𝑋_{1}, \dots, 𝑋_{𝑛}) \to \frac{𝑘 ℙ (𝑋_{1} = 𝑘)}{𝔼 [𝑋_{1}]} a.s.

Taking expectations on both sides, and using dominated convergence because $0 \leq ℙ (𝑁_{𝑛} = 𝑘 ∣ 𝑋_{1}, \dots, 𝑋_{𝑛}) \leq 1$ , we obtain

ℙ (𝑁_{𝑛} = 𝑘) = 𝔼 ℙ (𝑁_{𝑛} = 𝑘 ∣ 𝑋_{1}, \dots, 𝑋_{𝑛}) \to \frac{𝑘 ℙ (𝑋_{1} = 𝑘)}{𝔼 [𝑋_{1}]}

Thus, for every $𝑘 \in ℕ$ ,

ℙ (𝑁_{𝑛} = 𝑘) \to ℙ (𝑋_{1}^{*} = 𝑘)

Hence

𝑁_{𝑛} \overset{𝑑}{\to} 𝑋_{1}^{*}

□

Homework 6

Problem 1

Let $𝑋$ be a random variable such that $𝔼 [| 𝑋 |] < \infty$ , that is $𝑋 \in 𝐿^{1}$ . Denote by $𝜙_{𝑋} (𝑡) ≔ 𝔼 [𝑒^{𝑖 𝑡 𝑋}], 𝑡 \in ℝ$ , its characteristic function.

Show that $𝜙_{𝑋}$ is differentiable with $𝜙_{𝑋}^{'} (𝑡) = 𝑖 𝔼 [𝑋 𝑒^{𝑖 𝑡 𝑋}]$ . (Hint: DCT)
If, in addition, $𝑋$ has symmetric distribution(i.e. $𝑋, - 𝑋$ have the same distribution), then show that $𝜙_{𝑋} (𝑡) \in ℝ$ for any $𝑡 \in ℝ$ .

Proof

Fix $𝑡 \in ℝ$ . Consider the difference quotient
$\frac{𝜙_{𝑋} (𝑡 + ℎ) - 𝜙_{𝑋} (𝑡)}{ℎ} = 𝔼 [𝑒^{𝑖 𝑡 𝑋} \frac{𝑒^{𝑖 ℎ 𝑋} - 1}{ℎ}]$
Notice we have (for a.e. $𝜔$ ):
$lim_{ℎ \to 0} \frac{𝑒^{𝑖 ℎ 𝑋} - 1}{ℎ} = 𝑖 𝑋$
Hence
$𝑒^{𝑖 𝑡 𝑋} \frac{𝑒^{𝑖 ℎ 𝑋} - 1}{ℎ} \to 𝑖 𝑋 𝑒^{𝑖 𝑡 𝑋} a.s. as ℎ \to 0$
Using the mean value theorem for the function $𝑢 \mapsto 𝑒^{𝑖 𝑢 𝑋}$ , for $| ℎ | \leq 1$ we have
$| \frac{𝑒^{𝑖 ℎ 𝑋} - 1}{ℎ} | \leq | 𝑋 |$
Also, $| 𝑒^{𝑖 𝑡 𝑋} | = 1$ , so
$| 𝑒^{𝑖 𝑡 𝑋} \frac{𝑒^{𝑖 ℎ 𝑋} - 1}{ℎ} | = | 𝑒^{𝑖 𝑡 𝑋} | | \frac{𝑒^{𝑖 ℎ 𝑋} - 1}{ℎ} | \leq | 𝑋 |$
Since $𝑋 \in 𝐿^{1}$ , we have $𝔼 [| 𝑋 |] < \infty$ , therefore $| 𝑋 |$ is a dominating integrable random variable for $𝑒^{𝑖 𝑡 𝑋} \frac{𝑒^{𝑖 ℎ 𝑋} - 1}{ℎ}$ . Then by DCT,
$lim_{ℎ \to 0} \frac{𝜙_{𝑋} (𝑡 + ℎ) - 𝜙_{𝑋} (𝑡)}{ℎ} = 𝔼 [lim_{ℎ \to 0} 𝑒^{𝑖 𝑡 𝑋} \frac{𝑒^{𝑖 ℎ 𝑋} - 1}{ℎ}] = 𝔼 [𝑖 𝑋 𝑒^{𝑖 𝑡 𝑋}]$
Thus $𝜙_{𝑋}$ is differentiable and
$𝜙_{𝑋}^{'} (𝑡) = 𝑖 𝔼 [𝑋 𝑒^{𝑖 𝑡 𝑋}]$
Since random variables with the same distribution have the same expectation under measurable functions for which the expectation exists, we get
$𝜙_{𝑋} (𝑡) = 𝔼 [𝑒^{𝑖 𝑡 𝑋}] = 𝔼 [𝑒^{𝑖 𝑡 (- 𝑋)}] = 𝔼 [𝑒^{- 𝑖 𝑡 𝑋}] = 𝜙_{𝑋} (- 𝑡)$
On the other hand since $𝜙_{𝑋} (- 𝑡) = \bar{𝜙_{𝑋} (𝑡)}$ , we thus have
$𝜙_{𝑋} (𝑡) = \bar{𝜙_{𝑋} (𝑡)}$
A complex number equal to its own conjugate must be real. Therefore,
$𝜙_{𝑋} (𝑡) \in ℝ, \forall 𝑡 \in ℝ$
Writing
$𝜙_{𝑋} (𝑡) = 𝔼 [cos (𝑡 𝑋)] + 𝑖 𝔼 [sin (𝑡 𝑋)]$
Since $𝑋$ is symmetric, $sin (𝑡 𝑥)$ is odd, so
$𝔼 [sin (𝑡 𝑋)] = 0$
Therefore $𝜙_{𝑋} (𝑡) = 𝔼 [cos (𝑡 𝑋)] \in ℝ$ .

□

Problem 2

Let $𝑋 \sim Bin (𝑛, 𝑝)$ , where $𝑛 \in ℕ, 𝑝 \in (0, 1)$ and $𝑌 \sim Pois (𝜆)$ , where $𝜆 > 0$ .

Compute the characteristic functions of $𝑋, 𝑌$ .
Let ${(𝑝_{𝑛})}_{𝑛 \in ℕ}$ be a sequence in $(0, 1)$ such that ${lim}_{𝑛 \to \infty} 𝑛 𝑝_{𝑛} = 𝜆$ and ${(𝑋_{𝑛})}_{𝑛 \in ℕ}$ be a sequence of random variables with $𝑋_{𝑛} \sim Bin (𝑛, 𝑝_{𝑛})$ . Show that $𝑋_{𝑛} \overset{𝑑}{\to} 𝑌$ .

Proof

We have
$ℙ (𝑋 = 𝑘) = (\binom{𝑛}{𝑘}) 𝑝^{𝑘} (1 - 𝑝)^{𝑛 - 𝑘}, 𝑘 = 0, 1, \dots, 𝑛$
Therefore,
$𝜙_{𝑋} (𝑡) = 𝔼 [𝑒^{𝑖 𝑡 𝑋}] = \sum_{𝑘 = 0}^{𝑛} 𝑒^{𝑖 𝑡 𝑘} (\binom{𝑛}{𝑘}) 𝑝^{𝑘} (1 - 𝑝)^{𝑛 - 𝑘}$
We factor $𝑒^{𝑖 𝑡 𝑘}$ into $(𝑝 𝑒^{𝑖 𝑡})^{𝑘} / 𝑝^{𝑘}$ and obtain
$𝜙_{𝑋} (𝑡) = \sum_{𝑘 = 0}^{𝑛} (\binom{𝑛}{𝑘}) (𝑝 𝑒^{𝑖 𝑡})^{𝑘} (1 - 𝑝)^{𝑛 - 𝑘}$
By the binomial formula,
$𝜙_{𝑋} (𝑡) = (1 - 𝑝 + 𝑝 𝑒^{𝑖 𝑡})^{𝑛}$
Next for $𝑌 \sim Pois (𝜆)$ we have
$ℙ (𝑌 = 𝑘) = 𝑒^{- 𝜆} \frac{𝜆^{𝑘}}{𝑘!}, 𝑘 = 0, 1, 2, \dots$
Hence
$𝜙_{𝑌} (𝑡) = 𝔼 [𝑒^{𝑖 𝑡 𝑌}] = \sum_{𝑘 = 0}^{\infty} 𝑒^{𝑖 𝑡 𝑘} 𝑒^{- 𝜆} \frac{𝜆^{𝑘}}{𝑘!}$
Thus
$𝜙_{𝑌} (𝑡) = 𝑒^{- 𝜆} \sum_{𝑘 = 0}^{\infty} \frac{(𝜆 𝑒^{𝑖 𝑡})^{𝑘}}{𝑘!} = 𝑒^{- 𝜆} 𝑒^{𝜆 𝑒^{𝑖 𝑡}} = 𝑒^{𝜆 (𝑒^{𝑖 𝑡} - 1)}$
So the characteristic functions are
$𝜙_{𝑋} (𝑡) = (1 - 𝑝 + 𝑝 𝑒^{𝑖 𝑡})^{𝑛}, 𝜙_{𝑌} (𝑡) = 𝑒^{𝜆 (𝑒^{𝑖 𝑡} - 1)}$
By part (a),
$𝜙_{𝑋_{𝑛}} (𝑡) = (1 - 𝑝_{𝑛} + 𝑝_{𝑛} 𝑒^{𝑖 𝑡})^{𝑛} = {(1 + 𝑝_{𝑛} (𝑒^{𝑖 𝑡} - 1))}^{𝑛}$
We now compute the limit as $𝑛 \to \infty$ .
Set
$𝑎_{𝑛} ≔ 𝑝_{𝑛} (𝑒^{𝑖 𝑡} - 1)$
Since $𝑝_{𝑛} \to 0$ (as $𝑛 𝑝_{𝑛} \to 𝜆 < \infty$ ), we have $𝑎_{𝑛} \to 0$ . Therefore,
$log (1 + 𝑎_{𝑛}) \to 𝑎_{𝑛}, 𝑛 \to \infty$
Hence
$𝑛 log (1 + 𝑎_{𝑛}) \to 𝑛 𝑎_{𝑛} = 𝑛 𝑝_{𝑛} (𝑒^{𝑖 𝑡} - 1) ⟹ 𝜆 (𝑒^{𝑖 𝑡} - 1)$
Exponentiating, we get
$𝜙_{𝑋_{𝑛}} (𝑡) = exp 𝑛 log (1 + 𝑎_{𝑛}) \to exp 𝜆 (𝑒^{𝑖 𝑡} - 1)$
But by part (a),
$exp 𝜆 (𝑒^{𝑖 𝑡} - 1) = 𝜙_{𝑌} (𝑡)$
Thus for every $𝑡 \in ℝ$ ,
$𝜙_{𝑋_{𝑛}} (𝑡) \to 𝜙_{𝑌} (𝑡)$
Since $𝜙_{𝑌}$ is the characteristic function of $𝑌 \sim Pois (𝜆)$ , by the uniqueness theorem for characteristic functions, this implies
$𝑋_{𝑛} \overset{𝑑}{\to} 𝑌$

□

Problem 3

Show that

lim_{𝑛 \to \infty} \int_{0}^{𝑛 / 2} \frac{2^{𝑛}}{(𝑛 - 1)!} 𝑡^{𝑛 - 1} 𝑒^{- 2 𝑡} 𝑑 𝑡 = \frac{1}{2}

Hint: Observe that the integral is the probability of an event related to a Gamma distribution. Can we apply the central limit theorem?

Proof

Let

𝐼_{𝑛} ≔ \int_{0}^{𝑛 / 2} \frac{2^{𝑛}}{(𝑛 - 1)!} 𝑡^{𝑛 - 1} 𝑒^{- 2 𝑡} 𝑑 𝑡

Then

𝑓_{𝑛} (𝑡) = \frac{2^{𝑛}}{(𝑛 - 1)!} 𝑡^{𝑛 - 1} 𝑒^{- 2 𝑡} 𝟏_{(0, \infty)} (𝑡)

is the density of a Gamma distribution with parameters $(𝑛, 2)$ , that is, $𝑇_{𝑛} \sim Γ (𝑛, 2)$ . Hence

𝐼_{𝑛} = ℙ (𝑇_{𝑛} \leq 𝑛 / 2)

Now let $𝑋_{1}, 𝑋_{2}, \dots$ be i.i.d. random variables with

𝑋_{𝑖} \sim Exp (2)

We know that

𝑇_{𝑛} = 𝑋_{1} + \dots + 𝑋_{𝑛}

Also,

𝜇 ≔ 𝔼 [𝑋_{1}] = \frac{1}{2}, 𝜎^{2} ≔ Var (𝑋_{1}) = \frac{1}{4}

Therefore,

𝐼_{𝑛} = ℙ (𝑋_{1} + \dots + 𝑋_{𝑛} \leq 𝑛 / 2) = ℙ (\frac{𝑇_{𝑛} - 𝑛 𝜇}{𝜎 \sqrt{𝑛}} \leq 0)

Since $𝜇 = 1 / 2$ and $𝜎 = 1 / 2$ , this is

𝐼_{𝑛} = ℙ (\frac{𝑇_{𝑛} - 𝑛 / 2}{\sqrt{𝑛} / 2} \leq 0)

By the Central Limit Theorem,

\frac{𝑇_{𝑛} - 𝑛 𝜇}{𝜎 \sqrt{𝑛}} = \frac{𝑇_{𝑛} - 𝑛 / 2}{\sqrt{𝑛} / 2} \overset{𝑑}{\to} 𝑍, 𝑍 \sim 𝑁 (0, 1)

Hence, since the standard normal distribution function is continuous at $0$ ,

lim_{𝑛 \to \infty} 𝐼_{𝑛} = lim_{𝑛 \to \infty} ℙ (\frac{𝑇_{𝑛} - 𝑛 / 2}{\sqrt{𝑛} / 2} \leq 0) = ℙ (𝑍 \leq 0) = \frac{1}{2}

Thus

lim_{𝑛 \to \infty} \int_{0}^{𝑛 / 2} \frac{2^{𝑛}}{(𝑛 - 1)!} 𝑡^{𝑛 - 1} 𝑒^{- 2 𝑡} 𝑑 𝑡 = \frac{1}{2}

□

Problem 4

A casino offers the following random game: A player rolls a fair die once. If the outcome is 2 or 4, then the player wins 3 euros from the casino. If the outcome is $1, 3, 5$ , then the player loses 4 euros to the casino. If the outcome is 6 , then the player neither wins nor loses. If 90 players play the above game independently, find approximately the probability that the casino wins at least 30 euros in total.

Solution

Let $𝑋_{𝑖}$ be the gain of the casino from the $𝑖$ -th player, for $𝑖 = 1, \dots, 90$ . Then the random variables $𝑋_{1}, \dots, 𝑋_{90}$ are independent and identically distributed, with

𝑋_{𝑖} = {\begin{cases} - 3, & if the player wins 3 euros \\ 4, & if the player loses 4 euros \\ 0, & if the outcome is 6 \end{cases}

Since the die is fair, we have

ℙ (𝑋_{𝑖} = - 3) = \frac{2}{6} = \frac{1}{3}, ℙ (𝑋_{𝑖} = 4) = \frac{3}{6} = \frac{1}{2}, ℙ (𝑋_{𝑖} = 0) = \frac{1}{6}

Let

𝑆_{90} = 𝑋_{1} + \dots + 𝑋_{90}

be the total gain of the casino after 90 players. We want to approximate

ℙ (𝑆_{90} \geq 30)

We first compute the mean and variance of $𝑋_{1}$ . The mean is

𝜇 ≔ 𝔼 [𝑋_{1}] = (- 3) \cdot \frac{1}{3} + 4 \cdot \frac{1}{2} + 0 \cdot \frac{1}{6} = - 1 + 2 = 1

Also,

𝔼 [𝑋_{1}^{2}] = 9 \cdot \frac{1}{3} + 16 \cdot \frac{1}{2} + 0 = 3 + 8 = 11

Hence

𝜎^{2} ≔ Var (𝑋_{1}) = 𝔼 [𝑋_{1}^{2}] - 𝜇^{2} = 11 - 1 = 10

Therefore,

𝔼 [𝑆_{90}] = 90 𝜇 = 90, Var (𝑆_{90}) = 90 𝜎^{2} = 900

So the standard deviation of $𝑆_{90}$ is

\sqrt{900} = 30

By the Central Limit Theorem,

\frac{𝑆_{90} - 90}{30} \approx 𝑁 (0, 1)

Thus,

ℙ (𝑆_{90} \geq 30) = ℙ (\frac{𝑆_{90} - 90}{30} \geq \frac{30 - 90}{30}) \approx ℙ (𝑍 \geq - 2)

where $𝑍 \sim 𝑁 (0, 1)$ . Since

ℙ (𝑍 \geq - 2) = ℙ (𝑍 \leq 2) \approx 0.9772

we conclude that

ℙ (𝑆_{90} \geq 30) \approx 0.9772

Hence, the probability that the casino wins at least 30 euros in total is approximately 0.9772.

Problem 5

Assume that ${(𝑋_{𝑛})}_{𝑛 \in ℕ}$ is an i.i.d. sequence of random variables such that $𝔼 [𝑋_{1}] = 0$ and $𝔼 [𝑋_{1}^{2}] = 1$ . Show that

\frac{\sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖}}{\sqrt{\sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖}^{2}}} \overset{𝑑}{\to} 𝑍, 𝑍 \sim 𝑁 (0, 1)

Proof

Let

𝑆_{𝑛} ≔ \sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖}, 𝑄_{𝑛} ≔ \sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖}^{2}

We want to show that

\frac{𝑆_{𝑛}}{\sqrt{𝑄_{𝑛}}} \overset{𝑑}{\to} 𝑍, 𝑍 \sim 𝑁 (0, 1)

First, since $(𝑋_{𝑛})_{𝑛 \in ℕ}$ are i.i.d. with $𝔼 [𝑋_{1}] = 0,$ and $𝔼 [𝑋_{1}^{2}] = 1$ , we have $Var (𝑋_{1}) = 1$ . And thus by the Central Limit Theorem,

\frac{𝑆_{𝑛}}{\sqrt{𝑛}} \overset{𝑑}{\to} 𝑍, 𝑍 \sim 𝑁 (0, 1)

And then, consider $𝑄_{𝑛}$ . Since $(𝑋_{𝑖}^{2})$ are i.i.d. and $𝔼 [𝑋_{1}^{2}] = 1 < \infty$ , by the Law of Large Numbers we have

\frac{𝑄_{𝑛}}{𝑛} \overset{𝑃}{\to} 1

By continuity of the square root function,

\sqrt{\frac{𝑄_{𝑛}}{𝑛}} \overset{𝑃}{\to} 1

Write

\frac{𝑆_{𝑛}}{\sqrt{𝑄_{𝑛}}} = \frac{𝑆_{𝑛}}{\sqrt{𝑛}} \cdot \frac{1}{\sqrt{𝑄_{𝑛} / 𝑛}}

Define

𝐴_{𝑛} ≔ \frac{𝑆_{𝑛}}{\sqrt{𝑛}}, 𝐵_{𝑛} ≔ \frac{1}{\sqrt{𝑄_{𝑛} / 𝑛}}

Then we have shown that $𝐴_{𝑛} \overset{𝑑}{\to} 𝑍$ and $𝐵_{𝑛} \overset{𝑃}{\to} 1$ .

Now we claim that: $𝐴_{𝑛} 𝐵_{𝑛} \overset{𝑑}{\to} 𝑍$ .

It suffices to show that $𝔼 [𝑓 (𝐴_{𝑛} 𝐵_{𝑛})] \to 𝔼 [𝑓 (𝑍)]$ for every bounded continuous function $𝑓 : ℝ \to ℝ$ .

Let $𝑓 : ℝ \to ℝ$ be bounded and continuous. Then

𝔼 [𝑓 (𝐴_{𝑛} 𝐵_{𝑛})] - 𝔼 [𝑓 (𝑍)] = 𝔼 [𝑓 (𝐴_{𝑛} 𝐵_{𝑛})] - 𝔼 [𝑓 (𝐴_{𝑛})] + 𝔼 [𝑓 (𝐴_{𝑛})] - 𝔼 [𝑓 (𝑍)]

Since $𝐴_{𝑛} \overset{𝑑}{\to} 𝑍$ , the second term converges to $0$ . It remains to show that $𝔼 [𝑓 (𝐴_{𝑛} 𝐵_{𝑛})] - 𝔼 [𝑓 (𝐴_{𝑛})] \to 0$ .

Observe $𝐴_{𝑛} 𝐵_{𝑛} - 𝐴_{𝑛} = 𝐴_{𝑛} (𝐵_{𝑛} - 1)$ . Because $𝐴_{𝑛} \overset{𝑑}{\to} 𝑍$ , the sequence $(𝐴_{𝑛})$ is tight. Also, since $𝐵_{𝑛} \overset{𝑃}{\to} 1$ , we have

𝐵_{𝑛} - 1 \overset{𝑃}{\to} 0

It follows that

𝐴_{𝑛} (𝐵_{𝑛} - 1) = 𝐴_{𝑛} 𝐵_{𝑛} - 𝐴_{𝑛} \overset{𝑃}{\to} 0

We now show that

𝑓 (𝐴_{𝑛} 𝐵_{𝑛}) - 𝑓 (𝐴_{𝑛}) \overset{𝑃}{\to} 0

Fix $𝜀 > 0$ . Since $(𝐴_{𝑛})$ is tight, there exists $𝑀 > 0$ such that

sup_{𝑛 \geq 1} ℙ (| 𝐴_{𝑛} | > 𝑀) < 𝜀

Since $𝑓$ is continuous on the compact interval $[- 𝑀 - 1, 𝑀 + 1]$ , it is uniformly continuous there. Thus there exists $𝛿 > 0$ such that whenever $𝑥, 𝑦 \in [- 𝑀 - 1, 𝑀 + 1]$ and $| 𝑥 - 𝑦 | < 𝛿$ , we have

| 𝑓 (𝑥) - 𝑓 (𝑦) | < 𝜀

Now on the event

{| 𝐴_{𝑛} | \leq 𝑀, | 𝐴_{𝑛} 𝐵_{𝑛} - 𝐴_{𝑛} | < min (𝛿, 1)}

we also have $| 𝐴_{𝑛} 𝐵_{𝑛} | \leq 𝑀 + 1$ , so

| 𝑓 (𝐴_{𝑛} 𝐵_{𝑛}) - 𝑓 (𝐴_{𝑛}) | < 𝜀

Therefore,

ℙ | 𝑓 (𝐴_{𝑛} 𝐵_{𝑛}) - 𝑓 (𝐴_{𝑛}) | > 𝜀 \leq ℙ (| 𝐴_{𝑛} | > 𝑀) + ℙ (| 𝐴_{𝑛} 𝐵_{𝑛} - 𝐴_{𝑛} | \geq min (𝛿, 1))

The first term is less than $𝜀$ , and the second term tends to $0$ . Hence

𝑓 (𝐴_{𝑛} 𝐵_{𝑛}) - 𝑓 (𝐴_{𝑛}) \overset{𝑃}{\to} 0

Since $𝑓$ is bounded, the random variables $𝑓 (𝐴_{𝑛} 𝐵_{𝑛}) - 𝑓 (𝐴_{𝑛})$ are uniformly bounded. Therefore,

𝔼 [𝑓 (𝐴_{𝑛} 𝐵_{𝑛}) - 𝑓 (𝐴_{𝑛})] \to 0

Combining this with $𝔼 [𝑓 (𝐴_{𝑛})] \to 𝔼 [𝑓 (𝑍)]$ , we get

𝔼 [𝑓 (𝐴_{𝑛} 𝐵_{𝑛})] \to 𝔼 [𝑓 (𝑍)]

Thus

𝐴_{𝑛} 𝐵_{𝑛} \overset{𝑑}{\to} 𝑍

This finishes the proof that

\frac{𝑆_{𝑛}}{\sqrt{𝑄_{𝑛}}} = 𝐴_{𝑛} 𝐵_{𝑛} \overset{𝑑}{\to} 𝑍

That is,

\frac{\sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖}}{\sqrt{\sum_{𝑖 = 1}^{𝑛} 𝑋_{𝑖}^{2}}} \overset{𝑑}{\to} 𝑍, 𝑍 \sim 𝑁 (0, 1)

□