outcome_A <- sample(LETTERS, 1)

Question: What is the probability that outcome_A = Q?

• There are many random processes where all the possible outcomes are equally likely.

• There are many random processes where all the possible outcomes are equally likely.

• In these settings, \[ P(\text{Outcome}) = \frac{1}{\text{number of possible outcomes}} \]

• There are many random processes where all the possible outcomes are equally likely.

• In these settings, \[ P(\text{Outcome}) = \frac{1}{\text{number of possible outcomes}} \]

• Example:

\[ P(\text{Randomly selected letter from English alphabet}) = \frac{1}{\text{26}} \]

• There are many random processes where all the possible outcomes are equally likely.

• Informally, outcomes of this type are often called sample outcomes or urn outcomes.

• More formally, outcomes of this type are sometimes called discrete uniform random variables.

• There are many random processes where all the possible outcomes are equally likely.

• Informally, outcomes of this type are often called sample outcomes or urn outcomes.

• More formally, outcomes of this type are sometimes called discrete uniform random variables.

• There are many random processes where the possible outcomes are constructed from a sequence of urn outcomes (or discrete uniform random variables).

• There are many random processes where the possible outcomes are constructed from a sequence of urn outcomes (or discrete uniform random variables).

three_letter_word <- c(
    sample(LETTERS, 1)
  , sample(LETTERS, 1)
  , sample(LETTERS, 1)
)
three_letter_word

[1] “D” “T” “B”

• What is the probability that the sequence of outcomes spells “TOM”?

• There are many random processes where the possible outcomes are constructed from a sequence of urn outcomes (or discrete uniform random variables).

three_letter_word <- c(
    sample(LETTERS, 1)
  , sample(LETTERS, 1)
  , sample(LETTERS, 1)
)

• What is the probability that the sequence of outcomes spells TOM?

• What if

another_three_letter_word <- sample(LETTERS, 3)

What is the probability that another_three_letter_word = TOM?

In order to calculate

\[ \begin{array}{l} P(\texttt{three_letter_word = TOM}) = \\ \phantom{ = }\phantom{ = } \frac{1}{\text{Number of 3 letter sequences}} \end{array} \]

we need need to calculate the number of 3 letter sequences.

Q1: How many sequences of \(K\) cards from \(N\) are there?

Q1.a: with replacement

three_letter_word <- c(
    sample(LETTERS, 1)
  , sample(LETTERS, 1)
  , sample(LETTERS, 1)
)

Q1.b: without replacement

another_three_letter_word <- sample(LETTERS, 3)

Is the outcome TOM the same as MOT or OTM?

Is the outcome TOM the same as MOT or OTM?

Sequence: Order matters TOM \(\neq\) MOT

Hand: Order doesn’t matter TOM = MOT

Q2: How many hands of \(K\) cards from \(N\) are there?

Q2.a: with replacement

Q2.b: without replacement

Tool: Tree

A systematic way to write out all sequences.

Each path represents a sequence

The total number of paths is the product of the number of candidate cards at each draw

The total number of paths is the product of the number of candidate cards at each draw

\[ \underbrace{N \times N \times \cdots \times N}_{K \text{ draws}} = N^K \]

The total number of paths is the product of the number of candidate cards at each draw

Sequences without replacement?

How many possible sequences are there?

Enumerate possible sequences with a tree

The total number of sequences is the product of the number of candidate cards at each draw

\[ \underbrace{N \times (N-1) \times \cdots \times (N-K+1)}_{K \text{ draws}} = \frac{N!}{(N-K)!} \]

\[N! = N\times(N-1)\times(N-2)\times \cdots \times 3 \times 2 \times 1\]

\[ 5! = 5 \times 4 \times 3 \times 2 \times 1\]

In R:

factorial(5)

## [1] 120

Note

\[ 0! = 1 \]

• Total number of sequences is \(N^K\) when drawing with replacement
• Total number of sequences is \(N!/(N-K)!\) when drawing without replacement

• In a sequence of draws, order matters:

\[ABC \neq CAB\]

• In a hand (or sample) of draws, order does NOT matter:

\[ABC = CAB\]

Q2: How many hands of \(K\) cards from \(N\) are there?

Q2.a: with replacement (We’ll come back to this)

Q2.b: without replacement (Start here)

Go back to all the sequences:

Which are duplicates

• For each hand there are a certain number of duplicate sequences

• For each hand, how many sequences are there?

• For each hand, how many sequences are there?
• Hint: Think of the hand as a mini deck.

• For each hand, how many sequences are there?
• Hint: Think of the hand as a mini deck.
• How many sequences of \(K\) cards from a deck of \(K\) cards are there, without replacement?

• For each hand, how many sequences are there?
• Hint: Think of the hand as a mini deck.
• How many sequences of \(K\) cards from a deck of \(K\) cards are there, without replacement?
• \(K\times(K-1)\times \cdots \times 2 \times 1\)

• \(K\) cards from \(N\) without replacement has \(N!/(N-K)!\) possible sequences
• For each hand there are \(K!\) sequences (The hand to sequence multiplier)

• To get the number of hands, divide the number of sequences by the multiplier.

\[\frac{\text{Number sequences}}{\text{hand to sequence multiplier}} = \frac{\frac{N!}{(N-K)!}}{K!} = \frac{N!}{(N-K)!K!} \]

\[ {N \choose K} = \frac{N!}{(N-K)!K!} \]

The number of hands of size \(K\) from a deck of \(N\) cards when drawing without replacement.

In R

#The number of 5 card hands from a 52 card deck
choose(52,5)

## [1] 2598960

• A process or experiment that generates a binary outcome (0 or 1; heads or tails; success or failure)

• Successive replications of the process are independent

• The probability \(P(outcome = 1)\) is constant

• Notation:

  • \(p = P(outcome = 1)\)
  • \(q = (1-p) = P(outcome = 0)\)

\[ Success,\ Success,\ Failure,\ Success\] \[ 1,\ 0,\ 1,\ 1,\ 1,\ 0 \] \[tails,\ tails,\ tails,\ heads\]

Note: A Bernoulli sequence can be thought of as \(K\) draws with replacement from a deck of \(N=2\) cards.

Because successive outcomes are independent and \(p\) is constant,

\[ \begin{align*}P(1,\ 0,\ 1,\ 1,\ 1,\ 0) &= P(1)P(0)P(1)P(1)P(1)P(0) \\ & = p(1-p)ppp(1-p) \\ & = p^4(1-p)^2 \end{align*} \]

• The number of successes in a Bernoulli sequence

• Bernoulli properties still apply: independent outcomes, constant probability

• Notation:

  • \(p\) = probability of success in a single Bernoulli replicate
  • \(N\) = number of replicates in Bernoulli sequence

• For a sequence of 10 coin flips, what are the possible number of heads?

• For a sequence of 10 coin flips, what are the possible number of heads?
  • 0, 1, 2, 3, …, 10

• How do we calculate P(2 heads in 4 flips)?

• How do we calculate P(2 heads in 4 flips)?
  • If the sequence was \(H,\ H,\ T,\ T\), then \[P(H,\ H,\ T,\ T) = p^2(1-p)^2\]

• How do we calculate P(2 heads in 4 flips)?
  • If the sequence was \(H,\ H,\ T,\ T\), then \[P(H,\ H,\ T,\ T) = p^2(1-p)^2\]
  • If the sequence was \(T,\ T,\ H,\ H\), then \[P(T,\ T,\ H,\ H) = p^2(1-p)^2\]

• How do we calculate P(2 heads in 4 flips)?
  • We could list all possible 4 flip sequences …

• How do we calculate P(2 heads in 4 flips)?
  • We could list all possible 4 flip sequences …
  • Identify all the sequences that have 2 heads …

• How do we calculate P(2 heads in 4 flips)?
  • We could list all possible 4 flip sequences …
  • Identify all the sequences that have 2 heads …

\[\begin{align*}P(\text{2 heads}\ & \text{in 4 flips}) =\\ P(&\text{HHTT or HTHT or HTTH or} \\ &\text{THHT or THTH or TTHH})\end{align*}\]

• How do we calculate P(2 heads in 4 flips)?
  • We could list all possible 4 flip sequences …
  • Identify all the sequences that have 2 heads …
  • Because the sequences are mutually exclusive:

\[\begin{align*}P(\text{2 heads in 4 flips}) &= P(HHTT)\\& + P(HTHT)\\& + P(HTTH)\\& + P(THHT)\\& + P(THTH)\\& + P(TTHH) \end{align*}\]

• How do we calculate P(2 heads in 4 flips)?
  • We could list all possible 4 flip sequences …
  • Identify all the sequences that have 2 heads …
  • Because the sequences are mutually exclusive:

\[\begin{align*}P(\text{2 heads in 4 flips}) &= p^2(1-p)^2\\& + p^2(1-p)^2\\& + p^2(1-p)^2\\& + p^2(1-p)^2\\& + p^2(1-p)^2\\& + p^2(1-p)^2 \end{align*}\]

• How do we calculate P(2 heads in 4 flips)?
  • We could list all possible 4 flip sequences …
  • Identify all the sequences that have 2 heads …
  • Because the sequences are mutually exclusive:

\[P(\text{2 heads in 4 flips}) = 6\ p^2(1-p)^2\]

• How do we calculate \(P(X \text{ heads in } N \text{ flips})\)?

\[\begin{align*}P(X &\text{ heads in } N \text{ flips}) = \\ &\text{[Number of sequences with X heads]} \times p^X(1-p)^{N-X}\end{align*}\]

• How do we calculate \(\text{[Number of sequences with X heads in N flips]}\)?

• How do we calculate \(\text{[Number of sequences with X heads in N flips]}\)?
  • Trick: Consider a deck of cards numbered 1 to N
    • Let the number denote location in the sequence

• How do we calculate \(\text{[Number of sequences with X heads in N flips]}\)?
  • Trick: Consider a deck of cards numbered 1 to N
    • Let the number denote location in the sequence
    • Deal X cards

• How do we calculate \(\text{[Number of sequences with X heads in N flips]}\)?
  • Trick: Consider a deck of cards numbered 1 to N
    • Let the number denote location in the sequence
    • Deal X cards
    • Create a sequence by placing heads at the locations indicated by the cards
    • Place tails in all the other locations

\[ \text{cards: } [2],\ [1] \to \{\text{heads}, \text{heads}, \text{tails}, \text{tails}\} \] \[ \text{cards: } [2],\ [4] \to \{\text{tails}, \text{heads}, \text{tails}, \text{heads}\} \]

• Does the order of the cards matter?

\[ \text{cards: } [2],\ [1] \to \{\text{heads}, \text{heads}, \text{tails}, \text{tails}\} \] \[ \text{cards: } [1],\ [2] \to \{\text{heads}, \text{heads}, \text{tails}, \text{tails}\} \] \[ \text{cards: } [2],\ [4] \to \{\text{tails}, \text{heads}, \text{tails}, \text{heads}\} \] \[ \text{cards: } [4],\ [2] \to \{\text{tails}, \text{heads}, \text{tails}, \text{heads}\} \]

• How do we calculate \(\text{[Number of sequences with X heads in N flips]}\)?

\[\text{Every sequence with X heads in N flips}\] \[\text{corresponds to}\] \[\text{A hand of X cards from a deck of N (without replacement)}\]

• How do we calculate \(\text{[Number of sequences with X heads in N flips]}\)?

\[\text{Every sequence with X heads in N flips}\] \[\text{corresponds to}\] \[\text{A hand of X cards from a deck of N (without replacement)}\]

We already know how to calculate the number of possible hands of X cards from a deck of N (without replacement)

• How do we calculate \(\text{[Number of sequences with X heads in N flips]}\)?

\[{N \choose X}\]

• How do we calculate \(P(X \text{ heads in } N \text{ flips})\)?

\[P(X \text{ heads in } N \text{ flips}) = {N \choose X} p^X(1-p)^{N-X}\]

• In R

# Probability 2 heads in 4 flips when p = 0.5
dbinom(2,4,.5)
# Probability 2 or fewer heads in 4 flips when p = 0.5
pbinom(2,4,.5)
# Pseudo-random draw of the number of heads in 10 flips when p = 0.75
rbinom(1,10,.75)

## [1] 0.375
## [1] 0.6875
## [1] 5

• Suppose one flips a coin until there are 5 heads

• Suppose one flips a coin until there are 5 heads
  • HHHHH
  • HTTHHTH
  • TTTTHTTTHTTTHTTHH
  • …

• The number of failures before the \(K^{th}\) success in a Bernoulli sequence is a negative binomial random variable

• Bernoulli properties still apply: independent outcomes, constant probability

• Notation:

  • \(p\) = probability of success in a single Bernoulli replicate
  • \(K\) = number of successes in Bernoulli sequence

• What are the possible values?

• What are the possible values?
  • 0, 1, 2, 3, …

• What is P(3 tails before 5th head)?

• What is P(3 tails before 5th head)?
  • [3 tails before 5th head] = [hand of 3 tails and 4 heads] and [heads]

• What is P(3 tails before 5th head)?
  • P[3 tails before 5th head] =
  \[\text{P[hand of 3 tails and 4 heads] P[heads]}\]

• What is P(3 tails before 5th head)?
  • P[3 tails before 5th head] = dbinom(4,7,p)*p

• In R

# Probability of 3 tails before 5th heads with a fair coin
dnbinom(3,5,.5)
# Probability of 3 or fewer tails before 5th heads with a fair coin
pnbinom(3,5,.5)
# Pseudo-random draw of number of tails before 5th heads with a fair coin
rnbinom(1,5,.5)

## [1] 0.1367188
## [1] 0.3632813
## [1] 5