Chapter 1: Introduction

• Section 1.1 Equally Likely Outcomes
• Section 1.2 Interpretations
• Section 1.3 Distributions
• Section 1.4 Conditional Probability and Independence
• Section 1.5 Bayes Rule
• Section 1.6 Sequences of Events

Section 1.1 Equally Likely Outcomes

A. Basic Concepts and Terminology

• Conduct an experiment or observe a situation which has different possible random outcomes.
• Assume for simplicity that each outcome is equally likely.
• The set of all possible outcomes is called the outcome space (sometimes the sample space) and is denoted by the Greek letter omega  -- W.
• An event is any set of possible outcomes ( i.e. a subset of the outcome space).   Often use letters A, B, or C to denote events.

 

 
 
 
 
 
 
 
 
 

Two Examples:

1. Toss a coin once. Possible outcomes are H (heads) or T (tails). Outcome space = {H, T}

2. Toss a single six-sided die. Observe number of dots on face that comes up. Possible outcomes are 1, 2, 3, 4, 5, 6. Outcome space = {1, 2, 3, 4, 5, 6}
The event A = {2,4,6}  is the event rolling an even number -- a 2 or a 4 or a 6
 
 

• The probability of an event A, denoted by P(A), where the outcomes are equally likely, is defined to be
• P(A) = (number of outcomes in A) / (number of outcomes in the outcome space).
• See examples 1,2,3  pages 3, 4

B. Odds

• Assuming equally likely outcomes still, odds in favor of event A is (number of outcomes in A) to (number of outcomes not in A). Equivalently, it is the P(A) to 1 - P(A).
• Example: Roll die once. Odds of rolling a 1 or 2 = 2 to 4 (2:4 or 1:2)
• Fair Odds Rule: In a fair bet, the payoff odds equal the chance odds.
• Hence in the above example, a fair bet would be to pay $2 on a $1 bet to roll a 1 or a 2
• Obviously Las Vegas gambling games are never fair bets -- they favor the house.  See the example 4 on roulette.

Section 1.2 Interpretations

• Two interpretations of probability:

• Frequency: Approximations to long-run relative frequencies. Example: What is the likelihood (probability) that your child to be born will be a girl? (.487)
• Opinion: What is the probability of rain tomorrow? What is the probability that Pumas will win Saturday's game? What is the probability that O.J. did it? You have a 20% chance of contracting lung cancer.
• Relative Frequency: proportion of times something occurs in a sequence of observations

As the number of trials (observations) gets larger and larger, relative frequency of an outcome approaches the probability of that outcome.

Section 1.3 Distributions

A.  Review of Set Theory

Since we are concerned with events as subsets of the outcome space, we must understand the basics of set theory.

See page 19 of your text.   Make sure you understand all of the event language, set language, and notation, and Venn Diagrams  (Outcome space or universal set, event, intersection, union, empty set, partition, complement, inclusions (subset), disjoint sets.)

Venn Diagrams: Used to denote sets.

 

B.  Rules of Probability

Probability Distribution: A function P defined on the subsets of the outcome space which satisfies the following 3 rules:

• P(B) >= 0
• For a partition of a set B the probability of B is the sum of the probabilities of the sets in the partition.
• The probability of the outcome space is 1 (certain)

Complement Rule:

P(the complement of A) = 1 - P(A)

(Since A and A-complement form a partition of outcome space)
 

Difference Rule:

• If A is a subset of B, then P(B-A) = P(B)-P(A)
• If A is not a subset of B, then use P(B-A) = P(B) - P(B intersect A-complement)

Note: B-A = B intersected with A-complement
 

Inclusion-Exclusion Principle

• P(A union B) = P(A) + P(B) - P(AB)
• See the Venn Diagrams on page 22

Look at Example 1 page 23 (Rich and Famous)

Look at Example 2 page 23 (Numbered tickets in a box)

Look at Example 3 page 24 (Shape = "Rectangular-faced die" with "flat" sides numbered 1 and 6

Examples 2 and 3 represent same probability distribution.

 

Histograms for probability distributions: Area of bar over i is proportional to probability P(i)

 

(From Example 4) Consider the random experiment: Draw a ticket, replace it, and then draw again (called Sampling with Replacement). We show that the probability of drawing the ticket number i on the first draw and j on the second draw, P(i,j), is the product of the probabilities, P(i)P(j). Later we will learn that that means these two events of drawing the first ticket, and drawing the second ticket, are independent events.

 

A Few Important Named Distributions (more later)

• Bernoulli(p) distribution. Two outcomes (say 0 and 1). Often call one success and the other failure. Can think of as a biased coin toss where heads comes up with probability p

possible outcome	0	1
probability	p	1-p

• Uniform Distribution on a finite set. Each outcome equally likely. Example if tossing a fair die.
• Uniform(a,b) Distribution. Point picked at random on the interval (a,b) (Where a < b are real numbers). Each point is assumed to be equally likely to be selected. P(picking a point in the interval (x,y) where a <= x <= y <= b is (y-x)/(b-a). Many random number generators in computer languages or on calculators simulate the Uniform (0,1) distribution.
• Uniform distribution on (0,1) x (0,1) in the plane.

Section 1.4 Conditional Probability and Independence

• Consider Example 1: Toss coin 3 times.

 

 
 
 
 
 
 
 
 
 

A = event two or more heads occur in 3 tosses = {hhh, hht, hth, thh}
H = event that heads occurs on first toss = {hhh, hht, hth, htt}
P(A|H) = probability that A occurs given that H occurs = 3/4 since out of 4 outcomes in H , 3 are in A.
Similarly, P(A|T) = 1/4
 

• Counting formula for conditional probability P(A|B), in a finite set of equally likely outcomes:
• P(A|B) = #(AB)/#(B)
• General formula for conditional probability (use when outcomes not necessarily equally likely):
• P(A|B) = P(AB)|P(B)
It is easy to show that the counting formula is a special case of the general formula.

Tree Diagrams and the Multiplication Rule

• We can rearrange the General formula for conditional probability to get

Multiplication Rule:
• P(AB) = P(B)P(A|B)
This says that the chance of B followed by A is the chance of B times the chance of A given B
 
• Tree Diagrams such as in example 5 and example 6, pages 37, 38 show how to use the rule to calculate joint  probabilities using known conditional probabilities.

 

 
 
 
 
 
 
 

1.  Set up the tree representing representing the stages in a two (or more) stage experiment.
2.  Label the branches with the probabilities.  (First stage -- P(A),  second P(A|B) )
3.  To calculate the joint probability represented by any path, multiply the probabilities along that path ( P(AB) = P(A|B)P(B) )
4.  Two calculate the probability of a particular event, select the paths that are part of that event and add their probabilities.
 

• Averaging Conditional Probabilities

We know that A = A B union A B^c and this union is disjoint.  Hence we have:
 
•      P(A) = P(A B) + P(A B^c)
We can apply the multiplication rule to the each term in the right hand side to get
 
•     P(A) = P(A|B)P(B) + P(A|B^c)P(B^c)
This can be generalized to a partition of the outcome space, B₁, B₂, ..., B_n
•   P(A) = P(A|B₁ )P(B₁)+ P(A|B₂)P(B₂ )+ ... +P(A| B_n )P(B_n)
We use this sometimes when  we know the conditional probabilities on the right hand side and need to find the probability of A.
 
• Independence
• Two events A and B are independent if the probability of A occurring is not influenced by the probability of B occurring:

 P(A|B) = P(A)    ( or similarly P(B|A) = P(B) )
• Multiplication Rule:  We can also equivalentlly write two events are independent if
P(AB) = P(A)P(B)
• Note:  If two events are independent, their compliments are also independent!

Section 1.5  Bayes Rule -- Reverse Conditional Probabilities

• Bayes' Rule is used to calculate the  posterior probability,  P(B|A),  when we know P(A|B) and P(A)
• Derivation:

We know the following two formulas hold:
 
1. P(B|A) = P(BA) / P(A)
2. P(AB)  = P(A|B)P(B)
Substituting the right hand side of the second for the numerator on the right hand of the first formula, we get

         P(B|A) = P(A|B)P(B) / P(A)

Sometimes the denominator on the right hand side,   P(A),   will be calculated using the averaging formula:
 

• P(A) = P(A|B)P(B) + P(A|B^c)P(B^c)
• False positives example 3 (page 50)

 
1.  Set up the tree diagram with the first stage outcomes being diseased (D) or not diseased ( D^c ).  and the second stage being test is positive (+) or test is negative (-).

2.  From the tree diagram calculate
              P(D|+) = P(D +) / P( + )  = .95 * .01 /( .95 * .01 + .99 * .02) = approx  32%

• Diagnosis of a particular patient example 4 (page 52)

 
Suppose a patient is examined by a doctor, and the doctor has the opinion that there is a 30% change that particular patient has the disease.  This revises the tree diagram calculations since P(D) = .30 and P(D^c) = .70 now.   Revising the calculation we get:

P(D|+) = P(D +) / P( + )  = .95 * .30 /( .95 * .30 + .70 * .02) = approx 95%
 

 
• May need to calculate probabilities of events determined by a sequence of outcomes.  (Multistage experiments).
• This is done by just repeatedly applying the multiplication formula repeatedly.

 
• For a 3 stage experiment:

 
• P(ABC) = P(A)P(B|A)P(C|AB)
• This is easily generalized to an arbitrary number of events.

 
• Example 1 :  Completion of construction project in 3 stages.

 

 

P(all 3 on time) = P(C₁C₂C₃) =  P(C₁)P(C₂|C₁)P(C₃|C₁C₂) = .7 * .8 * .9 = .504

Example 4:  Probability of a flush. (all cards same suit)

5 - stage experiment.  Pick the first card, pick the second card, ..., pick the fifth card.

P(Spade flush) = (First Spade) * (Second Space | First Spade) *
                          (Third Spade | First and Second Spade) *
                          (Fourth Spade | First, Second, and Third Spade) *
                          (Fifth Spade | First, Second, Thirds, and Fourth Spade)
                       = (13/52)*(12/51)*(11/50)*(10/49)*(9/48)
P(Flush) = P(Spade flush) + P(Heart fllush) + P(Diamond Flush) + P(Club Flush)
             = 4 * P(Spade flush) = approx. .00198
 

• Example 5:  Birthday Surprise

What is the probability that at least two students in the class have the same birthday?
Use the "at least rule of thunb" -- easier to calculate probability of the complement
(none have the same birthday)
        P(at least two) = 1 - P(none have the same birthday)

Consider first student. Pick another student  -- Probability his/her birthday doesn't conflict with first  is (364/365).  Pick third student  -- He/she doesn't conflict with first two with probability P(363/365).  Continuing:

P(none  of n students have the same birthday) = (364/365)(363/365)...(365- (n-1))/365))

If n = 23,  This probability is approximately 50.6 % !
 

• Tree diagrams can be extended to three or more stage events (as long as tree doesn't get to unwieldy).

 
• Independence of  events can be extended to more than two events. This is a very strong condition.  Must have P(A|B) = P(A), P(C|AB) = P(C), etc.     The multiplication rule still holds.  P(ABC) = P(A)P(B)P(C).

 
• Pair wise independence is a weaker statement -- i.e. events can be pairwise independent but not independent.  In example 8, we have pair wise independence but not independence. They are not independent because P(S| H₁H₂) is not equal to P(S)