The Law of Total Probability helps determine the probability of an event by considering all possible ways it can occur.
It states that if an event can occur through several mutually exclusive and exhaustive events, then its total probability is the sum of the probabilities of each possible way it can occur.
- It expresses the probability of an event in terms of conditional probabilities.
- It is applicable when the sample space is partitioned into mutually exclusive and exhaustive events.
If A₁, A₂, A₃, ..., Aₙ form a partition of the sample space S, then for any event B,
P(B) = \sum_{i=1}^{n} P(B \cap A_i)
= \sum_{i=1}^{n} P(B \mid A_i)\, P(A_i)
The figure shows that the sample space is divided into disjoint events A₁, A₂, and A₃. Event B overlaps with these partitions. Therefore, the probability of B is obtained by adding the probabilities of B occurring within each partition.
Proof
Since A₁, A₂, A₃, ... form a partition of the sample space S,
S=\bigcup_{i} A_i For any event B,
B = B \cap S
B = B \cap \left(\bigcup_{i} A_i\right)
B = \bigcup_{i}(B \cap A_i) by the distributive law.
Since the sets (B ∩ Aᵢ) are disjoint (because the Aᵢ's are disjoint), by the addition axiom of probability,
P(B)=P\!\left(\bigcup_{i}(B \cap A_i)\right)
P(B)=\sum_{i} P(B \cap A_i) Using the definition of conditional probability,
P(B \cap A_i)=P(B \mid A_i)\,P(A_i) Substituting,
P(B)=\sum_{i} P(B \mid A_i)\,P(A_i) Hence proved.
Note: The Law of Total Probability is used when the direct probability of an event is unknown, but the conditional probabilities under different mutually exclusive cases and the probabilities of those cases are known. It is also known as the Total Probability Theorem or the Law of Alternatives.
Applications
1. Bayes' Theorem
The Law of Total Probability is used to calculate the denominator in Bayes' Theorem. It helps determine the probability of a cause after observing an event.
Used in:
- Spam email detection
- Text classification
- Sentiment analysis
2. Expectation-Maximization (EM) Algorithm
The EM algorithm is used to estimate hidden (latent) variables in probabilistic models. The Law of Total Probability helps compute the probability of observed data by considering all possible hidden variables.
Used in:
- Gaussian Mixture Models (GMMs)
- Data clustering
3. Hidden Markov Models (HMMs)
In HMMs, the actual states are hidden and only observations are visible. The Law of Total Probability is used to calculate the probability of an observed sequence by considering all possible hidden states.
Used in:
- Speech recognition
- Part-of-Speech (POS) tagging
- Gene prediction
4. Bayesian Networks and Marginalization
The Law of Total Probability is used to find the probability of an event by considering all possible values of unknown variables. This process is known as marginalization.
Used in:
- Medical diagnosis
- Sensor networks
- Decision-making systems
➢Practice: Solved Examples
