Probability and Its Axioms

February 27, 2025 · 6 min read · Page View:

If you have any questions, feel free to comment below.

This article is about the probability and its axioms.

Probability Theory #

Probability theory deals with the study of random phenomena, which under repeated experiments yield different outcomes that have certain underlying patterns about them.

the notion of an experiments: a set of repeatable conditions that allow any number of identical repetitions.

certain averages approach a constant value as the number of observations increases

Observation deduction and prediction #

definition #

Laplace’s classical definition #

The probability of an event A is defined a-priori without actual experimentation as $P(A)=\frac{\text{Number of outcomes favorable to }A}{\text{Total number of possible outcomes}}$

However, it still has some issues: eg Bertrand’s paradox.

relative frequency definition #

$P(A)=\lim_{n \to \infty} \frac{n_A}{n}$

Where $n_A$ is the number of occurrences of $A$ and $n$ is the total number of trials.

an useful example

Among 1 2 … n, the nums p, 2p … are divisible by p. Thus there are n/p such numbers between 1 and n. Hence:

$$
P{\text{a given number } N \text{ is divisible by a prime } p} = \lim_{n\rightarrow\infty} \frac{n/p}{n}=\frac{1}{p}. \ $$

set #

equal: two sets are declared to be equal if and only if they contain exactly the same elements, $A = B \Leftrightarrow A \subseteq B \land B \supseteq A$

subset: $A \subseteq B \Leftrightarrow \forall x \in A, x \in B$

superset: $A \supseteq B \Leftrightarrow B \subseteq A$

If a set consists of n elements, then the total number of its subsets equals 2^n (from empty to full)

cardinality #

|A| is a measure of how many different elements A has.

finiteness #

some infinite sets we’ve seen

• the set of all positive integers

cartesian products of sets #

$ \begin{equation} A\times B := {(a, b) \mid a\in A \land b\in B } \end{equation} $

eg. {a,b}x{1,2} = {(a,1),(a,2),(b,1),(b,2)}

• for finite A, B, |AxB|=|A||B|.

• Cartesian product is not commutative

transitivity #

$ \text{If } C\subseteq B \text{ and } B\subseteq A \text{ then } C\subseteq A $

equality #

$ \text{Equality: }A = B\text{ if }A\subseteq B\text{ and }B\subseteq A $

Operators #

Union #

$ A\cup B := {x \mid x\in A \text{ or } x\in B} $

$ \forall A, B: (A\cup B \supseteq A) \wedge (A\cup B \supseteq B) $

Intersection #

Formally, $\forall A,B: A\cap B\ (\text{or}=AB) \equiv {x \mid x\in A \land x\in B}$.

$\forall A, B:(A \cap B \subseteq A) \land (A \cap B \subseteq B)$

Properties:

• commutative: $A\cap B = B\cap A$
• associative: $(A\cap B)\cap C = A\cap (B\cap C)$
• distributive: $A\cap (B\cup C) = (A\cap B)\cup (A\cap C)$

Disjointedness #

$A$ and $B$ are disjoint if $A\cap B = \emptyset$

eg. the set of even numbers and the set of odd numbers are disjoint.

Inclusion-Exclusion Principle #

$ |A\cup B| = |A| + |B| - |A\cap B| $

Set difference #

$ A-B ={x | x \in A \Lambda x \notin B} $

Set complement #

The universe of discourse can itself be considered a set, call it U.

The complement of A, written (\bar{A}) , is the complement of A. ie, U-A = $\overline{A}={x | x \notin A}$

Mutually exclusive #

$A$ and $B$ are mutually exclusive(M.E.) if $A\cap B = \emptyset$

A partition of $\Omega$ is a collection of mutually exclusive subsets of $\Omega$ such that their union is $\Omega$.

$A_{i} \cap A_{j}=\phi$, and $\bigcup_{i=1} A_{i}=\Omega$

Set identities #

• Identity: $A\cup \emptyset = A$, $A\cap U = A$
• Domination: $A\cup U = U$, $A\cap \emptyset = \emptyset$
• Idempotent: $A\cup A = A$, $A\cap A = A$
• Double complement: $\overline{\overline{A}} = A$
• Commutative: $A\cup B = B\cup A$, $A\cap B = B\cap A$
• Associative: $(A\cup B)\cup C = A\cup (B\cup C)$, $(A\cap B)\cap C = A\cap (B\cap C)$
• Distributive: $A\cup (B\cap C) = (A\cup B)\cap (A\cup C)$, $A\cap (B\cup C) = (A\cap B)\cup (A\cap C)$
• De Morgan’s laws: $\overline{A\cup B} = \overline{A}\cap \overline{B}$, $\overline{A\cap B} = \overline{A}\cup \overline{B}$
• Duality principle: $A\cup B = \overline{\overline{A}\cap \overline{B}}$, $A\cap B = \overline{\overline{A}\cup \overline{B}}$

If in a set identity, we replace all unions by intersections, all intersections by unions, and the sets U and $\emptyset$ by the sets $\emptyset$ and U, the identity will be preserved.

eg.

$A \cap (B \cup C) = A \cap B \cup A \cap C$ ==> $A \cup B \cap C=(A \cup B) \cap(A \cup C)$

$U \cup A = U$ ==> $Q \cap A=\emptyset$

• more examples: suppose A and B are not mutually exclusive, then $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ and $P(\overline A B) = P(B) - P(A B)$.

Field #

Field: A collection of subsets of a nonempty set $\Omega$ forms a field F if

full collection is $\Omega$.

• $\Omega \in F$
• if $A \in F$, then $\overline{A} \in F$
• if $A, B \in F$, then $A \cup B \in F$

σ-FIELD #

σ-Field (Definition): A field F is a σ-field if in addition to the three conditions of a field ((i) – (iii)), we have the following:

For every sequence $(A_{i}, i=1 \to \infty)$, For every sequence of pair wise disjoint events belonging to F, their union also belongs to F, i.e.,

$$ A=\bigcup_{i=1}^{\infty} A_{i} \in F . $$

Probability space #

S (U) or $\sigma$ is called the certain event, its elements experimental outcomes, and its subsets events.

• Impossible event: ${\emptyset}$
• Certain event: $\Omega$
• Elementary event ${e_i}$: an event which contains only a single outcome in the sample space.

The totality of all $E_i$, known a priori, constitutes a set $\Omega$, the set of all experimental outcomes.

$$ \Omega={\xi_{1}, \xi_{2}, \cdots, \xi_{k}, \cdots} $$

$\Omega$ has subsets $A, B, C, \cdots$，If A is a subset of $\Omega$, then $\xi_i \in A$ implies $\xi_i \in \Omega$.

Probability axioms #

• $P(A) \geq 0$
• $P(\Omega) = 1$
• $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
• If $A_i$ are pair wise mutually exclusive, then $P\left(\bigcup_{n=1}^{\infty} A_{n}\right)=\sum_{n=1}^{\infty} P\left(A_{n}\right)$

Probability models #

• The triplet $(\Omega, F, P)$ form a probability model, where

• $\Omega$ is the sample space, not empty
• $F$ is the field of events, a field of subsets of $\Omega$
• $P$ is the probability measure

Conditional probability #

$$ P(A) \approx \frac{N_{A}}{N}, P(B) \approx \frac{N_{B}}{N}, P(A B) \approx \frac{N_{A B}}{N} . $$

Thus the ratio $\frac{N_{A B}}{N_{B}}=\frac{N_{A B} / N}{N_{B} / N}=\frac{P(A B)}{P(B)} = P(A|B)$, which is the conditional probability of $A$ given $B$.

If $A_i$ are pair wise disjoint, then

$$ P(B)=\sum_{i=1}^{n} P\left(B A_{i}\right)=\sum_{i=1}^{n} P\left(B | A_{i}\right) P\left(A_{i}\right) . $$

If A and B are independent, then $P(A|B) = P(A)$ and $P(B|A) = P(B)$.

Bayes’s theorem #

$$ P(A|B) = \frac{P(B|A)}{P(B)} P(A) $$

• P(A) represents the a-priori probability of the event A.

General version #

A set of mutually exclusive events with associated a-priori probabilities $P(A_{i}), i=1 \to n .$ With the new information B has occurred, the information about $A_i$ can be updated by the n conditional probabilities $P(B|A_{i})$

$$ P\left(A_{i} | B\right)=\frac{P\left(B | A_{i}\right) P\left(A_{i}\right)}{P(B)}=\frac{P\left(B | A_{i}\right) P\left(A_{i}\right)}{\sum_{i=1}^{n} P\left(B | A_{i}\right) P\left(A_{i}\right)}, $$

application #

eg. Two boxes B1 and B2 contain 100 and 200 light bulbs respectively. The first box (B1) has 15 defective bulbs and the second has 5. Suppose a box is selected at random and one bulb is picked out.

(a) What is the probability that it is defective?

Let D be the event that the bulb is picked out.

Then $P(D) = P(D|B1)P(B1) + P(D|B2)P(B2) = 0.15*0.5 + 0.025 * 0.5 = 0.0875$

(b) If it is defective, what is the probability that it came from box B1?

Let B1 be the event that the box is picked out.

Then $P(B1|D) = \frac{P(D|B1)P(B1)}{P(D)} = \frac{0.15*0.5}{0.0875} = 0.857$