Probability and Its Axioms
January 27, 2025 · 6 min read · Page View:
If you have any questions, feel free to comment below. Click the block can copy the code.
And if you think it's helpful to you, just click on the ads which can support this site. Thanks!
This article is about the probability and its axioms.
Probability Theory #
Probability theory deals with the study of random phenomena, which under repeated experiments yield different outcomes that have certain underlying patterns about them.
the notion of an experiments: a set of repeatable conditions that allow any number of identical repetitions.
certain averages approach a constant value as the number of observations increases
Observation deduction and prediction #
definition #
Laplace’s classical definition #
The probability of an event A is defined a-priori without actual experimentation as $P(A)=\frac{\text{Number of outcomes favorable to }A}{\text{Total number of possible outcomes}}$
However, it still has some issues: eg Bertrand’s paradox.
relative frequency definition #
$P(A)=\lim_{n \to \infty} \frac{n_A}{n}$
Where $n_A$ is the number of occurrences of $A$ and $n$ is the total number of trials.
an useful example
Among 1 2 … n, the nums p, 2p … are divisible by p. Thus there are n/p such numbers between 1 and n. Hence:
$$ \ P\{\text{a given number } N \text{ is divisible by a prime } p\} = \lim_{n\rightarrow\infty} \frac{n/p}{n}=\frac{1}{p}. \ $$set #
equal: two sets are declared to be equal if and only if they contain exactly the same elements, $A = B \Leftrightarrow A \subseteq B \land B \supseteq A$
subset: $A \subseteq B \Leftrightarrow \forall x \in A, x \in B$
superset: $A \supseteq B \Leftrightarrow B \subseteq A$
If a set consists of n elements, then the total number of its subsets equals 2^n (from empty to full)
cardinality #
|A| is a measure of how many different elements A has.
finiteness #
some infinite sets we’ve seen
- the set of all positive integers
cartesian products of sets #
$ \begin{equation} A\times B := {(a, b) \mid a\in A \land b\in B } \end{equation} $
eg. {a,b}x{1,2} = {(a,1),(a,2),(b,1),(b,2)}
for finite A, B, |AxB|=|A||B|.
Cartesian product is not commutative
transitivity #
$ \text{If } C\subseteq B \text{ and } B\subseteq A \text{ then } C\subseteq A $
equality #
$ \text{Equality: }A = B\text{ if }A\subseteq B\text{ and }B\subseteq A $
Operators #
Union #
$ A\cup B := {x \mid x\in A \text{ or } x\in B} $
$ \forall A, B: (A\cup B \supseteq A) \wedge (A\cup B \supseteq B) $
Intersection #
Formally, $\forall A,B: A\cap B\ (\text{or}=AB) \equiv {x \mid x\in A \land x\in B}$.
$\forall A, B:(A \cap B \subseteq A) \land (A \cap B \subseteq B)$
Properties:
- commutative: $A\cap B = B\cap A$
- associative: $(A\cap B)\cap C = A\cap (B\cap C)$
- distributive: $A\cap (B\cup C) = (A\cap B)\cup (A\cap C)$
Disjointedness #
$A$ and $B$ are disjoint if $A\cap B = \emptyset$
eg. the set of even numbers and the set of odd numbers are disjoint.
Inclusion-Exclusion Principle #
$ |A\cup B| = |A| + |B| - |A\cap B| $
Set difference #
$ A-B ={x | x \in A \Lambda x \notin B} $
Set complement #
The universe of discourse can itself be considered a set, call it U.
The complement of A, written \(\bar{A}\) , is the complement of A. ie, U-A = $\overline{A}={x | x \notin A}$
Mutually exclusive #
$A$ and $B$ are mutually exclusive(M.E.) if $A\cap B = \emptyset$
A partition of $\Omega$ is a collection of mutually exclusive subsets of $\Omega$ such that their union is $\Omega$.
$A_{i} \cap A_{j}=\phi$, and $\bigcup_{i=1} A_{i}=\Omega$
Set identities #
- Identity: $A\cup \emptyset = A$, $A\cap U = A$
- Domination: $A\cup U = U$, $A\cap \emptyset = \emptyset$
- Idempotent: $A\cup A = A$, $A\cap A = A$
- Double complement: $\overline{\overline{A}} = A$
- Commutative: $A\cup B = B\cup A$, $A\cap B = B\cap A$
- Associative: $(A\cup B)\cup C = A\cup (B\cup C)$, $(A\cap B)\cap C = A\cap (B\cap C)$
- Distributive: $A\cup (B\cap C) = (A\cup B)\cap (A\cup C)$, $A\cap (B\cup C) = (A\cap B)\cup (A\cap C)$
- De Morgan’s laws: $\overline{A\cup B} = \overline{A}\cap \overline{B}$, $\overline{A\cap B} = \overline{A}\cup \overline{B}$
- Duality principle: $A\cup B = \overline{\overline{A}\cap \overline{B}}$, $A\cap B = \overline{\overline{A}\cup \overline{B}}$
If in a set identity, we replace all unions by intersections, all intersections by unions, and the sets U and $\emptyset$ by the sets $\emptyset$ and U, the identity will be preserved.
eg.
$A \cap (B \cup C) = A \cap B \cup A \cap C$ ==> $A \cup B \cap C=(A \cup B) \cap(A \cup C)$
$U \cup A = U$ ==> $Q \cap A=\emptyset$
- more examples: suppose A and B are not mutually exclusive, then $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ and $P(\overline A B) = P(B) - P(A B)$.
Field #
Field: A collection of subsets of a nonempty set $\Omega$ forms a field F if
full collection is $\Omega$.
- $\Omega \in F$
- if $A \in F$, then $\overline{A} \in F$
- if $A, B \in F$, then $A \cup B \in F$
σ-FIELD #
σ-Field (Definition): A field F is a σ-field if in addition to the three conditions of a field ((i) – (iii)), we have the following:
For every sequence $(A_{i}, i=1 \to \infty)$, For every sequence of pair wise disjoint events belonging to F, their union also belongs to F, i.e.,
$$ A=\bigcup_{i=1}^{\infty} A_{i} \in F . $$Probability space #
S (U) or $\sigma$ is called the certain event, its elements experimental outcomes, and its subsets events.
- Impossible event: ${\emptyset}$
- Certain event: $\Omega$
- Elementary event ${e_i}$: an event which contains only a single outcome in the sample space.
The totality of all $E_i$, known a priori, constitutes a set $\Omega$, the set of all experimental outcomes.
$$ \Omega=\{\xi_{1}, \xi_{2}, \cdots, \xi_{k}, \cdots\} $$$\Omega$ has subsets $A, B, C, \cdots$，If A is a subset of $\Omega$, then $\xi_i \in A$ implies $\xi_i \in \Omega$.
Probability axioms #
- $P(A) \geq 0$
- $P(\Omega) = 1$
- $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
- If $A_i$ are pair wise mutually exclusive, then $P\left(\bigcup_{n=1}^{\infty} A_{n}\right)=\sum_{n=1}^{\infty} P\left(A_{n}\right)$
Probability models #
• The triplet $(\Omega, F, P)$ form a probability model, where
- $\Omega$ is the sample space, not empty
- $F$ is the field of events, a field of subsets of $\Omega$
- $P$ is the probability measure
Conditional probability #
$$ P(A) \approx \frac{N_{A}}{N}, P(B) \approx \frac{N_{B}}{N}, P(A B) \approx \frac{N_{A B}}{N} . $$Thus the ratio $\frac{N_{A B}}{N_{B}}=\frac{N_{A B} / N}{N_{B} / N}=\frac{P(A B)}{P(B)} = P(A|B)$, which is the conditional probability of $A$ given $B$.
If $A_i$ are pair wise disjoint, then
$$ P(B)=\sum_{i=1}^{n} P\left(B A_{i}\right)=\sum_{i=1}^{n} P\left(B | A_{i}\right) P\left(A_{i}\right) . $$If A and B are independent, then $P(A|B) = P(A)$ and $P(B|A) = P(B)$.
Bayes’s theorem #
$$ P(A|B) = \frac{P(B|A)}{P(B)} P(A) $$- P(A) represents the a-priori probability of the event A.
General version #
A set of mutually exclusive events with associated a-priori probabilities $P(A_{i}), i=1 \to n .$ With the new information B has occurred, the information about $A_i$ can be updated by the n conditional probabilities $P(B|A_{i})$
$$ P\left(A_{i} | B\right)=\frac{P\left(B | A_{i}\right) P\left(A_{i}\right)}{P(B)}=\frac{P\left(B | A_{i}\right) P\left(A_{i}\right)}{\sum_{i=1}^{n} P\left(B | A_{i}\right) P\left(A_{i}\right)}, $$application #
eg. Two boxes B1 and B2 contain 100 and 200 light bulbs respectively. The first box (B1) has 15 defective bulbs and the second has 5. Suppose a box is selected at random and one bulb is picked out.
(a) What is the probability that it is defective?
Let D be the event that the bulb is picked out.
Then $P(D) = P(D|B1)P(B1) + P(D|B2)P(B2) = 0.15*0.5 + 0.025 * 0.5 = 0.0875$
(b) If it is defective, what is the probability that it came from box B1?
Let B1 be the event that the box is picked out.
Then $P(B1|D) = \frac{P(D|B1)P(B1)}{P(D)} = \frac{0.15*0.5}{0.0875} = 0.857$
If you want to follow my updates, or have a coffee chat with me, feel free to connect with me: