Homework 6
Theory
The Fundamental Theorem of Calculus and Its Connection to Probability Distributions
The Fundamental Theorem of Calculus (FTC) is recognized as a cornerstone of mathematical analysis, linking differentiation and integration in a powerful and elegant way. This connection becomes especially significant in probability and statistics, where it provides the mathematical foundation for understanding continuous probability distributions and the relationship between probability density functions (PDFs) and cumulative distribution functions (CDFs).
The Fundamental Theorem of Calculus
The FTC establishes the inverse relationship between differentiation and integration and consists of two fundamental parts:
Part 1 (Evaluation Theorem)
If $F$ is an antiderivative of $f$ over an interval $[a,b]$, then:
\[\int_a^b f(x) , dx = F(b) - F(a)\]Part 2 (Fundamental Theorem of Calculus Proper)
If $f$ is continuous on $[a,b]$, then the function $F$ defined by:
\[F(x) = \int_a^x f(t) , dt\]is differentiable on $(a,b)$, and $F’(x) = f(x)$.
These principles are essential in probability theory because they describe how cumulative distribution functions and probability density functions relate through calculus operations.
Probability Density and Cumulative Distribution Functions
Definitions
Probability Density Function (PDF): For a continuous random variable $X$, the PDF $f(x)$ gives the relative likelihood density at any point $x$. While it doesn’t provide the probability of $X$ taking any single value, it defines how probability is distributed across the range of $X$.
Cumulative Distribution Function (CDF): The CDF $F(x)$ represents the probability that $X \leq x$ and is defined as:
\[F(x) = P(X \leq x) = \int_{-\infty}^x f(t) , dt\]This represents the accumulated probability from the lower bound up to point $x$.
The Direct Connection: FTC Applied to Probability
The Fundamental Theorem of Calculus directly underlies the fundamental relationship between PDFs and CDFs:
Primary Relationships
- CDF as integral of PDF: \(F(x) = \int_{-\infty}^x f(t) , dt\)
- PDF as derivative of CDF: \(f(x) = F'(x) = \frac{dF(x)}{dx}\)
This is a direct application of the FTC, where the CDF plays the role of the antiderivative function, and the PDF represents the instantaneous rate of change of cumulative probability.
Fundamental Properties Demonstrated
The FTC enables us to prove essential properties of probability distributions:
Property 1 - Total Probability: \(\int_{-\infty}^{\infty} f(x) , dx = F(\infty) - F(-\infty) = 1 - 0 = 1\)
Property 2 - Probability of Intervals: \(P(a \leq X \leq b) = \int_a^b f(x) , dx = F(b) - F(a)\)
Property 3 - Monotonicity: Since $f(x) \geq 0$ for all $x$, we have $F’(x) = f(x) \geq 0$, which means $F(x)$ is non-decreasing.
Concrete Examples
Example 1: Exponential Distribution
Consider the exponential distribution with parameter $\lambda > 0$:
PDF: $f(x) = \lambda e^{-\lambda x}$ for $x \geq 0$, and $f(x) = 0$ for $x < 0$ CDF: We can find the CDF by integration: \(F(x) = \int_0^x \lambda e^{-\lambda t} , dt = \left[-e^{-\lambda t}\right]_0^x = 1 - e^{-\lambda x} \text{ for } x \geq 0\)
Verification using FTC:
- $F’(x) = \frac{d}{dx}(1 - e^{-\lambda x}) = \lambda e^{-\lambda x} = f(x)$ ✓
- $\int_0^{\infty} \lambda e^{-\lambda x} , dx = F(\infty) - F(0) = 1 - 0 = 1$ ✓
Example 2: Standard Normal Distribution
For the standard normal distribution with $\mu = 0$ and $\sigma = 1$:
PDF: \(f(x) = \frac{1}{\sqrt{2\pi}} e^{-\frac{x^2}{2}}\)
CDF: \(F(x) = \int_{-\infty}^x \frac{1}{\sqrt{2\pi}} e^{-\frac{t^2}{2}} , dt\)
According to the FTC, differentiating the CDF recovers the PDF:
\[\frac{d}{dx}F(x) = f(x)\]This demonstrates that the normal distribution’s density at any point represents the instantaneous rate of change of the probability accumulated up to that point.
Practical Implications and Applications
Statistical Modeling
Understanding the FTC-PDF-CDF relationship is foundational for:
- Likelihood Functions: In maximum likelihood estimation, we often need to switch between PDFs and CDFs
- Bayesian Inference: Prior and posterior distributions rely on this relationship
- Hypothesis Testing: Critical values are found using CDFs, while test statistics use PDFs
Risk Assessment and Predictive Modeling
- Quantile Functions: The inverse CDF (quantile function) helps determine risk thresholds
- Tail Probabilities: $P(X > x) = 1 - F(x)$ for assessing extreme events
- Expected Values: Computed using $E[X] = \int_{-\infty}^{\infty} x f(x) , dx$
Computational Considerations
In practice, many statistical software packages use numerical methods to:
- Approximate CDFs when analytical forms are unavailable
- Compute quantiles through numerical inversion of CDFs
- Generate random samples using the inverse transform method
Advanced Extensions
Multivariate Distributions
The relationship extends to joint distributions where:
\[f(x,y) = \frac{\partial^2 F(x,y)}{\partial x \partial y}\]Discrete-Continuous Mixtures
For mixed distributions, the relationship involves both derivatives and discrete jumps in the CDF.
Conclusion
The Fundamental Theorem of Calculus provides the essential mathematical foundation that connects the “rate of probability accumulation” (PDF) with the “total accumulated probability” (CDF) in continuous probability theory. This relationship is not merely theoretical—it’s the cornerstone that enables practical computation of probabilities, statistical inference, and risk assessment in continuous distributions.
By understanding this connection, we gain deeper insight into how probability flows through continuous sample spaces and can more effectively analyze, model, and make predictions about uncertain phenomena in the real world.