Statistical Moments

Statistical moments are a family of summary statistics that describe the shape of a probability distribution. ToFUL computes the r-th moment of a random variable X about a reference point a, defined as:

\[\mu_r(a) = E\left[(X - a)^r\right]\]

Definition 

For a discrete random variable with PMF \(P(X = x)\):

\[\mu_r(a) = \sum_{x} (x - a)^r \cdot P(X = x)\]

For a continuous random variable with PDF \(f(x)\):

\[\mu_r(a) = \int_{-\infty}^{\infty} (x - a)^r \cdot f(x)\, dx\]

The choice of reference point a determines the type of moment.

Types of Moments 

Raw Moments (a = 0)

Setting \(a = 0\) gives the raw moments (also called moments about the origin):

\[\mu'_r = E[X^r]\]

The first few raw moments have direct interpretations:

Order	Raw moment	Interpretation
r = 1	\(\mu'_1\)	Mean mu = E[X]
r = 2	\(\mu'_2\)	E[X^2] – used to compute variance
r = 3	\(\mu'_3\)	E[X^3]
r = 4	\(\mu'_4\)	E[X^4]

Central Moments (a = μ)

Setting \(a = \mu\) gives the central moments:

\[\mu_r = E\left[(X - \mu)^r\right]\]

Central moments measure deviations from the mean and are therefore invariant to shifts in the distribution. Key central moments:

Order	Central moment	Interpretation
r = 1	\(\mu_1 = 0\)	Always zero by definition of mean
r = 2	\(\mu_2\)	Variance sigma^2
r = 3	\(\mu_3\)	Raw material for skewness
r = 4	\(\mu_4\)	Raw material for kurtosis

Moments about a Custom Point (a = c)

For any constant \(c\), the moment about \(c\) interpolates between raw moments (c = 0) and central moments (c = μ). This is useful when a distribution’s natural reference point differs from both.

Relationships Between Moment Types 

Central moments can be expressed in terms of raw moments using the binomial theorem:

\[\mu_r = \sum_{k=0}^{r} \binom{r}{k} (-\mu)^{r-k} \mu'_k\]

The first few expansions:

\[\mu_2 = \mu'_2 - (\mu'_1)^2 = E[X^2] - (E[X])^2\]

\[\mu_3 = \mu'_3 - 3\mu'_2\mu'_1 + 2(\mu'_1)^3\]

\[\mu_4 = \mu'_4 - 4\mu'_3\mu'_1 + 6\mu'_2(\mu'_1)^2 - 3(\mu'_1)^4\]

ToFUL does not use these formulae internally — it computes each moment type directly from the definition, which avoids catastrophic cancellation for large or broad distributions.

Existence of Moments 

Not all distributions have finite moments of all orders. A moment of order \(r\) exists (is finite) if and only if \(E[|X|^r] < \infty\).

For distributions with heavy tails, high-order moments may be infinite:

Cauchy distribution — no finite moments of any order.
Pareto distribution — moments of order \(r \geq \alpha\) are infinite, where \(\alpha\) is the tail index.
Student-t with ν degrees of freedom — moments of order \(r \geq \nu\) are infinite.

When a moment does not exist, the integration or sum will either return a very large number or fail to converge. The Convergence tab will report the method as partial-sum with a note about divergence.

Moment Generating Function 

The moment generating function (MGF) of X is:

\[M_X(t) = E\left[e^{tX}\right]\]

When it exists in a neighbourhood of \(t = 0\), the raw moments are obtained by differentiation:

\[\mu'_r = M_X^{(r)}(0) = \left.\frac{d^r}{dt^r} M_X(t)\right|_{t=0}\]

ToFUL does not use MGFs directly, but the SymPy symbolic path essentially performs the equivalent symbolic integration.

Standardised Moments 

The standardised r-th moment divides the central moment by the appropriate power of the standard deviation:

\[\tilde{\mu}_r = \frac{\mu_r}{\sigma^r}\]

The two most commonly standardised moments are skewness (\(\tilde{\mu}_3\)) and kurtosis (\(\tilde{\mu}_4\)). See Skewness and Kurtosis for a detailed treatment.