Dictionary of Applied Machine Learning
Typeset PDF version — the authoritative form of this entry
Python demo — a script that recomputes what this entry states and prints one line per check
The mean of a random variable (RV) is its expectation, defined as the Lebesgue integral of the RV with respect to its probability distribution. The term also refers to the sample mean, i.e., the average of the data points in a dataset. The two usages are consistent: the sample mean is the mean of the RV obtained by drawing a data point uniformly at random from the dataset. For an RV with a finite second-order moment, the mean is the unique solution of a risk minimization problem under the squared error loss. For the RV associated with a dataset, this optimization problem reduces to empirical risk minimization (ERM) in the simplest regression setting: predicting a numeric label without any features, for which the learned hypothesis is the sample mean of the labels.
P-defThe mean of a random vector $\featurevec$, which takes on values in a Euclidean space $\reals^{\dimlocalmodel}$, is its expectation $\expect\{\featurevec\}$. It is defined as the Lebesgue integral of $\featurevec$ with respect to the underlying probability distribution $\prob{\cdot}$ (e.g., see (Rudin, 1976) or (Billingsley, 1986)), i.e., \[ \expect\{\featurevec\} = \int_{\reals^{\dimlocalmodel}} \vx \, \mathrm{d}P(\vx) \text{.} \] The term is also used to refer to the sample mean of a finite dataset $\dataset = \left\{ \vx^{(1)}, \,\ldots, \,\vx^{(\samplesize)} \in \reals^{\dimlocalmodel}\right\}$. These two usages are consistent. Indeed, it is possible to use a dataset to construct a discrete random variable (discrete RV) $\widetilde{\vx}^{(\dataset)}=\vx^{(I)}$ on the sample space $\{1, \,\ldots, \,\samplesize\}$. Here, the index $I$ is chosen uniformly at random, i.e., $\prob{I=\sampleidx}=1/\samplesize$ for all $\sampleidx=1,\ldots,\samplesize$. The mean of $\widetilde{\vx}^{(\dataset)}$ is precisely the average $({1}/{\samplesize}) \sum_{\sampleidx=1}^{\samplesize} \vx^{(\sampleidx)}$.
P-argminFor a random variable (RV) with a finite second-order moment, i.e., $\expect\{ \normgeneric{\featurevec}{2}^{2} \}$ is well defined and finite, the mean is characterized as the unique solution of the following risk minimization problem, whose objective function is a strictly convex function of $\vc$ (Bertsekas and Tsitsiklis, 2008): \[ \expect\{\featurevec\} = \argmin_{\vc \in \reals^{\dimlocalmodel}} \expect \big\{\normgeneric{\featurevec - \vc}{2}^{2}\big \} \text{.} \]
P-ermThis characterization turns the mean into the unique solution
of the simplest machine learning (ML) problem: predicting a numeric label
without any features. Consider a training set
$\trainset = \big\{ \truelabel^{(1)}, \,\ldots, \,\truelabel^{(\samplesize)} \big\}$
that consists of numeric labels
$\truelabel^{(\sampleidx)} \in \reals$ only. A hypothesis
is then a single number $\hypothesis \in \reals$ that serves as
the prediction for every data point. Empirical risk minimization (ERM) with
the squared error loss learns
\[
\widehat{\hypothesis}
= \argmin_{\hypothesis' \in \reals}
\frac{1}{\samplesize} \sum_{\sampleidx=1}^{\samplesize}
\big(\truelabel^{(\sampleidx)} - \hypothesis'\big)^{2}
= \frac{1}{\samplesize} \sum_{\sampleidx=1}^{\samplesize}
\truelabel^{(\sampleidx)} \text{,}
\]
i.e., the learned hypothesis is the sample mean of
the labels (see Fig. 1). This setting
coincides with linear regression for data points that carry
the constant scalar feature $\feature = 1$.
See also: random variable, expectation, sample mean, probability distribution, Lebesgue integral, empirical risk minimization, squared error loss, linear regression.
@misc{dictml_mean,
author = {Jung, Alexander},
title = {mean},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-21},
url = {https://dictionaryofml.org/terms/mean.html}
}