Python demo — a script that recomputes what this entry states and prints one line per check
A norm on a vector space is a function that
assigns each vector a nonnegative number, measuring its
length. A norm must be definite and homogeneous, and it must
satisfy the triangle inequality. Every norm defines a
metric via the norm of the difference of two
vectors, and every inner product induces a norm. A
prominent family of norms in machine learning (ML) is the $\ell_{p}$-norms,
including the $\ell_{1}$-norm, the Euclidean norm, and the
$\ell_{\infty}$-norm. Norms are used, for instance, to define a
loss function or a regularizer.
Definition
P-axiomsA norm is a function that maps each (vector) element
of a vector space to a nonnegative real number. Formally, a norm
$\normgeneric{\, \cdot \,}{}$ on a vector space $\vecspace$ over a field $\mathbb{F}$ is a
function $\normgeneric{\, \cdot \,}{}: \vecspace \to \mathbb{R}_{+}$
that satisfies the following conditions (Horn and Johnson, 2013) for all $\vu, \vv \in \vecspace$
and $\expcoeff \in \mathbb{F}$:
Intuitively, norms measure the length or size of a vector.
Norms can also be used to measure distances between vectors, as they define a metric by
$\metric{\vu}{\vv} \defeq \normgeneric{\vu - \vv}{}$,
making $\pair{\vecspace}{\metric{\cdot}{\cdot}}$ a metric space.
On an inner product space, the inner product induces a
norm via $\normgeneric{\vu}{} \defeq \sqrt{\innerprod{\vu}{\vu}}$;
this is the norm of a Hilbert space
(see inner product, Hilbert space).
A prominent family of norms used in machine learning (ML) is the $\ell_p$-norms for $p \geq 1$,
defined for a vector $\featurevec \in \mathbb{R}^\featuredim$ as
$\normgeneric{\featurevec}{p} = \left( \displaystyle \sum_{\featureidx=1}^{\featuredim} |\feature_{\featureidx}|^p \right)^{1/p}$.
Important instances include the $\ell_1$-norm, the $\ell_2$-norm (or Euclidean norm),
and the $\ell_\infty$-norm $\normgeneric{\featurevec}{\infty} = \max_{\featureidx = 1, \ldots, \featuredim} |\feature_{\featureidx}|$, which are illustrated
by the geometry of their respective unit spheres in Fig.\ 1.
In ML, norms are fundamental, for instance for defining a loss function or a regularizer.
Figure 1: Unit spheres $\sphere{1}$ with respect to different $\ell_p$-norms for $p=1,\,2,\,\infty$
See also: vector space, metric, metric space,
Euclidean space, Hilbert space, inner product,
linear regression, least absolute shrinkage and selection operator (Lasso).
