projection

Typeset PDF version — the authoritative form of this entry

Python demo — a script that recomputes what this entry states and prints one line per check

The projection of a vector onto a closed non-empty subset of a Euclidean space is a point in the subset that is closest to the vector in the Euclidean norm. If the subset is convex, this closest point is unique. If the subset is a subspace, the projection is a linear map, namely the orthogonal projection onto the subspace. In machine learning (ML), projections enforce constraints on model parameters during training: projected gradient descent (projected GD) alternates gradient steps with projections onto the constraint set, e.g., the $\ell_{1}$-ball in an equivalent formulation of the least absolute shrinkage and selection operator (Lasso).

Definition

P-defConsider a closed non-empty subset $\paramspace \subseteq \reals^{\dimlocalmodel}$ of the $\dimlocalmodel$-dimensional Euclidean space. The projection $\projection{\paramspace}{\weights}$ of a vector $\weights \in \reals^{\dimlocalmodel}$ onto $\paramspace$ is defined as \begin{equation} \nonumber \label{equ_def_proj_generic_dict} \projection{\paramspace}{\weights} = \argmin_{\weights' \in \paramspace} \normgeneric{\weights - \weights'}{2}\text{.} \end{equation} In other words, $\projection{\paramspace}{\weights}$ is a vector in $\paramspace$ that is closest to $\weights$. The above minimum exists for every closed non-empty subset $\paramspace$; if $\paramspace$ is also convex, the closest point is unique (Boyd and Vandenberghe, 2004). If $\paramspace$ is a subspace, the map $\weights \mapsto \projection{\paramspace}{\weights}$ is linear: it is the orthogonal projection onto $\paramspace$.

P-projgdIn machine learning (ML), projections are used to enforce constraints on model parameters during training. As a case in point, consider linear regression with the constraint $\normgeneric{\weights}{1} \leq \tau$, which is an equivalent formulation of the least absolute shrinkage and selection operator (Lasso) (Tibshirani, 1996). This constrained problem can be solved by projected gradient descent (projected GD): each iteration takes a gradient step and then projects the result onto the $\ell_{1}$-ball $\paramspace = \big\{ \weights' \in \reals^{\dimlocalmodel} : \normgeneric{\weights'}{1} \leq \tau \big\}$. Since the $\ell_{1}$-ball is not a subspace, this projection is not a linear map (see Fig. 1).

Figure 1 of the entry projection — Figure 1: Projection of a vector $\weights \in \reals^{2}$ onto the $\ell_{1}$-ball $\paramspace = \big\{ \weights' \in \reals^{2} : \normgeneric{\weights'}{1} \leq \tau \big\}$. The projection $\projection{\paramspace}{\weights}$ is the point of $\paramspace$ closest to $\weights$ in the Euclidean norm

See also: Euclidean space, vector, minimum, convex, orthogonal projection, projected GD.

References

Boyd and Vandenberghe (2004). Convex Optimization. Cambridge Univ. Press. doi.org/10.1017/CBO9780511804441
Tibshirani (1996). Regression Shrinkage and Selection via the Lasso. J. Roy. Statist. Soc.: Ser. B (Methodological). doi.org/10.1111/j.2517-6161.1996.tb02080.x

Cite this entry

@misc{dictml_projection,
  author = {Jung, Alexander},
  title = {projection},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-21},
  url = {https://dictionaryofml.org/terms/projection.html}
}