Dot Product and Projection

The dot product is one of those things that I learned about early in life, but always felt a bit uncomfortable with the mechanics. I’ve even used it many times throughout my posts, but generally for the same property (a measure of angle similarity). I aim to cover more details than I’ve leveraged in the past.

Primer

Firstly, the dot product is defined as the sum of the component-wise multiplication of two vectors. For example, for 2D vectors $\vec{a}$ and $\vec{b}$:

$$\vec{a} \cdot \vec{b} = a_x b_x + a_y b_y$$

Generally, for n-dimensional vectors we have:

$$\vec{a} \cdot \vec{b} = \sum_{i=1}^n a_i b_i$$

I’m mostly interested in 2D and 3D (practical) applications, so I’ll stick to 2D vectors from here onwards.

Measure of similarity

An interesting property of the dot product is that it produces a scalar output. This output is related to the magnitudes of the vectors. It also gets bigger when the vectors are parallel and tends to zero when they are perpendicular. It is negative when they are in opposing directions. The following is a little interactive demonstration of this relationship¹:

In fact, the dot product is also equal to the following (where $\theta$ is the angle between $\vec{a}$ and $\vec{b}$):

$$\vec{a} \cdot \vec{b} = \|\vec{a}\| \|\vec{b}\| \cos{\theta}$$

The special thing to note is if you have two normalized vectors, you can just calculate $\cos{\theta}$ with some simple multiplication and addition. However, there’s more potential hidden in this equation which is not immediately clear from what’s been demonstrated so far.

Projections

If we draw a triangle from a line drawn perpendicularly from the end of one of the vectors to the other, you may notice that there’s the opportunity for some trigonometry.

Remaining compatible with the dot product, recall the definition of cosine:

$$\cos{\theta} = \frac{\text{adjacent}}{\text{hypotenuse}}$$

Rearranging, it’s:

$$\text{adjacent} = \text{hypotenuse} \times \cos{\theta}$$

The hypotenuse, in this case, is the length of the vector, $\vec{a}$. If we call the projected point on the vector, $\vec{b}$, $P_{proj}$ and the vector that points from the origin to $P_{proj}$, $\vec{p}_{proj}$, then the adjacent length is $\|\vec{p}_{proj}\|$. So, substituting into the above, we have:

$$\|\vec{p}_{proj}\| = \|\vec{a}\| \cos{\theta}$$

That’s very similar to the dot product! In fact, if you divide the dot product by $\|\vec{b}\|$, you’ll get the length of the projected point on $\vec{b}$:

$$\begin{align*} \vec{a} \cdot \vec{b} &= \|\vec{a}\| \|\vec{b}\| \cos{\theta} \\ \frac{\vec{a} \cdot \vec{b}}{\|\vec{b}\|} &= \|\vec{a}\| \cos{\theta} \\ &= \|\vec{p}_{proj}\| \end{align*}$$

That, in itself, is already useful. You can determine the length along $\vec{b}$ in which $\vec{a}$ projects (or “casts a shadow”). However, if you want a vector, you need to multiply this scalar by the normal vector, $\vec{n}_b$, of $\vec{b}$. If you expand this out, you’ll notice something interesting:

$$\begin{align*} \vec{p}_{proj} &= \|\vec{p}_{proj}\| \vec{n}_b \\ &= \|\vec{p}_{proj}\| \frac{\vec{b}}{\|\vec{b}\|} \\ &= \frac{\vec{a} \cdot \vec{b}}{\|\vec{b}\|} \frac{\vec{b}}{\|\vec{b}\|} \\ &= \frac{\vec{a} \cdot \vec{b}}{\|\vec{b}\|^2} \vec{b} \\ \end{align*}$$

This is interesting because another dot product arises. Consider the fact that a vector points in the same direction of itself, so the cosine of the dot product returns 1, and we’re just left with the magnitude squared. So, we have:

$$\vec{p}_{proj} = \frac{\vec{a} \cdot \vec{b}}{\vec{b} \cdot \vec{b}} \vec{b}$$

That means no square roots are even necessary to calculate this vector. No lengths were needed. If you look even more carefully you’ll note that we’ve pulled out a scalar and a vector component:

$$\vec{p}_{proj} = \underbrace{\frac{\vec{a} \cdot \vec{b}}{\vec{b} \cdot \vec{b}}}_{\mathclap{scalar}} \times \underbrace{\vec{b}}_{\text{vector}}$$

We can call this scalar $t$, and note that it represents the “ratio” of the new vector in relation to $\vec{b}$. If you clamp this value between $[0, 1]$ you will be able to determine the closest point on the vector, $\vec{b}$, to the original vector, $\vec{a}$. This is very useful if you’re using $\vec{a}$ to represent a point and $\vec{b}$ to represent a geometric line, for example. The following is an interactive diagram that encompasses all of these properties or tricks that can be performed with the dot product.

Footnotes

1. It may look like $\vec{b}$ is somehow in charge, here. It’s just framed this way, and that’s true for the entire post. Either vector could be the target of projection. ↩