Goat Product

I did a lot of maths at school, and while I’ve got a good memory and do use bits of it here and there, it’s been fifteen years and some of it’s a bit rusty. Partly in an attempt to remedy this, I recently I watched some videos on Youtube about calculus and linear algebra. They’re by someone called Grant Sanderson, whose account is called 3Blue1Brown, and they have a friendly style with lots of neat animations. The idea of the videos is that a lot of people are taught maths in a way that involves a lot of number crunching and rule following but doesn’t get them to understand the underlying concepts, and that this leaves a gap that can be filled by presenting the ideas using visual interpretations. For example, a 2x2 matrix represents a transformation of 2D space that turns every square of size 1 into a parallelogram whose size is the magnitude of the matrix’s determinant. (If it’s negative, that means things are mirrored.) It’s possible I learned this at school, but stressing it and showing a nice animation helps you really understand what the deal is with determinants. They’re cool videos, and if you like maths then I recommend them, although they’re not really designed to teach the topics from scratch. I also recommend this one by the same person, which shows how to give a topological proof of something that doesn’t really seem to be about topology. I watched it a while ago and it really is beautiful.

Anyway, as I was going through the linear algebra videos I wrote down some questions I still had so I could find answers to them later, and one of them was about the dot product (also known as the scalar product). The dot product of two vectors <a,b> and <x,y> is a dimensionless directionless scalar, equal to ax + by. Why? What’s it measuring? The video on the dot product gave it a couple of interpretations.

One was to project <a,b> onto the line <x,y> is on via the shortest route, and then multiply the length of the new vector by the length of <x,y>. That might be OK for giving a visual heuristic for calculating or estimating the dot product, but for me it didn’t get me much of a handle on what sorts of thing it represents. Multiplying the length of two vectors is something you’d do if they were the sides of a rectangle and you wanted to find its area, but if the vectors are on the same line, where’s the rectangle? And if it’s an area, why is the dot product a scalar? It makes sense for the determinant to be a scalar because it’s the scale factor for areas, but with the projection interpretation of the dot product I couldn’t really see which areas were getting scaled up.

The second interpretation replaced one of the vectors with a 1x2 matrix, and then visualized this as a transformation taking points in 2D space onto a number line embedded in that space. The 1x2 matrix <a,b> will take the point <x,y> in 2D space to the point ax+by on the number line. That’s great, and it shows you the nice duality between the dot product and this kind of transformation, and indeed that was the point of this part of the video. (The video was called “Dot Products And Duality”, after all.) But a 2x1 vector isn’t a 1x2 matrix, and the fact that that taking the dot product is dual to performing a process that outputs a scalar doesn’t really tell you why the dot product itself is a scalar.

There will be good reasons for the dot product to be a scalar, of course, but this video didn’t help me see what they are. I also thought it’d be good to have a better visual interpretation of the dot product, where I could point to something I could see and say “that’s the dot product”, instead of pointing at two things I could see and having to multiply them together to find the dot product, even though the multiplication didn’t correspond to anything in the picture. In the video series on calculus he actually flags up that he doesn’t have a nice picture for the derivative of 2t where we can point to it and say: “See? That part! That is the derivative of 2t!” He says he’d like one, but doesn’t know one. Perhaps there isn’t one. Although perhaps there must be one. I guess different people will have different views about how high our expectations about this sort of thing should be.

Anyway, I made a note of the question, and when I got to the end of the videos I still didn’t have an answer, so I googled things like “visual interpretation of dot product” and “why is the dot product a scalar”. And you know what? Nobody else seemed to have a decent answer either! I mean, I didn’t look very hard, but there seems to be a fair bit of mystification out there, both from people asking the question and people trying to answer it. On this Quora page about the question, one person said that’s just how it’s defined, which isn’t helpful, and another said it was the distance between points the two vectors point at, which isn’t true on any interpretation I could work out. On another page I found more helpful someone said it was like a booster ramp in Mario Kart, which boosts you best if you drive over it in the direction of the ramp, and doesn’t boost you so well if you drive over it obliquely. I quite liked that one, but it has some problems. One is that if you drive over it backwards it still boosts you forwards, so there’s nothing much that seems to correspond to negative dot products. The other problem is that it’s not obvious what corresponds to multiplication here. It seems to me that the ramp adds a certain amount of velocity, rather than multiplying a vector from the ramp by the vector given by your initial velocity. Also the result of going on a ramp seems more like a vector than a scalar. I think these three issues are related. I did still find it helpful, and maybe I just don’t have a good enough feel for how the booster ramps work (although I have played the game quite a lot), but I wanted something better. He had one involving the angles of solar panels too but he didn’t like that as much as the Mario Kart one, and I didn’t really either. (I couldn’t see how it produced negative dot products, for example.)

So anyway, I had a go at coming up with my own visualization, and I found one that I think works pretty well. You know how people share those pictures of goats standing on almost vertical surfaces, seemingly defying gravity? Well, imagine a goat standing on one of those. It could be a very steep one like the cliffs in the pictures, or it could be shallower like a hillside. Basically any inclined plane will do. We can represent the slope with a vector u, for upslope. The vector points up the slope and its magnitude is the gradient of the slope in that direction. <0,0> corresponds to a horizontal plane, which is fine, but no vector corresponds to a vertical one, which is just as well because even a goat can’t stand on a vertical plane. Now the goat sets off along the surface in a straight line, with its horizontal velocity given by a vector v. It may be easiest to picture v if you look down on the goat from above. Now the dot product is the rate of change in the goat’s elevation. Positive means the goat is going up, and negative means the goat is going down. It’s a scalar, not a vector, because it’s measuring how fast the goat is moving up or down, not measuring how fast it’s moving and which direction it’s moving. (Well, it’s measuring whether the goat is going up or down, but even scalars distinguish a positive and a negative direction. It’s not measuring the direction in three dimensions, or even two.)

The sums seem to work out. Think of u = <a,b> as giving the gradient a when moving north and the gradient b when going east, and think of v = <x,y> as giving the velocity’s northern component x and eastern component y. Negative gradients are downhill, and negative velocity north or east is movement south or west. So if it goes x north and then y east it will go a*x up and then b*y up.

It gives straightforward visual interpretations to some properties of the dot product too. To get maximum height for its horizontal displacement, the goat goes straight uphill, which means u and v have the same direction. Change in elevation for a given horizontal dispacement is proportional to gradient, and change in elevation for a given gradient is proportional to horizontal displacement. It’s clear when the dot product will be positive, negative or zero, depending on whether it’s moving uphill, downhill or along a contour. It’s a good visualization of how a difference in the angle between u and v makes less of a difference to the dot product when the angle is small than when it’s big. It’s also clear that if u = <0,0> the plane is flat and the goat won’t go up or down however much it moves horizonatally, and it’s clear that if v = <0,0> the goat is stationary so it won’t go up or down however steep the slope is.

Finally, we can think about how different vectors u can multiply by the same vector v to give the same dot product. To do that, imagine that the plane is one side of a V-shaped valley whose edges and base are horizontal, and the goat wants to get to the top or the bottom, depending on whether the dot product is positive or negative. The dot product represents the height to the top or bottom of the valley, and u represents the slope of the valley. Now imagine you’re looking at the goat from above, and it wants to get to the top. There are lots of ways it could get to its destination. It can run straight uphill, and then it won’t have gone so far horizontally. Or it can set off almost at right angles to the slope, and then it’ll go a very long way horizontally before it gets to the top. Or it can do something in between. But it can’t go at right angles or an obtuse angle to the slope, or it’ll never get to the top. And there’s also a minimum horizontal distance it has to go. Viewing from above, you can draw a little circle around the goat and see that if the circle is small enough everything inside it will still be in the valley.

So there’s my visualization for the dot product. I like it! Commutativity holds, as it must, but it doesn’t immediately drop out of the setup the way it might if u and v represented more similar sorts of things. I won’t try making a virtue of that. But I think it’s OK, and I like it better than the other ones I found, and assuming I’ve not made mistakes with it, I think other people would like it too. I’m sure there are other adequate ones out there, but not everyone is aware of them, and coming up with these things yourself is all part of the learning process anyway. I didn’t really get what the deal was with the dot product, and now I think I do. The dot product is the goat product.