Tivadar Danka

The Roadmap of Mathematics for Machine Learning A complete guide to linear algebra, calculus, and probability theory

Most machine learning practitioners don’t understand the math behind their models.

That's why I've created a FREE roadmap so you can master the 3 main topics you'll ever need: algebra, calculus, and probabilities.

Get the roadmap here: thepalindrome.org/p/the-roadm...

What’s the lesson here?

That visual and algebraic thinking go hand in hand. The Japanese method neatly illustrates how multiplication works, but with the algebra behind it, we feel the pulse of long multiplication.

We are not just mere users; we see behind the curtain now.

Why is this relevant?

Because this is exactly what happens with the Japanese multiplication method!

Check this out one more time.

There’s more. How do we multiply 21 · 32 by hand?

First, we calculate 21 · 30 = 630, then 21 · 2 = 42, which we sum up to get 21 · 32 = 672.

We learn this at elementary school like a cookbook recipe: we don’t learn the why, just the how.

Here it is, visualized on our line representation.

Let’s decompose the operands into tens and ones before multiplying them together.

By carrying out the product term by term, we are doing the same thing!

Now comes the magic.

Count the intersections among the lines. Turns out that they correspond to the digits of the product 21 · 32.

What is this sorcery?

Similarly, the second operand (32) is encoded with two groups of lines, one for each digit.

These lines are perpendicular to the previous ones.

First, the method.

The first operand (21 in our case) is represented by two groups of lines: two lines in the first (1st digit), and one in the second (2nd digit).

One group for each digit.

The following multiplication method makes everybody wish they had been taught math like this in school.

It's not just a cute visual tool: it illuminates how and why long multiplication works.

Here is the full story:

The Palindrome | Tivadar Danka | Substack mathematics ∪ machine learning. Click to read The Palindrome, a Substack publication with tens of thousands of subscribers.

Join 33,000+ ML practitioners who get 2 actionable emails every week to help them understand the math behind ML, make smarter decisions, and avoid costly mistakes.

Subscribe here (it’s free): thepalindrome.org/

Peter Lax sums it up perfectly: "So what is gained by abstraction? First of all, the freedom to use a single symbol for an array; this way we can think of vectors as basic building blocks, unencumbered by components."

Linear algebra is powerful exactly because it abstracts away the complexity of manipulating data structures like vectors and matrices.

Instead of explicitly dealing with arrays and convoluted sums, we can use simple expressions AB.

That's a huge deal.

The same logic can be applied, thus giving an explicit formula to calculate the elements of a matrix product.

We can collapse the linear combination into a single vector, resulting in a formula for the first column of AB.

This is straight from the mysterious matrix product formula.

Recall that matrix-vector products are linear combinations of column vectors.

With this in mind, we see that the first column of AB is the linear combination of A's columns. (With coefficients from the first column of B.)

Now, about the matrix product formula.

From a geometric perspective, the product AB is the same as first applying B, then A to our underlying space.

(If unwrapping the matrix-vector product seems too complex, I got you.

The computation below is the same as in the above tweet, only in vectorized form.)

Moreover, we can look at a matrix-vector product as a linear combination of the column vectors.

Make a mental note of this, because it is important.

Matrices represent linear transformations. You know, those that stretch, skew, rotate, flip, or otherwise linearly distort the space.

The images of basis vectors form the columns of the matrix.

We can visualize this in two dimensions.

By the same logic, we conclude that A times eₖ equals the k-th column of A.

This sounds a bit algebra-y, so let's see this idea in geometric terms.

Yes, you heard right: geometric terms.

Similarly, multiplying A with a (column) vector whose second component is 1 and the rest is 0 yields the second column of A.

That's a pattern!

Now, let's look at a special case: multiplying the matrix A with a (column) vector whose first component is 1, and the rest is 0.

Let's name this special vector e₁.

Turns out that the product of A and e₁ is the first column of A.

Here is a quick visualization before the technical details.

The element in the i-th row and j-th column of AB is the dot product of A's i-th row and B's j-th column.

First, the raw definition.

This is how the product of A and B is given. Not the easiest (or most pleasant) to look at.

We are going to unwrap this.

Matrix multiplication is not easy to understand.

Even looking at the definition used to make me sweat, let alone trying to comprehend the pattern. Yet, there is a stunningly simple explanation behind it.

Let's pull back the curtain!

The Palindrome | Tivadar Danka | Substack mathematics ∪ machine learning. Click to read The Palindrome, by Tivadar Danka, a Substack publication with tens of thousands of subscribers.

Join 33,000+ ML practitioners who get 2 actionable emails every week to help them understand the math behind ML, make smarter decisions, and avoid costly mistakes.

Subscribe here (it’s free): thepalindrome.org/

The removal of the edge ab produces two connected components.

In the previous graph, the removal of vertex a (and its corresponding edges) produces five connected components, while the removal of vertex b produces two.

An interesting question regarding connectivity is how critical a vertex or edge is regarding connectivity.

If removing a single vertex (or edge) from a graph splits a connected component into two or more, then that vertex is called a cut vertex (or cut edge).