# Matrix Multiplication

Composing two linear transformations by taking dot products of rows with columns.

**Rests on:** Linear Transformations, Tensors and Shapes

**Leads to:** Cross-Modal Fusion, Determinants and Inverses, Eigenvalues and Eigenvectors, Forward Propagation, Normal Equations, Orthogonality and Projections, Singular Value Decomposition

*Mathematical Foundations · Linear Algebra*


## Why learn this

Matrix multiplication is the single most executed operation in machine learning, and it is the reason GPUs exist. It underpins every modern AI system you will meet in this course: when a large language model predicts the next word, it is doing matrix multiplication; when your phone's voice assistant recognizes 'turn on the lights', a small model on the device is doing matrix multiplication; when a recommendation system suggests a movie, it is doing matrix multiplication. You cannot understand how any of these systems work — or debug why they fail — without this operation. Mastering it unlocks: Linear Transformations, Layers, Backpropagation, Embeddings, Attention, and the Transformer architecture. It is the arithmetic heart of everything that follows.


## The intuition

Imagine you are a shopkeeper with three products — pens, notebooks, and erasers. You have two customers who bought different quantities. Customer A bought 2 pens, 3 notebooks, and 1 eraser. Customer B bought 1 pen, 0 notebooks, and 4 erasers. You want to know how much each customer owes you, so you need to multiply the quantities by the prices (say 10 rupees per pen, 50 rupees per notebook, 5 rupees per eraser) and add them up.

This is a dot product: for each customer, you take each quantity, multiply it by the corresponding price, and add the results. Customer A owes 2×10 + 3×50 + 1×5 = 20 + 150 + 5 = 175 rupees. Customer B owes 1×10 + 0×50 + 4×5 = 10 + 0 + 20 = 30 rupees.

Now, what if you have multiple customers, and you want to do this all at once? You line up the customers as rows of a table (each row is a customer's quantities), and the prices as a column. Multiplying the table of customers by the price column gives you a new column — the amount each customer owes. That is matrix multiplication: a way to apply the same rule (prices) to many different inputs (customers) at once.

In AI, the 'customers' are data points (like images or sentences), and the 'prices' are weights the model has learned. Matrix multiplication is how the model applies its knowledge to new data.


## An analogy

Think of matrix multiplication as a recipe for mixing paint. You have a shelf of primary colors: red, blue, yellow. Each paint can (a row in the first matrix) contains a recipe: how many parts of each primary to mix. For example, can 1: 30% red, 20% blue, 50% yellow. Can 2: 10% red, 60% blue, 30% yellow. Now you have a set of 'target colors' (a second matrix), each column telling you how much of each can to use. For example, target A: use 2 parts of can 1, 1 part of can 2. Target B: use 1 part of can 1, 3 parts of can 2.

The result of multiplying the recipe matrix by the mixing matrix is a new matrix where each entry tells you how much of each primary color you need for each target. For target A, you mix 2 parts of can 1 (which has 30% red) and 1 part of can 2 (10% red), so the red needed is 2×30% + 1×10% = 70% — wait, that is not right, because the parts are not percentages. Let me fix: the recipe gives the proportion of primary in each can, and the mixing matrix gives the number of parts of each can. So for target A, red = 2×0.3 + 1×0.1 = 0.7 (in some unit). Blue = 2×0.2 + 1×0.6 = 1.0. Yellow = 2×0.5 + 1×0.3 = 1.3. So target A needs 0.7 red, 1.0 blue, 1.3 yellow.

This shows how matrix multiplication composes two sets of relationships: the recipes (how each can is made from primaries) and the mixing (how each target is made from cans). The result tells you the final composition.

Where the analogy breaks down: In paint mixing, the numbers represent physical quantities that must be non-negative and add up to a sensible total. In matrix multiplication, numbers can be negative, and there is no such constraint. Also, in paint mixing, the order of operations does not matter — mixing 2 parts of can 1 and 1 part of can 2 is the same as 1 part of can 2 and 2 parts of can 1. In matrix multiplication, order matters enormously: A×B is usually not equal to B×A. This is because matrix multiplication represents the composition of transformations — the order in which you apply transformations changes the result, just as putting on socks then shoes is different from shoes then socks.


## Definition

Matrix multiplication is an operation that takes two tables of numbers (matrices) and produces a new table by taking the dot product of each row of the first matrix with each column of the second matrix, provided the number of columns in the first equals the number of rows in the second.


## How it connects

You have no prior mathematics beyond arithmetic, so this page builds from scratch. The key idea you must take from arithmetic is multiplication and addition, which you have used. Matrix multiplication is a way to organize many multiplications and additions at once. It is the foundation of linear algebra, which is the branch of mathematics that studies tables of numbers and the transformations they represent. Once you master matrix multiplication, you will be ready for: linear transformations (how a matrix moves points in space), eigenvalues and eigenvectors (directions that a matrix stretches), and later, neural network layers (which are just matrix multiplications followed by a nonlinear step). The shape rules you will learn here are exactly what let you check whether a neural network architecture is valid on paper.


## The mathematics

Matrix multiplication is an operation that combines two tables of numbers (called matrices) into a single new table. It is not simply multiplying corresponding numbers side-by-side. Instead, it works by combining rows from the first table with columns from the second table.

Before we multiply matrices, we need to understand the dot product. The dot product is a way to combine two lists of numbers of the same length into one single number. To compute it, you multiply the first number of the first list by the first number of the second list, then the second number by the second number, and so on. Finally, you add all those products together.

For example, if you have the lists (2, 3) and (4, 5), the dot product is calculated as:
(2 times 4) + (3 times 5) = 8 + 15 = 23.

Matrix multiplication uses this dot product repeatedly. You take two tables, A and B. To multiply them, you must check a condition called the shape rule: the number of columns in the first table (A) must equal the number of rows in the second table (B). If they do not match, the multiplication is impossible.

Let us define the sizes clearly:
- Let A be a table with m rows and n columns. We write this as an m-by-n matrix.
- Let B be a table with n rows and p columns. We write this as an n-by-p matrix.
Because A has n columns and B has n rows, they fit together.

The result, which we call C, will be a new table with m rows and p columns (an m-by-p matrix).

To find any single number in the result table C, you pick one row from A and one column from B. Specifically, to find the number in row i and column j of the result (where i counts down from 1 to m, and j counts across from 1 to p), you take the i-th row of A and the j-th column of B. You multiply their matching numbers together and add them up. This sum is exactly the dot product of that row and that column.

In mathematical notation, if we label the numbers in A as a_ik (the number in row i, column k) and the numbers in B as b_kj (the number in row k, column j), the number in row i and column j of the result C is:

C[i][j] = (a_i1 times b_1j) + (a_i2 times b_2j) + ... + (a_in times b_nj)

Here, the index k runs from 1 to n. This means you sum the products of corresponding elements as k goes from 1 up to n.

Important note: The order matters. In general, multiplying A by B gives a different result than multiplying B by A. This is because matrix multiplication represents applying one transformation after another. Doing step A then step B is usually not the same as doing step B then step A.


## A worked example

Let us work through a specific example with real numbers.

Let A be a 2-by-3 matrix (2 rows, 3 columns):
A = [[1, 2, 3],
     [4, 5, 6]]

Let B be a 3-by-2 matrix (3 rows, 2 columns):
B = [[7, 8],
     [9, 10],
     [11, 12]]

Check the shape rule: A has 3 columns. B has 3 rows. They match. The result C will have 2 rows (from A) and 2 columns (from B). So C is a 2-by-2 matrix.

We need to calculate four numbers for C: the top-left, top-right, bottom-left, and bottom-right entries.

Step 1: Calculate the entry in row 1, column 1 of C (C[1][1]).
Take the first row of A: [1, 2, 3].
Take the first column of B: [7, 9, 11] (reading down).
Multiply matching pairs and add:
(1 times 7) + (2 times 9) + (3 times 11)
= 7 + 18 + 33
= 58.
So the top-left number is 58.

Step 2: Calculate the entry in row 1, column 2 of C (C[1][2]).
Take the first row of A: [1, 2, 3].
Take the second column of B: [8, 10, 12] (reading down).
Multiply matching pairs and add:
(1 times 8) + (2 times 10) + (3 times 12)
= 8 + 20 + 36
= 64.
So the top-right number is 64.

Step 3: Calculate the entry in row 2, column 1 of C (C[2][1]).
Take the second row of A: [4, 5, 6].
Take the first column of B: [7, 9, 11] (reading down).
Multiply matching pairs and add:
(4 times 7) + (5 times 9) + (6 times 11)
= 28 + 45 + 66
= 139.
So the bottom-left number is 139.

Step 4: Calculate the entry in row 2, column 2 of C (C[2][2]).
Take the second row of A: [4, 5, 6].
Take the second column of B: [8, 10, 12] (reading down).
Multiply matching pairs and add:
(4 times 8) + (5 times 10) + (6 times 12)
= 32 + 50 + 72
= 154.
So the bottom-right number is 154.

The final result matrix C is:
C = [[58, 64],
     [139, 154]]


## From scratch, in Python

```python
# We implement matrix multiplication using only Python lists and a loop.
# This shows the arithmetic underneath before we use any libraries.

# Define matrix A as a list of lists (each inner list is a row).
A = [[1, 2, 3],
     [4, 5, 6]]

# Define matrix B similarly.
B = [[7, 8],
     [9, 10],
     [11, 12]]

# Get the number of rows and columns of A, and columns of B.
rows_A = len(A)          # number of rows in A
cols_A = len(A[0])       # number of columns in A (length of first row)
rows_B = len(B)          # number of rows in B
cols_B = len(B[0])       # number of columns in B

# Check that the inner dimensions match.
if cols_A != rows_B:
    raise ValueError("Inner dimensions must match!")

# Create an empty result matrix of size rows_A x cols_B, filled with 0s.
C = [[0 for _ in range(cols_B)] for _ in range(rows_A)]

# Loop over each row of A.
for i in range(rows_A):
    # Loop over each column of B.
    for j in range(cols_B):
        # Compute the dot product of A[i] and the j-th column of B.
        total = 0
        for k in range(cols_A):  # or range(rows_B), they are equal
            total += A[i][k] * B[k][j]
        C[i][j] = total

print("Result C:")
for row in C:
    print(row)
```


## The same thing, with a framework

```python
import numpy as np

A = np.array([[1, 2, 3],
              [4, 5, 6]])
B = np.array([[7, 8],
              [9, 10],
              [11, 12]])

C = np.dot(A, B)  # or A @ B
print(C)
```


## What the code does

The plain Python code does the following: it defines two matrices as lists of lists. A list is an ordered collection of items; a list of lists is a table where each inner list is a row. The code then checks that the number of columns in A equals the number of rows in B, otherwise multiplication is impossible. It creates an empty result matrix filled with zeros, using a nested list comprehension (a compact way to build a list). Then it loops over each row i of A and each column j of B, and inside a third loop over k, it accumulates the sum of A[i][k]*B[k][j] into a variable `total`. That total becomes the entry C[i][j]. This is exactly the dot product you computed by hand. The NumPy version does the same thing in one line: `np.dot(A, B)` (or the shorthand `A @ B`). Underneath, NumPy runs the exact same multiply-and-add loops, but in optimized C code, which makes it thousands of times faster for large matrices.


## Common misconceptions

**Matrix multiplication is just element-wise multiplication** — To multiply two matrices, I just multiply the matching numbers in each position, like adding two tables.

That is called the Hadamard product (element-wise), which is a different operation. Matrix multiplication involves dot products of rows and columns, not just matching entries. In fact: Each entry of the result is the dot product of a row from the first matrix and a column from the second. This combines information across the entire row and column, not just one pair.

**Order doesn't matter** — A × B is the same as B × A, just like with ordinary numbers.

Matrix multiplication represents applying transformations in sequence. Rotating then scaling is not the same as scaling then rotating. Also, the shapes may not even allow the reverse order. In fact: The order matters. A × B is generally not equal to B × A. Always check the shape rules and the semantics of the transformations.

**The shape rule is a mere formality** — I can multiply any two matrices; the shape thing is just a technicality I can ignore.

If the inner dimensions do not match, the operation is undefined. In practice, a shape mismatch causes a runtime error in NumPy or PyTorch, or worse, silent wrong results if broadcasting kicks in. In fact: The shape rule is essential. The number of columns of A must equal the number of rows of B. The result has the rows of A and the columns of B.

**The result is just another table of numbers** — The entries of the product matrix have no meaning; they are just arbitrary numbers.

Each entry encodes how much a particular output component is influenced by a particular input component, given the transformation represented by the matrices. In fact: Matrix multiplication composes transformations. The result represents the combined effect: first apply B, then apply A (or vice versa depending on order). The entries are meaningful coordinates in the transformed space.

**Matrix multiplication is hard to compute by hand** — I will never need to do this by hand, so I can skip learning the steps.

If you cannot do it by hand, you cannot reason about shape errors or understand what a library call is doing. The hand computation is the same as the code. In fact: You can always compute by hand using the dot product rule. Practice with small 2×2 and 3×3 matrices until it becomes second nature.


## In short

Matrix multiplication is the core operation of AI. It takes two matrices and produces a new one by taking the dot product of each row of the first with each column of the second. The inner dimensions must match; the result has the outer dimensions. It is not commutative: order matters. In code, use NumPy's `@` operator, but know that the library is doing the multiply-and-add loops you wrote by hand.


## Where this is used

Matrix multiplication is the workhorse of all AI systems. Here is how it appears in each area:

- **Large language models (LLMs):** Every token in a sentence is converted into a vector (a list of numbers) and placed in a matrix. Each layer of the transformer does a matrix multiplication between that matrix and a learned weight matrix. For example, the query, key, and value projections are all matrix multiplications. When you train a 70-billion-parameter model, you are adjusting the numbers in all those weight matrices. Inference — generating text — repeats these multiplications billions of times per response. A developer using PyTorch writes `output = input @ weight.T + bias`, but underneath the GPU executes the batched matrix multiply that this page taught by hand.

- **Small models on devices (TinyML):** When your phone runs a voice assistant that detects 'Hey Google' or 'turn on the lights', it uses a small neural network, often with a few million parameters instead of billions. Matrix multiplication still drives everything, but the operation is optimized for low power and limited memory. Developers use frameworks like TensorFlow Lite, which converts the model's matrix multiplications into efficient kernels for the phone's CPU or a tiny neural processing unit. The shapes are small, but the principle is identical.

- **Retrieval-augmented generation (RAG):** RAG systems search a database of documents to find relevant context for a language model. The search is done by embedding — converting each document and the query into a vector using an embedding model. The similarity between the query and each document is computed with a dot product (which is the building block of matrix multiplication). Often, the embeddings of all documents are stored as a matrix, and the query vector is multiplied by that matrix to get scores for every document in one step. This is a matrix-vector multiplication, a special case of matrix multiplication.

- **Agentic systems that plan and call tools:** Agents use language models to decide which tool to call. The decision may involve a classifier that is a neural network, which again is a sequence of matrix multiplications. Planning — breaking a task into steps — is driven by the same model, so the underlying computation is the same. The agent may also use RAG to retrieve knowledge, again via matrix multiplication. There is no new operation here; it is the same workhorse.

- **Everyday applied ML (recommendation, vision, forecasting, fraud, speech):**
  - Recommendation: Collaborative filtering represents users and items as vectors; the predicted rating for a user-item pair is a dot product of those vectors, which is one entry of a matrix multiplication if you compute all pairs at once.
  - Computer vision: An image is a matrix of pixel values (or a stack of matrices for colors). A convolutional neural network applies filters, which are small matrices, via convolution — but the dense layers that follow are pure matrix multiplication.
  - Forecasting: Time-series models like LSTMs or transformers use matrix multiplication for every timestep's update.
  - Fraud detection: Features of a transaction are a vector; a neural network scores it by matrix multiplications.
  - Speech recognition: Audio is converted to spectrograms (matrices), and the acoustic model is a neural network with many matrix multiplications.

**What a developer actually writes:** In NumPy, `np.dot(A, B)` or `A @ B`. In PyTorch, `torch.matmul(A, B)` or `A @ B`. In TensorFlow, `tf.matmul(A, B)`. Under the hood, these call highly optimized libraries (BLAS) that implement the same loops, but they are tuned at the assembly level and run on GPUs that have thousands of cores, each crunching a small block of the result.

**What breaks when done badly:** The most common bug is a shape mismatch: multiplying a matrix of shape (2,3) by another of shape (2,3) throws an error. In deep learning, tensors have more dimensions (batch, sequence, channel), and debugging shape errors is the daily work of ML engineers. If you accidentally transpose a matrix, your result is silently wrong. Performance-wise, doing matrix multiplication with Python loops is orders of magnitude slower than with BLAS, which is why you always use the library version.


---

Read this on the web: https://basavapurushottam.com/learn-ai/matrix-multiplication
