Matrices and Matrix Multiplication — cheat sheet — Mathematics

A matrix product is a grid of dot products, legal only when inner dimensions agree, and fast because hardware runs those dot products in parallel.

Shapes

A matrix is a grid of numbers with shape (rows × columns).
A B is defined only when inner dimensions match: (m × n) times (n × p) gives (m × p).
Shared n is consumed; outer m and p survive. Mismatched inner dims = no product.
Tracking shapes at every step catches the most common ML bug.

(m × n) · (n × p)  →  (m × p)
      \____/
    must match

(AB)[i][j] = sum over k = 1..n of A[i][k] * B[k][j]

A dot product multiplies two equal-length vectors element by element and sums to one number.
Matrix-vector product A x returns a new vector; a linear layer computes y = W x + b, and training learns the entries of W.

Property	Rule	Note
Not commutative	`A B ≠ B A`	Order matters; `B A` may not exist
Associative	`(A B) C = A (B C)`	Regroup to pick cheapest order
Distributive	`A (B + C) = A B + A C`
Identity	`A I = I A = A`	`I` = ones on diagonal, zeros elsewhere
Transpose	`(A B)^T = B^T A^T`	Product reverses

Naive cost: O(m·n·p) multiply-adds (m·p output cells, each a length-n dot product).
For square n × n matrices: O(n³).
Strassen and successors lower the exponent but are rarely used in practice.
Real speedups come from parallel hardware (GPUs, TPUs), which is why ML work is cast as large matrix products.

Broadcasting virtually stretches a smaller array to match a larger one along size-1 dimensions.
Adding a bias (p,) to outputs (m × p) hits every row with no explicit loop.
It is elementwise, not matrix multiplication. Confusing the two gives wrong shapes or plausible-looking wrong answers.