---
name: mathematical-foundations
description: "Mathematical Foundations: 77 concepts, no maths assumed."
version: 1.0.0
author: Learn AI
license: CC-BY-4.0
metadata:
  hermes:
    tags: [Mathematical, Foundations]
    source: https://basavapurushottam.com/learn-ai
---
# Mathematical Foundations

77 concepts, written for an adult with no computer science and no mathematics past school arithmetic. Each concept is a file under `references/`; load one when it is the thing being asked about rather than reading the whole domain.


## When to Use

- The reader asks about mathematical foundations and wants it explained from the ground up rather than summarised.

- A term below comes up and you want the plain-language account of it, with a worked example.

- You need to know what a concept rests on before teaching it — every reference lists its prerequisites.


## When NOT to Use

- The reader wants current events, papers or library documentation. This is a curriculum, not a news source, and it does not update itself.

- The reader wants code they can ship. Examples here are written to be understood, not deployed.


## How to Run

Open the file listed beside a concept in Quick Reference below. The files live in the `references` directory, are plain Markdown with no front matter and no dependencies — read the one you want and ignore the rest.


## Quick Reference


### Bayesian Methods

- `references/bayesian-inference-and-decision-theory.md` — **Bayesian Inference and Decision Theory**: Updating beliefs with Bayes' rule and then choosing the action that minimises expected…
- `references/fisher-information.md` — **Fisher Information**: A measure of how sharply the likelihood identifies a parameter, equivalently the curvat…
- `references/information-geometry.md` — **Information Geometry**: Treating families of probability distributions as curved manifolds, with divergences pl…

### Calculus

- `references/automatic-differentiation.md` — **Automatic Differentiation**: Computing exact derivatives by tracking elementary operations and applying the chain ru…
- `references/computational-graphs.md` — **Computational Graphs**: Representing a calculation as a directed graph of operations, so the forward pass evalu…
- `references/gradient-checking.md` — **Gradient Checking**: Verifying an analytic gradient against a finite-difference approximation, the standard…
- `references/matrix-calculus-and-jacobians.md` — **Matrix Calculus and Jacobians**: Differentiating functions whose inputs and outputs are vectors or matrices.

### Discrete Mathematics

- `references/catalan-numbers.md` — **Catalan Numbers**: A sequence counting many structurally similar objects — balanced parentheses, binary tr…
- `references/integer-partitions.md` — **Integer Partitions**: The ways of writing an integer as a sum of positive integers, ignoring order — a classi…
- `references/modular-arithmetic.md` — **Modular Arithmetic**: Arithmetic on remainders after division by a fixed modulus, giving a finite cyclic stru…
- `references/proof-by-induction.md` — **Proof by Induction**: Establishing a statement for all natural numbers by proving a base case and showing eac…

### Graph Theory

- `references/dijkstra-shortest-path.md` — **Dijkstra Shortest Path**: Finding shortest paths from a source in a graph with non-negative edge weights by repea…
- `references/graph-traversal-bfs-and-dfs.md` — **Graph Traversal BFS and DFS**: The two canonical ways to explore a graph: breadth-first expands level by level, depth-…
- `references/pagerank.md` — **PageRank**: Ranking nodes by the stationary distribution of a random walk that occasionally telepor…
- `references/social-network-analysis.md` — **Social Network Analysis**: Studying social structure through the graph of relationships — centrality, communities,…
- `references/topological-sort.md` — **Topological Sort**: Ordering the nodes of a directed acyclic graph so every edge points forward, which is e…

### Information Theory

- `references/cross-entropy-loss.md` — **Cross-Entropy Loss**: The negative log-likelihood of the correct class under the model's predicted distributi…
- `references/data-compression.md` — **Data Compression**: Encoding data in fewer bits, losslessly by exploiting statistical redundancy or lossily…
- `references/elbo.md` — **ELBO**: The evidence lower bound, the tractable objective variational inference maximises in pl…
- `references/huffman-compression.md` — **Huffman Compression**: Building an optimal prefix-free code by repeatedly merging the two least frequent symbo…
- `references/kl-divergence.md` — **KL Divergence**: The extra cost, in bits, of coding data from one distribution using a code built for an…
- `references/kl-divergence-asymmetry.md` — **KL Divergence Asymmetry**: Which distribution you place first changes what minimising KL divergence actually does…
- `references/mutual-information.md` — **Mutual Information**: How much knowing one variable reduces uncertainty about another, capturing any dependen…
- `references/shannon-entropy.md` — **Shannon Entropy**: The average surprise in a distribution, measured in bits, and the theoretical floor on…

### Linear Algebra

- `references/determinants-and-inverses.md` — **Determinants and Inverses**: The determinant measures how much a transformation scales volume, and vanishes exactly…
- `references/eigenvalues-and-eigenvectors.md` — **Eigenvalues and Eigenvectors**: Directions a transformation merely stretches rather than rotates, and the factors by wh…
- `references/gram-schmidt.md` — **Gram-Schmidt**: A procedure that turns any linearly independent set into an orthonormal one by repeated…
- `references/linear-transformations.md` — **Linear Transformations**: Maps that preserve addition and scalar multiplication.
- `references/matrix-multiplication.md` — **Matrix Multiplication**: Composing two linear transformations by taking dot products of rows with columns.
- `references/normal-equations.md` — **Normal Equations**: The linear system whose solution is the least-squares fit, obtained by requiring the re…
- `references/numpy-broadcasting.md` — **NumPy Broadcasting**: The rule set that lets arrays of different shapes combine by implicitly expanding size-…
- `references/orthogonality-and-projections.md` — **Orthogonality and Projections**: Two vectors are orthogonal when their dot product vanishes; a projection drops a vector…
- `references/singular-value-decomposition.md` — **Singular Value Decomposition**: Factoring any matrix into a rotation, a scaling and another rotation.
- `references/tensors-and-shapes.md` — **Tensors and Shapes**: Multi-dimensional arrays and the discipline of tracking their dimensions through a comp…
- `references/vector-spaces-and-rank.md` — **Vector Spaces and Rank**: A vector space is a set closed under addition and scaling; rank is the dimension of the…

### Numerical Methods

- `references/kalman-filter.md` — **Kalman Filter**: A recursive estimator combining a dynamic model with noisy measurements to track a syst…
- `references/quantum-computing.md` — **Quantum Computing**: Computation exploiting superposition and entanglement, promising asymptotic speedups on…

### Optimization

- `references/convex-optimization.md` — **Convex Optimization**: Problems where any local minimum is global, giving strong guarantees and reliable algor…
- `references/double-descent.md` — **Double Descent**: The observation that test error falls, rises around the interpolation threshold, then f…
- `references/duality-primal-and-dual.md` — **Duality Primal and Dual**: Every optimisation problem has a dual whose optimum bounds the original.
- `references/gradient-descent.md` — **Gradient Descent**: Iteratively stepping parameters against the gradient of the loss.
- `references/hyperparameter-tuning.md` — **Hyperparameter Tuning**: Searching over settings that are not learned from data — learning rate, depth, regulari…
- `references/inverse-problems.md` — **Inverse Problems**: Recovering unknown causes from observed effects, typically ill-posed because many cause…
- `references/kkt-conditions.md` — **KKT Conditions**: The first-order necessary conditions for optimality with inequality constraints — stati…
- `references/lagrange-multipliers.md` — **Lagrange Multipliers**: Folding equality constraints into the objective with multipliers, converting a constrai…
- `references/least-squares-regression.md` — **Least Squares Regression**: Fitting parameters by minimising the sum of squared residuals, the oldest and still mos…
- `references/line-search.md` — **Line Search**: Choosing how far to move along a chosen descent direction, either exactly or by accepti…
- `references/linear-programming-and-simplex.md` — **Linear Programming and Simplex**: Optimising a linear objective over linear constraints.
- `references/newton-method.md` — **Newton Method**: Using second-derivative information to jump toward the optimum, converging quadraticall…
- `references/nonsmooth-optimization.md` — **Nonsmooth Optimization**: Optimising objectives with kinks or discontinuous gradients, where classical descent as…
- `references/optimal-control.md` — **Optimal Control**: Choosing a control policy over time to optimise a cost functional subject to system dyn…
- `references/proximal-gradient-methods.md` — **Proximal Gradient Methods**: Splitting an objective into a smooth part handled by a gradient step and a nonsmooth pa…
- `references/stochastic-gradient-descent.md` — **Stochastic Gradient Descent**: Estimating the gradient from a minibatch instead of the full dataset, trading exactness…
- `references/training-dynamics.md` — **Training Dynamics**: How loss, gradients and representations evolve over the course of training, including w…

### Probability Theory

- `references/concentration-inequalities.md` — **Concentration Inequalities**: Bounds showing that a random quantity stays close to its expectation with high probabil…
- `references/empirical-cdf.md` — **Empirical CDF**: The step function that jumps by 1/n at each observation, estimating the cumulative dist…
- `references/fitting-distributions.md` — **Fitting Distributions**: Choosing a parametric family and estimating its parameters from data, then checking whe…
- `references/markov-decision-processes.md` — **Markov Decision Processes**: The formal model of sequential decision-making: states, actions, transition probabiliti…
- `references/sampling-distributions.md` — **Sampling Distributions**: The distribution of a statistic across repeated samples from the same population.

### Recursion

- `references/base-cases-and-recurrence.md` — **Base Cases and Recurrence**: The terminating conditions of a recursion and the rule relating a case to smaller ones…
- `references/depth-first-search-with-backtracking.md` — **Depth-First Search with Backtracking**: Exploring choices recursively and undoing them when a branch fails, the standard patter…
- `references/recursive-thinking.md` — **Recursive Thinking**: Framing a problem in terms of a smaller instance of itself, identifying the parameters,…
- `references/stack-memory-behaviour.md` — **Stack Memory Behaviour**: Each recursive call consumes a stack frame holding its locals and return address, so re…
- `references/tail-recursion.md` — **Tail Recursion**: A recursive call in final position, where nothing remains to do afterwards, allowing th…
- `references/tower-of-hanoi.md` — **Tower of Hanoi**: The classic puzzle of moving a stack of disks between pegs, whose recursive solution is…

### Statistical Inference

- `references/anova.md` — **ANOVA**: Analysis of variance partitions total variability into components attributable to group…
- `references/cross-validation-partitions.md` — **Cross-Validation Partitions**: Splitting data into folds so every observation serves once as validation, giving a less…
- `references/descriptive-statistics.md` — **Descriptive Statistics**: Summarising a dataset's central tendency, spread and shape before any inference — means…
- `references/gaussian-process-regression.md` — **Gaussian Process Regression**: A nonparametric Bayesian method placing a distribution over functions, returning both a…
- `references/hypothesis-testing.md` — **Hypothesis Testing**: A formal procedure for deciding whether data provide enough evidence against a null hyp…
- `references/mixed-effects-models.md` — **Mixed-Effects Models**: Models combining fixed effects shared across all observations with random effects varyi…
- `references/null-and-alternate-hypotheses.md` — **Null and Alternate Hypotheses**: The two competing statements a test adjudicates: a null of no effect, and an alternativ…
- `references/partial-least-squares.md` — **Partial Least Squares**: A regression method that constructs components maximising covariance with the response,…
- `references/q-q-plots-and-normality.md` — **Q-Q Plots and Normality**: Plotting sample quantiles against theoretical ones so that agreement appears as a strai…
- `references/scatterplots-and-relationships.md` — **Scatterplots and Relationships**: Plotting two variables against each other to reveal association, nonlinearity, clusteri…
- `references/significance-levels.md` — **Significance Levels**: The pre-chosen probability of rejecting a true null hypothesis, conventionally 0.05, wh…
- `references/sufficient-statistics.md` — **Sufficient Statistics**: A statistic that captures everything in the data relevant to a parameter, so that condi…


## Pitfalls

- The concepts are ordered by what rests on what, not by difficulty. A later concept is not always harder; it is later because something else has to come first.

- Explanations avoid notation the reader has not met. If you reintroduce standard symbols, say what they mean, or you undo the thing this curriculum is for.


## Verification

`ls references/ | wc -l` should print 77.


---

From [Learn AI](https://basavapurushottam.com/learn-ai). Prose is licensed CC BY 4.0.
