---
name: deep-learning
description: "Deep Learning: 55 concepts, no maths assumed."
version: 1.0.0
author: Learn AI
license: CC-BY-4.0
metadata:
  hermes:
    tags: [Deep, Learning]
    related_skills: [machine-learning, mathematical-foundations]
    source: https://basavapurushottam.com/learn-ai
---
# Deep Learning

55 concepts, written for an adult with no computer science and no mathematics past school arithmetic. Each concept is a file under `references/`; load one when it is the thing being asked about rather than reading the whole domain.


## When to Use

- The reader asks about deep learning and wants it explained from the ground up rather than summarised.

- A term below comes up and you want the plain-language account of it, with a worked example.

- You need to know what a concept rests on before teaching it — every reference lists its prerequisites.


## When NOT to Use

- The reader wants current events, papers or library documentation. This is a curriculum, not a news source, and it does not update itself.

- The reader wants code they can ship. Examples here are written to be understood, not deployed.


## How to Run

Open the file listed beside a concept in Quick Reference below. The files live in the `references` directory, are plain Markdown with no front matter and no dependencies — read the one you want and ignore the rest.


## Quick Reference


### Backpropagation

- `references/shortcut-connections.md` — **Shortcut Connections**: Paths that add a layer's input directly to its output, giving gradients an unobstructed…
- `references/upstream-gradients.md` — **Upstream Gradients**: The gradient arriving at a node from later in the network, which local derivatives mult…

### Computer Vision

- `references/image-segmentation.md` — **Image Segmentation**: Assigning a label to every pixel, delineating exact object boundaries rather than bound…

### Convolutional Networks

- `references/cnn-visualization-for-interpretation.md` — **CNN Visualization for Interpretation**: Inspecting what a convolutional network has learned by visualising filters, activation…
- `references/convolution-arithmetic.md` — **Convolution Arithmetic**: The relationship between input size, kernel size, stride, padding and dilation that det…
- `references/convolutional-layers.md` — **Convolutional Layers**: Layers applying learned filters across a signal with shared weights, detecting the same…
- `references/fourier-analysis-of-networks.md` — **Fourier Analysis of Networks**: Analysing what frequencies a network represents and learns, revealing systematic prefer…
- `references/pooling-as-a-statistic.md` — **Pooling as a Statistic**: Summarising a local region with a single value — max or average — reducing resolution w…
- `references/receptive-fields.md` — **Receptive Fields**: The region of the input that influences a particular unit's activation, growing with ea…
- `references/scattering-transforms.md` — **Scattering Transforms**: A convolutional architecture with fixed wavelet filters rather than learned ones, whose…
- `references/stride-and-padding.md` — **Stride and Padding**: Stride is the step between filter applications; padding adds border values so filters c…

### Deep Learning

- `references/activation-functions.md` — **Activation Functions**: The nonlinearity applied after each layer's linear transformation.
- `references/autoencoders.md` — **Autoencoders**: Networks trained to reconstruct their input through a narrow bottleneck, forcing a comp…
- `references/barron-spaces.md` — **Barron Spaces**: A function class whose approximation by shallow networks escapes the curse of dimension…
- `references/dropout.md` — **Dropout**: Randomly zeroing units during training so the network cannot rely on any single pathway…
- `references/forward-propagation.md` — **Forward Propagation**: Computing a network's output by passing inputs layer by layer through weights, biases a…
- `references/gelu.md` — **GELU**: A smooth activation that weights an input by the probability a standard normal falls be…
- `references/keras-and-tensorflow.md` — **Keras and TensorFlow**: A high-level model-building API over a production-oriented deep learning runtime, empha…
- `references/kolmogorov-arnold-networks.md` — **Kolmogorov-Arnold Networks**: Networks placing learnable activation functions on edges rather than fixed activations…
- `references/loss-function-selection.md` — **Loss Function Selection**: Choosing the objective that encodes what counts as a good prediction — squared error, a…
- `references/multi-layer-perceptrons.md` — **Multi-Layer Perceptrons**: Fully connected feedforward networks — the baseline architecture where every unit conne…
- `references/neural-tangent-kernel.md` — **Neural Tangent Kernel**: In the infinite-width limit, gradient descent on a network behaves like kernel regressi…
- `references/perceptron-and-decision-boundaries.md` — **Perceptron and Decision Boundaries**: The single-unit linear classifier and the hyperplane it induces — the historical starti…
- `references/relu-and-dead-neurons.md` — **ReLU and Dead Neurons**: ReLU passes positives unchanged and zeroes negatives.
- `references/sigmoid-and-tanh.md` — **Sigmoid and Tanh**: The classic saturating activations, squashing inputs into a bounded range.
- `references/softmax-and-logits.md` — **Softmax and Logits**: Logits are the raw unnormalised scores a network produces; softmax exponentiates and no…
- `references/universal-approximation-theorems.md` — **Universal Approximation Theorems**: Results showing that sufficiently wide networks can approximate any continuous function…
- `references/vanishing-gradients.md` — **Vanishing Gradients**: Gradients shrinking multiplicatively as they propagate backwards, so early layers recei…
- `references/width-vs-depth.md` — **Width vs Depth**: Whether to add units per layer or add layers — a tradeoff between representational effi…
- `references/xavier-initialization.md` — **Xavier Initialization**: Setting initial weight scale from layer fan-in and fan-out so activation and gradient v…

### Multimodal AI

- `references/cross-modal-fusion.md` — **Cross-Modal Fusion**: Combining representations from different modalities into a joint one the model can reas…
- `references/diffusion-image-generation.md` — **Diffusion Image Generation**: Producing images by iteratively denoising from random noise, conditioned on a text prompt.
- `references/modality-specific-pipelines.md` — **Modality-Specific Pipelines**: Separate ingestion and processing paths per modality before any fusion, because video,…
- `references/trocr-and-ocr.md` — **TrOCR and OCR**: Converting images of text into machine-readable characters, now commonly with transform…
- `references/vision-encoders.md` — **Vision Encoders**: Components turning images into embeddings a language model can consume, usually a pretr…

### Natural Language Processing

- `references/bi-encoders-and-cross-encoders.md` — **Bi-Encoders and Cross-Encoders**: Bi-encoders embed query and document separately for fast retrieval; cross-encoders scor…
- `references/emotional-intelligence-in-nlp.md` — **Emotional Intelligence in NLP**: Systems that recognise and respond appropriately to affect, going beyond polarity to in…
- `references/fake-news-detection.md` — **Fake News Detection**: Identifying misleading or fabricated content from linguistic signals, source characteri…
- `references/nlp-for-mobility-and-transportation.md` — **NLP for Mobility and Transportation**: Applying language processing in transport contexts — incident reports, passenger commun…
- `references/sentiment-analysis.md` — **Sentiment Analysis**: Classifying the polarity or attitude expressed in text, the most widely deployed text c…
- `references/speech-emotion-recognition.md` — **Speech Emotion Recognition**: Inferring emotional state from vocal characteristics — prosody, energy and spectral fea…

### Neural Network Architectures

- `references/cnn-lstm-hybrids.md` — **CNN-LSTM Hybrids**: Stacking convolutional feature extraction under a recurrent layer, so spatial structure…
- `references/encoder-decoder.md` — **Encoder-Decoder**: An architecture that compresses input into a representation and then generates output f…
- `references/lstm-cell-state.md` — **LSTM Cell State**: The carried vector running through an LSTM across time steps, modified only by gentle g…
- `references/lstm-networks.md` — **LSTM Networks**: Recurrent networks with gated cells designed to carry information across long sequences…
- `references/layer-normalization.md` — **Layer Normalization**: Normalising activations across features within a single example, independent of batch c…
- `references/stateful-and-bidirectional-lstms.md` — **Stateful and Bidirectional LSTMs**: Stateful variants carry state across batches for continuous sequences; bidirectional on…

### Transformers

- `references/causal-attention-masking.md` — **Causal Attention Masking**: Preventing a position from attending to later positions by masking future scores before…
- `references/cross-attention.md` — **Cross-Attention**: Attention where queries come from one sequence and keys and values from another, condit…
- `references/feed-forward-blocks.md` — **Feed-Forward Blocks**: The position-wise multilayer perceptron applied after attention in each transformer lay…
- `references/gpt-architecture.md` — **GPT Architecture**: A decoder-only transformer trained autoregressively — stacked masked self-attention and…
- `references/multi-head-attention.md` — **Multi-Head Attention**: Running several attention operations in parallel on projected subspaces, then concatena…
- `references/positional-encoding.md` — **Positional Encoding**: Injecting order information into attention, which is otherwise permutation-invariant, t…
- `references/rope-rotary-embeddings.md` — **RoPE Rotary Embeddings**: Encoding position by rotating query and key vectors by an angle proportional to positio…
- `references/self-attention-from-scratch.md` — **Self-Attention from Scratch**: Computing queries, keys and values from the same sequence, scoring every position again…


## Pitfalls

- The concepts are ordered by what rests on what, not by difficulty. A later concept is not always harder; it is later because something else has to come first.

- Explanations avoid notation the reader has not met. If you reintroduce standard symbols, say what they mean, or you undo the thing this curriculum is for.


## Verification

`ls references/ | wc -l` should print 55.


---

From [Learn AI](https://basavapurushottam.com/learn-ai). Prose is licensed CC BY 4.0.
