# Scattering Transforms

A convolutional architecture with fixed wavelet filters rather than learned ones, whose stability and invariance properties can be proven in advance.

**Rests on:** Convolution Arithmetic, Linear Transformations

*Deep Learning · Convolutional Networks*


## Why learn this

Scattering transforms offer a rare, transparent view into how signal processing works before any training happens. There is no learning from data, no guessing, and the behavior is guaranteed in advance. This makes it the clearest possible entry point for understanding how convolutional networks actually process information. By mastering this, you gain a solid foundation for exploring modern AI systems, including large language models, which rely on similar principles of pattern recognition and stability.


## The intuition

Imagine you have a one-dimensional signal—like a sound wave or a stock price over time—and you want to understand its structure. A scattering transform does this by repeatedly applying two operations: a filter that looks for specific local patterns (such as a sudden jump or a smooth curve), and an averaging operation that summarizes a local region by taking the average of values within it.

Crucially, these filters are fixed. They are not learned from data but are designed using base functions used to generate these filters to be sensitive to different frequencies and scales. The process builds a hierarchy: first it looks at the signal itself, then it captures the energy of the filtered patterns at different scales, then it sees how that energy varies across those scales, and so on.

For example, if you take a segment of audio, calculate its energy in one frequency band, then look at how that energy changes over time using another band, you get a description of the signal's structure. This output is stable to small deformations (like a slight time shift) and invariant to translations (shifting the whole signal does not change the result). Because everything is deterministic and linear—except for taking the absolute value of numbers (which we call modulus), which introduces non-linearity—the properties can be proven mathematically. You know in advance how the transform will behave, unlike a standard neural network where you only hope it generalizes well.


## An analogy

Think of a scattering transform as a team of inspectors examining a manuscript. The manuscript is the signal. The team has a set of fixed stencils—the base functions used to generate these filters—where one stencil detects sharp edges, another detects gentle slopes, and another detects periodic patterns. Each inspector slides their stencil across the page and notes how well it matches at each position. The raw match values are then pooled: for each stencil, the team records the average absolute match over a window. This gives a summary of what patterns are present and roughly where. But that is not enough—the team also wants to know how the pattern changes as you move your finger along the page. So they take another set of stencils, these ones finer, and slide them across the sequence of pooled matches, recording how much the match fluctuates. They pool that too. By iterating this procedure—stencil, match, take absolute value, pool—they build a hierarchical description that is robust to small shifts: if you shift the entire manuscript by a few centimeters, the team's summary barely changes, because they are looking at the pattern of matches, not the exact positions. This is exactly the translation invariance and deformation stability of the scattering transform. The core idea of fixed templates, local matching, and summarizing is exactly right.

It works like a fingerprint scanner that doesn't care where your finger lands on the pad, only that the ridges match the expected pattern.


## Definition

The scattering transform is a fixed architecture that applies a cascade of wavelet filters interleaved with modulus (taking the absolute value) and averaging operators, producing coefficients that are translation invariant and stable to small deformations, with properties that can be proven before any training.


## How it connects

This concept is the analytical backbone of convolutional networks. It sits at the intersection of your notes on Convolutional Layers (where filters are learned, here they are fixed) and Pooling as a Statistic (averaging is the pooling here, and it serves as a sufficiency argument). It also connects closely to Receptive Fields (deeper scattering layers see wider context) and Fourier Analysis of Networks (scattering transforms are often analyzed via their frequency response).


## The mathematics

The scattering transform is a way to summarize a signal (like an image or a sound wave) so that small changes in the input do not cause large changes in the output. It does this by combining three basic operations: filtering, taking absolute values, and averaging.

**1. Filtering (Convolution)**
Filtering means sliding a small window of numbers (called a filter or kernel) across your signal. At each position, you multiply the numbers under the window by the corresponding numbers in the filter, then add them up. This produces a new number for that position.

Imagine your signal is a row of heights: [1, 2, 3, 4, 5].
Imagine your filter is [0.5, 1, 0.5].
To find the first output number, you align the filter with the start of the signal:
(1 * 0.5) + (2 * 1) + (3 * 0.5) = 0.5 + 2 + 1.5 = 4.
You slide the filter one step to the right and repeat: (2 * 0.5) + (3 * 1) + (4 * 0.5) = 6.
The result is a new, shorter signal [4, 6, ...].

**2. Taking Absolute Values (Modulus)**
After filtering, some numbers might be negative. We take the absolute value of every number in the filtered signal. This means if a number is -3, it becomes 3. If it is 5, it stays 5.
Why? Because we care about the *strength* or *energy* of the feature, not its direction (positive or negative). This step also introduces non-linearity, which allows the system to detect complex patterns.

**3. Averaging (Pooling)**
Finally, we group the numbers from the previous step into small windows and replace each window with its average. For example, if the signal after absolute value is [4, 6, 8, 10], and our averaging window size is 2, we take the first two numbers: (4 + 6) / 2 = 5. Then the next two: (8 + 10) / 2 = 9. The result is [5, 9].
Averaging makes the result stable. If one number in the input changes slightly, the average changes only a little.

**The Scattering Cascade**
The power of scattering comes from repeating these steps. We do not just filter once. We filter, take absolute value, then filter *again* on the result of the absolute value, and so on.

Let S_k x be the output after k layers of this process.
- **Layer 0 (S_0 x):** Just smooth the original signal x using a low-pass filter (a filter that keeps only slow changes). This gives us the general shape.
- **Layer 1 (S_1 x):** 
  1. Filter x with a wavelet filter ψ (which detects edges or spikes).
  2. Take the absolute value: |x * ψ|.
  3. Average the result with a low-pass filter φ: S_1 x = |(x * ψ)| * φ.
- **Layer 2 (S_2 x):**
  1. Take the output from Layer 1, which is |x * ψ|.
  2. Filter it again with another wavelet filter ψ₂.
  3. Take the absolute value: ||x * ψ| * ψ₂|.
  4. Average it: S_2 x = ||(x * ψ) * ψ₂|| * φ.

Each layer captures details at a different scale. Layer 1 sees local edges. Layer 2 sees how those edges are arranged relative to each other. The final output is a set of numbers that describe the signal’s structure in a way that is robust to small shifts and distortions.


## A worked example

Let us walk through a complete example step-by-step.

**The Setup**
- Input signal x: [1, 2, 3, 4, 5]
- Wavelet filter ψ: [0.5, -1, 0.5] (This detects changes in slope)
- Low-pass averaging filter φ: [0.5, 0.5] (This averages pairs of numbers)

**Step 1: First Convolution (x * ψ)**
We slide the wavelet filter ψ over x. We assume 'valid' convolution, meaning we only compute where the filter fully overlaps the signal.

Position 1 (covering indices 0,1,2 of x):
(1 * 0.5) + (2 * -1) + (3 * 0.5) = 0.5 - 2 + 1.5 = 0

Position 2 (covering indices 1,2,3 of x):
(2 * 0.5) + (3 * -1) + (4 * 0.5) = 1 - 3 + 2 = 0

Position 3 (covering indices 2,3,4 of x):
(3 * 0.5) + (4 * -1) + (5 * 0.5) = 1.5 - 4 + 2.5 = 0

Result after convolution: [0, 0, 0]
*(Note: In this specific simple linear example, the wavelet detected no curvature because the signal is a straight line. Let us adjust the input slightly to show non-zero results. Let x = [1, 3, 2, 4, 5])* 

**Revised Step 1: First Convolution (x * ψ) with x = [1, 3, 2, 4, 5]**
Position 1:
(1 * 0.5) + (3 * -1) + (2 * 0.5) = 0.5 - 3 + 1 = -1.5

Position 2:
(3 * 0.5) + (2 * -1) + (4 * 0.5) = 1.5 - 2 + 2 = 1.5

Position 3:
(2 * 0.5) + (4 * -1) + (5 * 0.5) = 1 - 4 + 2.5 = -0.5

Result after convolution: [-1.5, 1.5, -0.5]

**Step 2: Modulus (Absolute Value)**
Take the absolute value of each number in the result from Step 1.
|-1.5| = 1.5
|1.5| = 1.5
|-0.5| = 0.5

Result after modulus: [1.5, 1.5, 0.5]

**Step 3: Averaging (Low-pass filter φ)**
We apply the averaging filter φ = [0.5, 0.5] to the result from Step 2.
Since our signal [1.5, 1.5, 0.5] has length 3 and the filter has length 2, we can compute two averaged values (assuming valid convolution again).

Average 1 (first two numbers):
(1.5 * 0.5) + (1.5 * 0.5) = 0.75 + 0.75 = 1.5

Average 2 (next two numbers):
(1.5 * 0.5) + (0.5 * 0.5) = 0.75 + 0.25 = 1.0

Final Output S_1 x: [1.5, 1.0]

**Interpretation**
The original signal was [1, 3, 2, 4, 5]. The scattering transform reduced this to two numbers: [1.5, 1.0]. These numbers represent the average strength of the edges detected in the first half and second half of the signal. If we had shifted the input signal slightly, these averages would change very little, demonstrating stability.


## From scratch, in Python

```python
# Plain Python: no frameworks, just lists and loops.
# We'll implement a 1D scattering transform for a tiny signal.

# Python building blocks used:
# - list: an ordered collection that can be changed, e.g., [1, 2, 3]
# - loop: repeat an action for each item, e.g., for i in range(len(x))
# - function: a reusable block of code that takes inputs and returns a result

# Step 0: define a simple moving-average filter (low-pass) and a wavelet filter (high-pass)
# For simplicity, we use a Haar-like filter: [0.5, -0.5] (difference) and a smoothing filter: [0.5, 0.5]

phi = [0.5, 0.5]   # averaging filter (low-pass)
psi = [0.5, -0.5]  # wavelet filter (high-pass, detects differences)

def convolve(signal, filt):
    """Convolve a 1D signal with a filter, returning the valid (full-overlap) outputs."""
    output = []
    # The output has length len(signal) - len(filt) + 1 if we only take valid positions
    for i in range(len(signal) - len(filt) + 1):
        # Compute the sum of signal[i+j] * filt[j] for each j
        acc = 0
        for j in range(len(filt)):
            acc = acc + signal[i + j] * filt[j]
        output.append(acc)
    return output

def modulus(signal):
    """Take the absolute value of each element."""
    return [abs(s) for s in signal]  # this is a list comprehension, a compact loop

def average(signal):
    """Return the average of the signal's elements."""
    if len(signal) == 0:
        return 0.0
    total = 0
    for s in signal:
        total = total + s
    return total / len(signal)

# Step 1: define an example signal
x = [1.0, 2.0, 3.0, 4.0, 5.0]

# Step 2: compute first-order scattering coefficient
#   first: convolve with wavelet
w = convolve(x, psi)      # w = [1*0.5 + 2*(-0.5), 2*0.5 + 3*(-0.5), ...] = [0.5-1, 1-1.5, 1.5-2, 2-2.5] = [-0.5, -0.5, -0.5, -0.5]
#   then: take modulus
m = modulus(w)            # m = [0.5, 0.5, 0.5, 0.5]
#   then: average (here we average over all coefficients, which is a global pooling)
S1 = average(m)           # S1 = (0.5+0.5+0.5+0.5)/4 = 0.5

print("First-order scattering coefficient:", S1)  # prints 0.5

# Step 3: compute second-order scattering coefficient
#   take the modulus of the first-order wavelet output, then convolve with another wavelet (say the same or different scale)
#   For simplicity, we reuse the same wavelet: convolve m with psi
w2 = convolve(m, psi)     # m = [0.5, 0.5, 0.5, 0.5], psi = [0.5, -0.5] -> w2 = [0.5*0.5 + 0.5*(-0.5), 0.5*0.5 + 0.5*(-0.5), ...] = [0, 0, 0]
#   then modulus and average
m2 = modulus(w2)          # m2 = [0, 0, 0]
S2 = average(m2)          # S2 = 0

print("Second-order scattering coefficient:", S2)  # prints 0.0

# In practice, scattering transforms use many wavelets at different scales and orientations.
# This toy example shows the core operations: convolution, modulus, pooling.
```


## The same thing, with a framework

```python
In practice, you would use a library like Kymatio or PyTorch. Kymatio provides a Scattering1D class that computes the full scattering transform efficiently. Here's how you'd do the same in PyTorch (a tensor is a list of lists that also knows its shape):

import torch
import kymatio
from kymatio.torch import Scattering1D

# Set up a scattering transform with 8 wavelets per octave, 2 octaves of scales
scattering = Scattering1D(J=2, shape=8, Q=8)  # shape is the signal length, J is the number of scales

x = torch.tensor([1.0, 2.0, 3.0, 4.0, 5.0])  # input signal as a tensor
S = scattering(x)  # returns a tensor of scattering coefficients, shape [number_of_coefficients]

print(S)  # you'll see the coefficients including zeroth, first, and second order.

# Underneath, Kymatio does exactly what we did in plain Python: it convolves with wavelets, takes absolute values, and averages, but it uses optimized Fast Fourier Transforms and GPU support.
```


## What the code does

The plain Python code first defines two filters: phi (averaging) and psi (wavelet). The 'convolve' function slides the filter across the signal, multiplying and adding, producing a shorter output. 'modulus' takes the absolute value of each element. 'average' computes the mean. We then compute S1 by convolving x with psi, taking the modulus, and averaging—this gives 0.5, which reflects the average local difference in the signal. S2 does the same on the modulus coefficients, yielding 0, because the differences are constant (no variation). In the framework version, Kymatio encapsulates all this, using a built-in Scattering1D class. The tensor result includes all coefficients; you don't have to write the loops yourself.


## Common misconceptions

**Scattering filters need to be trained** — The scattering transform is just a convolutional network without training, so it must be less powerful or need to learn its filters eventually.

The filters are fixed wavelets chosen analytically, not learned. They are designed to satisfy specific mathematical properties (e.g., They are band-pass filters). Training is unnecessary because the stability and invariance properties are guaranteed by construction. That is what makes it a 'provably works before training' architecture. In fact: Scattering transforms use fixed wavelets; training is not needed to achieve translation invariance and deformation stability. Learning would trade away those guarantees.

**Scattering is the same as a regular CNN** — A scattering transform is just a CNN with fixed random filters, so it behaves like any convolutional network.

Regular CNNs learn filters from data, which means their behavior is not analytically predictable. Scattering uses a cascade of specific wavelets and non-linear modulus, giving it provable stability and invariance—not just empirical. In fact: Scattering is an analytically designed convolutional architecture with provable properties; CNNs rely on learned filters with empirical performance.

**Translation invariance means the output is identical for any shift** — If I shift the input signal, the scattering output will be exactly the same for all shifts, even at the boundaries.

Translation invariance is approximate, not exact, because the finite signal and boundary effects (padding) cause small changes. However, as the signal length grows, the output becomes essentially invariant to shifts. In fact: Scattering gives near translation invariance, with small boundary differences; for large signals, the effect is negligible.

**More scales (J) is always better** — If J=2 gives good features, J=10 will give even better results.

Larger J means more scales, which increases the number of coefficients and computation. Beyond a point, additional scales capture only fine details that are often noise, and can overfit or slow down. There's a trade-off between discriminative power and computational cost. In fact: The number of scales J is a hyperparameter; you need to choose it based on the signal's frequency content and available compute.

**Scattering is only useful for audio or images** — Scattering transforms are a signal-processing trick for audio, not for other data like text or tabular data.

While the classic use is in audio and image analysis, scattering has been applied to any data that can be represented as a signal—time series, graphs, and even text (if you discretize characters as a signal). The underlying operations (convolution, modulus, pooling) are generic. In fact: Scattering transforms work on any ordered data where local pattern detection is useful, not just audio/images.


## In short

The scattering transform is a fixed, provably stable convolutional feature extractor. It uses wavelets instead of learned filters, applies convolution, modulus, and pooling in a cascade, and guarantees translation invariance and deformation stability. It is the analytical skeleton of convolutional networks—the version that works before training. Remember: filters are fixed, not learned; the mathematics is proven, not empirical; and it is a powerful tool for low-resource and interpretable signal processing.


## Where this is used

Scattering transforms appear in several places in modern AI, often as a principled front-end that guarantees stability and invariance before any learned components are added.

**Large language models (LLMs):** LLMs process text, not raw signals, but they often use tokenization and embeddings that could be seen as a learned analogue of scattering. The scattering transform itself isn't used directly in LLM architectures like transformers, but the mathematical principles—translation invariance and deformation stability—guide how positional encodings are designed. For instance, the sinusoidal positional embeddings in the original transformer are similar in spirit to wavelets: they capture scale and frequency. A researcher might use scattering transforms to analyze the frequency content of token embeddings or to design fixed, provably stable input representations that are then fed to a transformer.

**Small language models (on-device):** On devices with limited compute, you cannot afford a huge learned transformer. A scattering transform can serve as a fixed, cheap feature extractor that reduces the input signal (e.g., audio or sensor data) into a compact, informative representation, which is then passed to a small classifier or a tiny neural network. Because the scattering filters are fixed, they require no training and no backpropagation, making them ideal for real-time processing on a phone. For example, a keyword-spotting system might use a scattering transform on raw audio to extract stable features that are then classified by a simple logistic regression, achieving high accuracy with minimal power.

**Retrieval-augmented generation (RAG):** RAG systems retrieve documents from a vector database based on embeddings. Scattering transforms can be used to compute embeddings that are stable to small variations in the input—for example, a scattering-based embedding of a text passage could be more robust to rephrasing or minor typos than a learned embedding. In medical imaging, scattering transforms have been used to compute features that are stable to deformations, which could be applied to image retrieval. However, in text RAG, transformers' learned embeddings are typically used; scattering transforms are more common in signal processing domains. A developer might replace a learned embedding with a scattering transform to get provable invariance, but the trade-off is that scattering features are less expressive for high-level semantics.

**Agentic systems (planning and tools):** Agents that plan and act in loops often need to process sensor data or real-time streams. A scattering transform provides a stable, low-latency feature extractor that can be run on each incoming frame, feeding a decision policy. For example, a robot's vision system might use a scattering transform to detect edges and shapes invariantly to lighting changes, giving the agent a reliable perception layer. The deterministic nature of the transform means the agent's behavior is more predictable and debuggable—you know exactly how the input is being encoded, unlike a trained CNN.

**Everyday applied ML (vision, forecasting, fraud, speech):** 
- **Vision:** Scattering transforms are used for texture classification and object recognition, especially when labeled data is scarce. Because the filters are fixed, they can be precomputed and used with a shallow classifier, achieving strong performance on small datasets (e.g., in medical imaging where data is limited).
- **Forecasting:** In time-series forecasting, scattering transforms capture multi-scale temporal patterns that are stable to noise and small shifts, making them a robust feature extractor for predicting stock prices, energy demand, or weather.
- **Fraud detection:** Scattering features of transaction sequences can reveal patterns that are invariant to small changes in timing or amount, helping detect anomalous behavior with fewer false positives.
- **Speech:** Speech recognition often uses scattering transforms (or mel-spectrograms, which are related) as a front-end. The scattering transform provides a richer representation than mel-spectrograms because it captures second-order modulation, improving robustness to noise and speaker variation.

**What a developer actually writes:** In practice, a developer would use the Kymatio library (for Python) or the ScatteringTransform in PyTorch. For a vision task, they'd call `Scattering2D` on a batch of images, get the scattering coefficients, and feed them to a small classifier. For audio, `Scattering1D` or `Scattering3D` for video. They would write code like:

```python
from kymatio.torch import Scattering2D
scattering = Scattering2D(J=4, shape=(224, 224))
x = torch.randn(1, 3, 224, 224)  # dummy image
S = scattering(x)  # scattering coefficients, shape (1, num_channels, height, width)
```

**What breaks when done badly:** If you use scattering transforms without understanding the scale parameter J, you might undersmooth or oversmooth the features. Too few scales miss important structure; too many scales increase computation and memory. Also, scattering coefficients are not invariant to all transformations—they are invariant to translations but not to rotations (unless you use specific filters). Using them in a context where rotation invariance is needed without accounting for it will degrade accuracy. Finally, scattering transforms are not learned, so they may not adapt to the specific data distribution—if the signal characteristics differ from the wavelets' assumptions (e.g., text vs. audio), the features may be suboptimal. But for many signal-processing tasks, they provide a solid, provably stable foundation.


---

Read this on the web: https://basavapurushottam.com/learn-ai/scattering-transforms
