# Pooling as a Statistic

Summarising a local region with a single value — max or average — reducing resolution while retaining the presence of a feature.

**Rests on:** Descriptive Statistics

*Deep Learning · Convolutional Networks*


## Why learn this

Pooling is the quiet workhorse of convolutional networks. It shrinks an image or feature map while keeping the important bits — a face still looks like a face even if you blur and shrink it. This matters because: 1) It makes the network more robust to small shifts (a cat is still a cat if it moves a few pixels), 2) It drastically cuts the number of calculations, which is why your phone can run a vision model without melting, and 3) It is the step that lets later layers see a bigger picture without exploding in size. You will meet pooling again in nearly every convolutional network, from image classifiers to the early stages of large language models that process images. Learning it now gives you the vocabulary to understand how spatial information is compressed and why some networks are faster than others.


## The intuition

Imagine you have a photograph of a forest, and you want to know if there is a bear in it. You don't need to look at every single pixel — you just need to know if a bear-shaped blob exists. Pooling is the act of squinting. You divide the image into small blocks, and for each block you keep only a single summary number: the brightest pixel (max pooling) or the average brightness (average pooling). The result is a smaller image that still shows the bear, just fuzzier. This is what pooling does to feature maps inside a neural network: it takes a region of numbers, reduces it to one number, and slides that region across the whole map. To understand how this sliding works, consider the concept of stride. Stride is simply how far the window moves each time it takes a measurement. If you have a 4-pixel wide block and you choose a stride of 4, your next measurement starts exactly where the previous one ended, with no overlap. If you choose a stride of 2, each new measurement overlaps by half its width with the previous one. The network loses fine detail but keeps the presence of a feature — a nose, an edge, a texture — which is often all that matters. It also gives a little bit of translation invariance: if the bear shifts by a few pixels, the summary for each block barely changes, so the network still says 'bear'.


## An analogy

Think of reading a newspaper through a frosted glass window. You can't read the fine print, but you can tell if there is a big headline, a photo, or an advertisement. Pooling does the same: it deliberately throws away fine detail to keep the big picture. A max pooling window is like asking 'is there any bright spot here?' — it takes the maximum value in the region. An average pooling window is like asking 'what is the general brightness here?' — it averages the values. The window slides across the whole image, step by step, covering non-overlapping regions (though you can also use overlapping windows with a stride). This analogy holds well until you need to locate a feature precisely: pooling blurs locations, so if you need exact pixel positions (like for drawing a bounding box), pooling can hurt. That is where the analogy breaks — pooling is great for 'is there a bear?', bad for 'exactly where is its nose?'.


## Definition

Pooling is a downsampling operation that takes a small region of a feature map (the receptive field) and replaces it with a single summary statistic — usually the maximum or the average — thereby reducing spatial resolution while preserving the presence of salient features.


## How it connects

This concept builds on nothing yet — it is your first step. It belongs to the parent concept of Convolutional Networks, and it will connect to: Convolutional Layers (the filters that create the feature maps you pool), Stride and Padding (the arithmetic of how windows move), Receptive Fields (the region each pooled value summarizes), and later, more advanced topics like Scattering Transforms, which replace learned pooling with fixed wavelet filters. You will also see pooling in the context of CNN Visualization, where you inspect what the network has learned.


## The mathematics

Pooling is a way to summarize a group of numbers into one number. In mathematics, this kind of summary is called a statistic. Common statistics are the average (the sum divided by the count), the maximum (the largest number in the group), and the minimum (the smallest). In convolutional networks, we use pooling to shrink the size of the data while keeping the most important information.

There are two main types of pooling you will meet: max pooling and average pooling. Both work on a small square window that moves across the input grid. This window is called the kernel. The size of the window is k by k. For example, if k is 2, the window covers 4 numbers at once (2 rows times 2 columns). The window does not always move one step at a time. It jumps by a distance called the stride, denoted by s. If the stride is equal to the kernel size (s = k), the windows do not overlap. This is the most common case.

Let us look at how we calculate the size of the output grid. Suppose the input grid has height H and width W. The output grid will have a smaller height and width because we are grouping numbers together. The new height is found by taking the input height H, subtracting the window height k, dividing by the stride s, and then rounding down to the nearest whole number. We add 1 to this result. In plain words:

New Height = floor((H minus k) divided by s plus 1)

The new width is calculated exactly the same way, using W instead of H:

New Width = floor((W minus k) divided by s plus 1)

Here, 'floor' means round down. For example, if the result is 3.9, the floor is 3. If the result is 4.0, the floor is 4.

Now let us look at what number goes into each cell of the output grid. The output grid has rows indexed by i and columns indexed by j. For each position (i, j) in the output, we look back at the input grid to find which numbers belong to that window. The top-left corner of this window in the input grid is at row i times s and column j times s. Inside this window, we use offsets p and q to reach every number. The offset p goes from 0 up to k minus 1 (the rows inside the window). The offset q goes from 0 up to k minus 1 (the columns inside the window).

For max pooling, the value at output position (i, j) is simply the largest number found in that window. We compare every number input[i*s + p, j*s + q] for all valid p and q, and pick the biggest one.

For average pooling, the value at output position (i, j) is the sum of all numbers in that window, divided by how many numbers are in the window. Since the window is k by k, there are k times k numbers. So we add them all up and divide by k squared.

This process repeats for every i and j until the entire output grid is filled.


## A worked example

Let us work through a concrete example with real numbers.

Suppose our input feature map is a 4 by 4 grid of numbers. We want to apply max pooling with a window size k = 2 and a stride s = 2. This means each window covers 2 rows and 2 columns, and the window jumps 2 steps at a time.

First, let us determine the size of the output grid. The input height H is 4. The input width W is 4. The kernel size k is 2. The stride s is 2.

New Height = floor((4 minus 2) divided by 2 plus 1)
= floor(2 divided by 2 plus 1)
= floor(1 plus 1)
= floor(2)
= 2

New Width = floor((4 minus 2) divided by 2 plus 1)
= 2

So the output grid will be 2 by 2. There are four positions in the output: (0,0), (0,1), (1,0), and (1,1). We calculate each one separately.

Position (0,0) of the output:
Here i = 0 and j = 0. The window starts at input row 0 times 2 = 0, and column 0 times 2 = 0. The window covers rows 0 to 1 and columns 0 to 1 in the input.

The numbers in this top-left 2x2 block are:
Row 0: 1, 3
Row 1: 5, 6

For max pooling, we find the maximum of {1, 3, 5, 6}. The largest number is 6. So output[0,0] = 6.

For average pooling, we sum these numbers and divide by 4 (since k*k = 2*2 = 4).
Sum = 1 + 3 + 5 + 6 = 15.
Average = 15 divided by 4 = 3.75. So output[0,0] = 3.75.

Position (0,1) of the output:
Here i = 0 and j = 1. The window starts at input row 0 times 2 = 0, and column 1 times 2 = 2. The window covers rows 0 to 1 and columns 2 to 3 in the input.

The numbers in this top-right 2x2 block are:
Row 0: 2, 4
Row 1: 7, 8

For max pooling, we find the maximum of {2, 4, 7, 8}. The largest number is 8. So output[0,1] = 8.

For average pooling, we sum these numbers and divide by 4.
Sum = 2 + 4 + 7 + 8 = 21.
Average = 21 divided by 4 = 5.25. So output[0,1] = 5.25.

Position (1,0) of the output:
Here i = 1 and j = 0. The window starts at input row 1 times 2 = 2, and column 0 times 2 = 0. The window covers rows 2 to 3 and columns 0 to 1 in the input.

The numbers in this bottom-left 2x2 block are:
Row 2: 9, 10
Row 3: 13, 14

For max pooling, we find the maximum of {9, 10, 13, 14}. The largest number is 14. So output[1,0] = 14.

For average pooling, we sum these numbers and divide by 4.
Sum = 9 + 10 + 13 + 14 = 46.
Average = 46 divided by 4 = 11.5. So output[1,0] = 11.5.

Position (1,1) of the output:
Here i = 1 and j = 1. The window starts at input row 1 times 2 = 2, and column 1 times 2 = 2. The window covers rows 2 to 3 and columns 2 to 3 in the input.

The numbers in this bottom-right 2x2 block are:
Row 2: 11, 12
Row 3: 15, 16

For max pooling, we find the maximum of {11, 12, 15, 16}. The largest number is 16. So output[1,1] = 16.

For average pooling, we sum these numbers and divide by 4.
Sum = 11 + 12 + 15 + 16 = 54.
Average = 54 divided by 4 = 13.5. So output[1,1] = 13.5.

Final Results:
The max-pooled grid is:
6    8
14   16

The average-pooled grid is:
3.75   5.25
11.5   13.5


## From scratch, in Python

```python
import numpy as np

def max_pool_2x2(image):
    """Apply 2x2 max pooling with stride 2."""
    h, w = image.shape
    out_h = h // 2
    out_w = w // 2
    pooled = np.zeros((out_h, out_w))
    for i in range(out_h):
        for j in range(out_w):
            # Extract the 2x2 block
            block = image[2*i:2*i+2, 2*j:2*j+2]
            pooled[i, j] = np.max(block)
    return pooled

def average_pool_2x2(image):
    """Apply 2x2 average pooling with stride 2."""
    h, w = image.shape
    out_h = h // 2
    out_w = w // 2
    pooled = np.zeros((out_h, out_w))
    for i in range(out_h):
        for j in range(out_w):
            block = image[2*i:2*i+2, 2*j:2*j+2]
            pooled[i, j] = np.mean(block)
    return pooled

# Example: a 4x4 image
example = np.array([[1, 3, 2, 4],
                    [5, 6, 7, 8],
                    [9, 10, 11, 12],
                    [13, 14, 15, 16]])

print("Max pooled:")
print(max_pool_2x2(example))
print("Average pooled:")
print(average_pool_2x2(example))
```


## The same thing, with a framework

```python
In PyTorch, you can use torch.nn.MaxPool2d and torch.nn.AvgPool2d. For example:

import torch
import torch.nn as nn

x = torch.tensor([[[[1., 3., 2., 4.],
                     [5., 6., 7., 8.],
                     [9., 10., 11., 12.],
                     [13., 14., 15., 16.]]]])

max_pool = nn.MaxPool2d(kernel_size=2, stride=2)
avg_pool = nn.AvgPool2d(kernel_size=2, stride=2)

print("Max pooled:")
print(max_pool(x))
print("Average pooled:")
print(avg_pool(x))
```


## What the code does

The from-scratch code defines two functions. `max_pool_2x2` and `average_pool_2x2`. They both take a 2D numpy array (a list of lists) as input. The `shape` attribute gives the number of rows and columns; we store them in `h` and `w`. The output dimensions are half the input dimensions (since we use a 2x2 window with stride 2), so we compute `out_h = h // 2` and `out_w = w // 2`. The `//` operator is integer division — it rounds down to a whole number. We create an empty array of that size using `np.zeros` — this fills it with zeros. Then we loop over every position `(i, j)` in the output. For each, we extract a 2x2 block from the input using `image[2*i:2*i+2, 2*j:2*j+2]` — this is slicing, which grabs a sub-grid. `np.max(block)` finds the maximum value in that block; `np.mean(block)` finds the average. We assign that single number to the output at `pooled[i, j]`. After all loops, the function returns the pooled array. The example at the bottom uses the same 4x4 grid from the maths section, and the printed results match the hand-calculated numbers.


## Common misconceptions

**Pooling is just downsampling, a way to make the image smaller.** — Pooling is only there to reduce the size of the feature map, like shrinking an image in a photo editor.

Downsampling is the effect, but the purpose is to compute a statistic that keeps the presence of a feature. Max pooling keeps the strongest response in a region; average pooling keeps the general level. It is a sufficiency argument, not just a size reduction. The choice of statistic matters — max is good for edges, average is good for smoothing. In fact: Pooling summarizes a region with a statistic. It compresses the representation while retaining the features that matter.

**Max pooling and average pooling are the same thing.** — Since both reduce the size, it doesn't matter which one you use.

Max and average behave differently. Max keeps the strongest response — ideal for detecting a bright edge or a specific pattern. Average smooths out noise and gives an overall sense of the region. That difference can change the model's performance on a task. In fact: Max pooling keeps the maximum value in a window, emphasizing the presence of a feature. Average pooling takes the mean, which is smoother and more noise-robust.

**Pooling is necessary and cannot be replaced.** — You always need pooling to reduce the size of feature maps.

Strided convolutions can learn to downsample instead of using a fixed statistic. They are now common in many networks, and they often perform better because they adapt to the data. In fact: Pooling is a fixed, non-learned downsampler. Strided convolutions are a learned alternative that can replace it in modern architectures.

**Pooling gives perfect translation invariance.** — If you shift the input by a few pixels, pooling ensures the output is exactly the same.

Pooling reduces sensitivity to small shifts but does not guarantee invariance. If the feature moves across a boundary between pooling windows, the summary can change significantly. It is approximate, not exact. In fact: Pooling provides approximate translation invariance — small shifts often leave the pooled values unchanged, but it is not guaranteed.


## In short

Pooling is a statistic that summarizes a local region of a feature map — max pooling keeps the strongest value, average pooling keeps the mean. It reduces spatial size, cuts computation, and provides a small amount of translation invariance. It is a sufficiency argument: you keep the presence of a feature, not its exact location. Strided convolutions are a learned alternative that often replace pooling in modern models. If you remember one thing: pooling is about summarising, not just shrinking.


## Where this is used

Pooling appears everywhere in convolutional neural networks, but it is also being slowly replaced by strided convolutions, which learn how to downsample rather than using a fixed statistic.

**Large language models (LLMs) that process images.** Many modern multimodal LLMs (like those that accept image inputs) use a vision encoder based on a convolutional network. That encoder includes pooling layers (or strided convolutions) to progressively shrink the image representation from, say, 224×224 pixels down to a small grid of features, like 7×7. Each pooled feature vector then gets fed into the transformer as a token, just like a word. Without pooling, the number of tokens would be enormous, and the transformer would be too slow. So pooling is what makes image-understanding possible.

**Small language models and on-device AI.** When a model has to run on a phone or a smart speaker, every operation costs battery and time. Pooling reduces the spatial size of feature maps, so the number of calculations in later layers drops dramatically. For example, after applying a 2×2 max pooling, the feature map is a quarter of the size, so the next convolution does a quarter of the multiplications. That is why many tiny keyword spotters (like 'Hey Siri') use a convolutional front-end with pooling before a small recurrent network. The pooling step is what fits the model into memory.

**Retrieval-augmented generation (RAG).** RAG systems retrieve documents by comparing embeddings (lists of numbers) of text. Pooling shows up in the embedding models themselves. Many sentence-embedding models use a transformer, but they still need to turn a sequence of tokens into a single vector per sentence. Some do this by pooling (often mean pooling) over all token embeddings. The final sentence vector is the average of the token vectors. This is exactly the pooling concept applied to the sequence dimension. Without it, you'd have a vector per token, which is harder to compare. So pooling is part of the machinery that matches your question to the right documents.

**Agentic systems.** Agents often need to call tools that return images or grids — like a map, a chart, or a screenshot. To feed this back to the language model, they must compress it into a fixed-size representation. Pooling (or strided conv) is one of the standard ways to do that compression. For example, an agent that reads a dashboard screenshot might run it through a small CNN with pooling to get a compact set of feature maps, then pass that to the model. The agent's 'vision' is only possible because pooling reduces the huge pixel array into something smaller and more manageable.

**Everyday applied ML.** In image classification (like detecting defects in factory parts), pooling appears after each convolutional layer in a classic CNN. For fraud detection, if the input is a time series, 1D pooling can summarize a window of transactions into a single statistic, helping the model focus on the presence of a spike. In forecasting, average pooling over a sliding window is essentially a moving average, which smooths out noise before feeding into a deeper model.

**What a developer actually writes.** In PyTorch, you add `nn.MaxPool2d(kernel_size=2, stride=2)` between convolutional layers. In TensorFlow/Keras, you write `layers.MaxPooling2D(pool_size=(2,2))`. In many modern architectures, developers replace pooling with a strided convolution (a conv layer with `stride=2`), which learns the best way to downsample instead of using a fixed max or average. That is the learned alternative. It is not that pooling is bad — it is that a learned downsampler can, in principle, do at least as well and often better.

**What breaks when done badly.** If pooling is applied too aggressively (e.g., a 4×4 window on a small feature map), you lose too much spatial detail and the model cannot locate objects. If you use average pooling when you need to preserve sharp edges (like in edge detection), the edges blur away. And in localization tasks (like object detection), pooling hurts because it destroys the exact positions of features — that is why modern detectors use strided convolutions instead.


---

Read this on the web: https://basavapurushottam.com/learn-ai/pooling-as-a-statistic
