# Stride and Padding

Stride is the step between filter applications; padding adds border values so filters can cover the edges.

**Leads to:** Convolution Arithmetic, Image Segmentation

**Rests on nothing.** A place to start.

*Deep Learning · Convolutional Networks*


## Why learn this

Every time an image passes through a layer in a neural network, its width and height change. Stride and padding are the two controls that determine these new dimensions. With them, you can keep an image's size exactly the same as it moves deeper into the network, or shrink it deliberately to reduce the amount of data the computer must process. This control allows you to design a resolution pyramid: high detail at the start for spotting edges and textures, and broader context later for recognizing whole objects. If you do not understand these controls, you cannot predict how much information is left in your image after each step, nor can you build a model that fits your memory limits or accuracy goals.


## The intuition

Imagine you have a 5×5 grid of pixels — a small image. You want to examine it through a small window (the filter) that is 3×3. You can slide this window across the image. Two decisions control your scanning: how far you step each time (stride) and whether you can peek beyond the edges (padding).


## An analogy

Think of reading a book with a magnifying glass. The glass is a 3×3 window. You move it across a page line by line. If you move it one character at a time, you examine every character — but that takes many positions. If you move it two characters at a time, you skip every second character and finish faster, but you see less detail. That is stride.


## Definition

Stride is the number of steps the window moves forward after each calculation; padding is the border of extra values added around the input so the window can cover the edge pixels without shrinking the output size.


## How it connects

This concept sits inside Convolutional Networks. It connects directly to how layers calculate their output size, which uses stride and padding in its formula. It also relates to how much of the original image a later layer can 'see', a property called the receptive field; if you use a large stride, each step covers more area, changing what the network notices. Finally, it connects to pooling, which is another method for shrinking images by selecting representative values, whereas stride shrinks them by skipping positions entirely.


## The mathematics

To understand how a convolutional layer changes the size of an image, we need to look at two settings: **stride** and **padding**. These settings control where the filter (also called a kernel) looks at the input image.

First, let us define the variables we will use. We assume the input is a square grid of numbers with height H_in and width W_in. The filter is also a square grid, with side length K. For example, if the input is 5 pixels high and wide, H_in = 5 and W_in = 5. If the filter is 3 pixels by 3 pixels, then K = 3.

**Stride (S)**
Stride is the step size. When the filter has produced one output number, it moves to a new position on the input grid. Stride tells us how many pixels it jumps over each time. If S = 1, the filter moves one pixel at a time. If S = 2, it skips one pixel and lands on the next available spot. This skipping reduces the number of output positions, making the output smaller.

**Padding (P)**
Padding is the number of rows and columns of zeros added around the edge of the input image before the filter starts moving. If P = 0, no extra border is added. If P = 1, one layer of zeros surrounds the original image. Adding padding increases the effective size of the input, which allows the filter to take more steps or stay in the center longer.

**Calculating Output Size**
The output of a convolutional layer is also a grid of numbers. We need to find its height (H_out) and width (W_out). Because the rules for height and width are identical, we will calculate the height first.

The formula for the output height is:
H_out = floor((H_in + 2 * P - K) / S) + 1

Let us break down what this formula does, step by step.

1. **Add the padding**: We take the original input height H_in and add 2 times the padding P. We multiply P by 2 because we add zeros to both the top and the bottom of the image. This gives us the total effective height of the padded input.
   Effective Height = H_in + 2 * P

2. **Subtract the filter size**: From this effective height, we subtract the height of the filter K. Why? Because the filter needs space to sit on the input. If the effective height is exactly equal to the filter height, the filter fits perfectly in one position. If the effective height is larger, there is room for more positions.
   Remaining Space = Effective Height - K

3. **Divide by the stride**: We divide the remaining space by the stride S. This tells us how many full steps of size S fit into that remaining space.
   Steps = Remaining Space / S

4. **Round down (floor)**: The division might result in a fraction if the stride does not divide the remaining space evenly. Since we can only take whole steps, we round down to the nearest integer. This is what "floor" means.
   Whole Steps = floor(Steps)

5. **Add one**: Finally, we add 1 to the number of whole steps. Why? Because the very first position (where the filter starts at the top-left) counts as a valid output position, even if there are zero steps after it. So, the total number of positions is the number of steps plus the starting position.
   H_out = Whole Steps + 1

The width W_out is calculated using the same formula, but with W_in instead of H_in.

**Example Calculation**
Let us use real numbers to see how this works. Suppose:
- Input height H_in = 5
- Filter size K = 3
- Stride S = 2
- Padding P = 0

Step 1: Add padding. Since P = 0, we add 0.
Effective Height = 5 + 2 * 0 = 5

Step 2: Subtract filter size.
Remaining Space = 5 - 3 = 2

Step 3: Divide by stride.
Steps = 2 / 2 = 1.0

Step 4: Round down.
Whole Steps = floor(1.0) = 1

Step 5: Add one.
H_out = 1 + 1 = 2

So, the output height is 2. The width would be calculated similarly and would also be 2. The output grid is therefore 2 by 2.


## A worked example

We will compute the output size and then show one filter application.

**Setup**
- Input image: A 5x5 grid of numbers. Let us label the rows 0 to 4 and columns 0 to 4.
- Filter: A 3x3 grid. The values in the filter are:
  [[1, 0, -1],
   [1, 0, -1],
   [1, 0, -1]]
- Stride S = 2
- Padding P = 0 (no extra border)

**Step 1: Calculate Output Height**
Using the formula derived above:
H_out = floor((H_in + 2 * P - K) / S) + 1
Substitute the values: H_in = 5, P = 0, K = 3, S = 2.
H_out = floor((5 + 2*0 - 3) / 2) + 1
H_out = floor((5 - 3) / 2) + 1
H_out = floor(2 / 2) + 1
H_out = floor(1) + 1
H_out = 1 + 1 = 2

**Step 2: Calculate Output Width**
The width calculation is identical because the input and filter are square.
W_out = 2

The output map will be a 2x2 grid. We need to compute the value for each of the four positions in this output grid.

**Step 3: Determine Filter Positions**
The stride is 2. This means the top-left corner of the filter moves by 2 pixels at a time.
- Row positions: The filter starts at row 0. The next position is 0 + 2 = 2. The next would be 4, but a 3-high filter starting at row 4 would end at row 6, which is outside the 5-row input (rows 0-4). So, valid row starts are 0 and 2.
- Column positions: Similarly, valid column starts are 0 and 2.

This gives us four combinations:
1. Top-left: Row start 0, Col start 0
2. Top-right: Row start 0, Col start 2
3. Bottom-left: Row start 2, Col start 0
4. Bottom-right: Row start 2, Col start 2

**Step 4: Compute Output Value for Position (0,0)**
This corresponds to the top-left of the output grid.
The filter covers input rows 0-2 and columns 0-2.
Let us define the input values in this region. For clarity, let us assume the input image contains sequential numbers from 1 to 25, arranged row by row:
Row 0: [1, 2, 3, 4, 5]
Row 1: [6, 7, 8, 9, 10]
Row 2: [11, 12, 13, 14, 15]
Row 3: [16, 17, 18, 19, 20]
Row 4: [21, 22, 23, 24, 25]

The sub-grid at rows 0-2 and cols 0-2 is:
[[1, 2, 3],
 [6, 7, 8],
 [11, 12, 13]]

The filter is:
[[1, 0, -1],
 [1, 0, -1],
 [1, 0, -1]]

We multiply corresponding elements and sum them up (this is called an element-wise product followed by a sum).

Row 0 of filter: [1, 0, -1] * Input Row 0 [1, 2, 3]
Products: 1*1 + 0*2 + (-1)*3 = 1 + 0 - 3 = -2

Row 1 of filter: [1, 0, -1] * Input Row 1 [6, 7, 8]
Products: 1*6 + 0*7 + (-1)*8 = 6 + 0 - 8 = -2

Row 2 of filter: [1, 0, -1] * Input Row 2 [11, 12, 13]
Products: 1*11 + 0*12 + (-1)*13 = 11 + 0 - 13 = -2

Total Sum = (-2) + (-2) + (-2) = -6

So, the value at output position (0,0) is -6.

**Step 5: Compute Output Value for Position (0,1)**
This corresponds to the top-right of the output grid.
The filter covers input rows 0-2 and columns 2-4.
The sub-grid is:
[[3, 4, 5],
 [8, 9, 10],
 [13, 14, 15]]

Row 0: [1, 0, -1] * [3, 4, 5] = 3 + 0 - 5 = -2
Row 1: [1, 0, -1] * [8, 9, 10] = 8 + 0 - 10 = -2
Row 2: [1, 0, -1] * [13, 14, 15] = 13 + 0 - 15 = -2

Total Sum = (-2) + (-2) + (-2) = -6

So, the value at output position (0,1) is -6.

**Step 6: Compute Output Value for Position (1,0)**
This corresponds to the bottom-left of the output grid.
The filter covers input rows 2-4 and columns 0-2.
The sub-grid is:
[[11, 12, 13],
 [16, 17, 18],
 [21, 22, 23]]

Row 0: [1, 0, -1] * [11, 12, 13] = 11 + 0 - 13 = -2
Row 1: [1, 0, -1] * [16, 17, 18] = 16 + 0 - 18 = -2
Row 2: [1, 0, -1] * [21, 22, 23] = 21 + 0 - 23 = -2

Total Sum = (-2) + (-2) + (-2) = -6

So, the value at output position (1,0) is -6.

**Step 7: Compute Output Value for Position (1,1)**
This corresponds to the bottom-right of the output grid.
The filter covers input rows 2-4 and columns 2-4.
The sub-grid is:
[[13, 14, 15],
 [18, 19, 20],
 [23, 24, 25]]

Row 0: [1, 0, -1] * [13, 14, 15] = 13 + 0 - 15 = -2
Row 1: [1, 0, -1] * [18, 19, 20] = 18 + 0 - 20 = -2
Row 2: [1, 0, -1] * [23, 24, 25] = 23 + 0 - 25 = -2

Total Sum = (-2) + (-2) + (-2) = -6

So, the value at output position (1,1) is -6.

**Result**
The output map is a 2x2 grid with values:
[[-6, -6],
 [-6, -6]]

Note: In this specific example with a sequential input and a filter that detects vertical edges (difference between left and right), the result is uniform because the gradient is constant. The stride of 2 reduced the spatial size from 5x5 to 2x2.


## From scratch, in Python

```python
# Python building blocks: lists, nested lists, range, for loops

# A 5x5 input image (each inner list is a row, values are pixel intensities)
image = [
    [1, 2, 3, 4, 5],
    [6, 7, 8, 9, 1],
    [2, 3, 4, 5, 6],
    [7, 8, 9, 1, 2],
    [3, 4, 5, 6, 7]
]

# A 3x3 filter (edge detection: highlights vertical edges)
filter_ = [
    [1, 0, -1],
    [1, 0, -1],
    [1, 0, -1]
]

stride = 2
padding = 0

# Add padding: create a new list with zeros around the original image
if padding > 0:
    height = len(image)
    width = len(image[0])
    # New size: original + 2*padding on each side
    padded = [[0] * (width + 2*padding) for _ in range(height + 2*padding)]
    # Copy original into the middle
    for i in range(height):
        for j in range(width):
            padded[i + padding][j + padding] = image[i][j]
else:
    padded = image

# Compute output size using the formula
H_in = len(padded)
W_in = len(padded[0])
K = len(filter_)
H_out = (H_in - K) // stride + 1
W_out = (W_in - K) // stride + 1

# Initialize output as a list of zeros
output = [[0] * W_out for _ in range(H_out)]

# Slide the filter over the padded image
for i in range(H_out):
    for j in range(W_out):
        # Starting row and column for this filter position
        start_i = i * stride
        start_j = j * stride
        # Compute the sum of element-wise products
        total = 0
        for m in range(K):
            for n in range(K):
                total += padded[start_i + m][start_j + n] * filter_[m][n]
        output[i][j] = total

# Print the result
for row in output:
    print(row)
```


## The same thing, with a framework

```python
import numpy as np

image = np.array([...])  # same as above
filter_ = np.array([...])
stride = 2
padding = 0

# 'same' padding would use padding such that output size equals input size
# Here we use explicit calculation
out_h = (image.shape[0] - filter_.shape[0]) // stride + 1
out_w = (image.shape[1] - filter_.shape[1]) // stride + 1
output = np.zeros((out_h, out_w))

for i in range(out_h):
    for j in range(out_w):
        region = image[i*stride:i*stride+filter_.shape[0], j*stride:j*stride+filter_.shape[1]]
        output[i, j] = np.sum(region * filter_)

print(output)
# In PyTorch, you'd use torch.nn.Conv2d with stride and padding parameters.
```


## What the code does

This code builds a small convolution from scratch. First, we define the image and filter as lists of lists (nested lists). A list is an ordered collection of items; a list of lists is like a grid. We add padding if requested by creating a new list of zeros around the original. The output size is computed with the formula, and we create an output grid of that size. Then, for each position (i, j) in the output map, we slide the filter to the corresponding location (start_i, start_j) and multiply each overlapping pair, summing them into a total. The nested loops go through every output position and every filter element. Finally, we print the output rows.


## Common misconceptions

**Stride=2 means the filter is twice as big** — Stride=2 doubles the size of the filter or makes it cover twice the area.

Stride is the step between filter positions, not the filter size. The filter stays the same size; it just jumps further. In fact: Stride dictates how far the filter moves each time. Stride 1 moves one pixel, stride 2 moves two pixels, skipping some positions.

**Padding always increases the output size** — If I add padding, the output will always be bigger than the input.

Padding adds extra border pixels, but the output size depends on the formula. With certain stride and kernel combos, even with padding the output may be the same or smaller. In fact: Padding allows the filter to cover edges, but the output size is computed by the formula; 'same' padding is a specific amount of padding that preserves the input size exactly (when stride=1).

**Stride and pooling are the same thing** — A stride-2 convolution just does the same as a max-pooling layer.

Stride in convolution still applies the filter and computes a weighted sum at every step (even if steps are larger). Pooling computes a fixed statistic (max or average) over a region without learnable weights. In fact: Both reduce spatial size, but stride convolutions still learn features; pooling is a fixed operation. In modern networks, stride convolutions often replace pooling entirely.

**Padding with zeros does not change the output values** — Adding zeros around the input does not affect the result at all — it is just a trick to make the shape work.

Zeros interact with the filter and produce non-zero values at the edges, which propagate through the network. Edge responses are different from not having padding. In fact: Padding changes what the filter sees at the border, producing new activations that influence deeper layers. The zeros themselves are meaningful inputs.


## In short

Stride is how far the filter jumps; padding is how much border you add. The output size follows a simple arithmetic formula. Together they let you control the spatial resolution of a network — keeping it the same, shrinking it deliberately, or expanding it — and they are the first tool you have for building the resolution pyramid of a convolutional network.


## Where this is used

Stride and padding are not just academic parameters — they are the two knobs that control the shape of every convolutional network, from the small vision models on your phone to the huge backbones behind modern image generation.

In **large language models** (LLMs) themselves, convolution is mostly absent — transformers dominate. But when LLMs are paired with vision (multimodal models like LLaVA or GPT-4V), the image is often split into patches and processed by a convolutional stem. In that stem, stride is used to reduce the spatial resolution quickly: for example, a conv layer with stride 2 downsamples the feature map by half, cutting computation for the transformer that follows. Padding is typically set to 'same' so that each conv layer does not shrink the map unexpectedly, preserving the patch structure. A developer implementing a vision encoder writes code like `nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=2, padding=1)` — the `padding=1` keeps the output size half of the input exactly.

In **small models running on phones or edge devices** (TinyML), stride is a workhorse. These models have strict memory and battery budgets, so designers use stride-2 convolutions early on to shrink the feature map, reducing the number of floating-point operations. This is often preferred over adding a separate pooling layer — you get downsampling for free while convolving. Padding is usually 'same' at early layers to avoid losing edge information too fast. The practical effect: a model that would be too big to run on a device becomes real when stride cuts the computation by a factor of 4 at each step.

In **retrieval-augmented generation** (RAG), convolutional networks appear in the encoders that turn images or documents into embeddings. For example, a CLIP-style image encoder is a convolutional network where stride and padding determine the resolution of the final feature map. A stride of 16 or 32 at the last layers yields a small map (like 7×7 for a 224×224 image), which is then flattened into an embedding vector. That embedding is what gets compared against a database of pre-computed embeddings using a similarity measure (like dot product) to retrieve relevant passages or images. The developer does not usually adjust stride manually here — they just choose a pretrained backbone like ResNet, where the stride is fixed — but understanding stride helps when fine-tuning a model for higher-resolution inputs (adjusting stride and padding is the standard trick to make a model accept larger images without breaking).

In **agentic systems** that plan and call tools, computer vision is a key component — for example, an agent that reads a screen or a chart. Vision encoders in these agents use convolutional layers with specific stride and padding to produce feature maps that are then passed to a policy. The agent's ability to understand spatially detailed information depends on how much the network downsampled. If stride is too aggressive, small text in a screenshot becomes unreadable in the feature map; if too small, the model is slow and may overfit. Agent builders often use 'same' padding for early layers and stride-2 downsampling later, balancing detail and speed.

In **everyday applied machine learning** — recommendation, forecasting, fraud detection, speech — convolution is used on 1D signals (time series, sequences). Stride controls how much the filter jumps in time; padding controls how the edges of the sequence are handled. For example, in a speech recognition front-end, a convolution with stride 2 halves the time resolution, making downstream modelling faster. In fraud detection on transaction sequences, padding allows the model to see the beginning of the sequence fully. A developer using Keras might write `Conv1D(filters=32, kernel_size=3, strides=2, padding='same')` — the strides parameter is literally the stride. Getting this wrong produces a model that cannot consume the input tensor because the output length does not match the next layer's expectation, a very common debugging error.

**What breaks when done badly:** The most frequent mistake is miscomputing the output size, leading to a shape mismatch that crashes the training run. Another is using valid padding exclusively and shrinking the map so fast that edge features (often important) are lost. Conversely, too much padding can make the model insensitive to edges because the zero-border washes out boundary information. A developer must know the formula by heart to design a network that exactly meets the target output size.


---

Read this on the web: https://basavapurushottam.com/learn-ai/stride-and-padding
