# CNN Visualization for Interpretation

Inspecting what a convolutional network has learned by visualising filters, activation maps and the input regions driving a prediction.

**Rests on:** Convolutional Layers, Forward Propagation

*Deep Learning · Convolutional Networks*


## Why learn this

You are about to learn how to look inside a convolutional network and see what it has actually learned. This skill turns a mysterious algorithm into something you can examine. It matters because knowing why a system made a decision is essential for fixing errors, trusting it in high-stakes settings like medical imaging or fraud detection, and explaining its results to people who do not build the system themselves. After this topic, you will be able to identify which parts of an input image most influenced a prediction, something that is directly useful in computer vision and multimodal AI.


## The intuition

A convolutional network learns by looking at images through a series of filters. Each filter is a small pattern, like a tiny stamp, that slides across the image and highlights where that pattern appears. In the early layers of the network, these filters learn simple patterns: edges, colours, or blobs of light and dark. In later layers, the network combines those simple patterns into more complex ones, like a corner, a wheel, or eventually an entire object.

When you want to interpret what the network is doing, you have three main tools. First, you can look at the filters themselves: each one is just a small table of numbers, and you can display those numbers as an image to see what pattern the filter is looking for. Second, you can look at the activation maps: after a filter slides over the image, the network produces a new image (smaller or same size) showing how strongly the filter matched at each location. Imagine this as a heatmap where bright spots indicate strong matches and dark spots indicate weak matches. Third, you can look at what region of the original image made the network decide 'dog' vs 'cat' by using techniques like class activation maps. Class activation maps are heatmaps that show which parts of the input image were most important for a specific prediction. For example, if the network predicts 'dog', the heatmap might highlight the dog's ears and nose.

These three views give you a window into the network's thinking at different levels of abstraction.


## An analogy

Imagine you are a detective investigating how a team of art critics judges paintings. Each critic has a specialty: one only looks for straight lines, another for the colour red, another for curves. When a critic sees a painting, they walk across it, looking for their specialty. After they finish, they give you a map of the painting with marks where they found their pattern. You, the detective, want to know why a particular painting was judged 'modern art'. So you ask each critic for their map. The lines-critic shows you a map full of marks on the building edges, the red-critic shows marks on the red circle, the curves-critic shows marks on the spiral. You can also ask directly what each critic's specialty is, by asking them to draw a generic example of what they look for. And finally, you can ask, 'Which part of this painting most convinced you it was modern?' They would point to the spiral, saying that is the defining feature. Now, in a convolutional network, the critics are filters, their maps are activation maps, and the pointing finger is the class activation map. This analogy works well, but it breaks down in one way: a human critic can tell you their specialty in words ('I look for curves'), but a neural network filter is just numbers—it cannot explain itself. The numbers are all we have, and we must translate them into something visual to understand them.


## Definition

CNN visualization for interpretation is the process of inspecting the learned filters, activation maps, and input regions that most strongly drive a convolutional network's prediction, in order to understand what features the network has learned and why it made a particular decision.


## How it connects

You are studying this as part of Convolutional Networks. Your library notes mention that this is one of the more successful interpretability approaches, and it connects directly to your notes on Convolutional Layers (where filters live), Convolution Arithmetic (how filter sizes affect what you see), Receptive Fields (the specific region of the input image that influences a single output value in a layer, explaining why later layers see more context), and Pooling as a Statistic (how pooling affects the resolution of activation maps).


## The mathematics

To understand how a Convolutional Neural Network sees an image, we need to look at the math of a single step: applying one filter to one small part of the picture. This is not magic; it is simple arithmetic you already know: multiplication and addition.

Imagine a filter as a tiny grid of numbers. Let us call this grid F. Imagine a small square patch of the image that the filter is currently sitting on. Let us call the pixel values in that patch X.

The first thing to understand is what the filter does. It compares itself to the image patch. It asks: "Do the bright and dark spots in my pattern match the bright and dark spots in this part of the image?"

To answer this, it performs a dot product. Here is how that works, step by step.

Step 1: Match each number in the filter with the corresponding pixel in the image patch. If the filter has four numbers, and the image patch has four pixels, you line them up one-to-one.

Step 2: Multiply each pair. Take the first number of the filter and multiply it by the first pixel value. Take the second number of the filter and multiply it by the second pixel value. Do this for every position in the grid.

Step 3: Add up all those products. The final sum is a single number. This number is called the activation at that location.

What does this number mean?
- If the result is a large positive number, it means the filter and the image patch agree. The pattern in the filter was found in the image.
- If the result is a negative number, it means they are opposites. Where the filter expected light, there was dark.
- If the result is near zero, it means there was no clear match.

We create an "activation map" by sliding this filter across the entire image, one step at a time. At every new position, we repeat the multiplication and addition. We write down the result in a new grid. This grid shows us where the pattern exists in the original image.

Now, let us look at how we decide which part of the image matters for a specific class, like "cat." We use a weighted sum. Imagine we have many different activation maps from earlier layers. Each map highlights a different feature (edges, textures, shapes). To find out if the image contains a cat, we combine these maps.

We assign a weight to each map. A positive weight means that feature helps identify a cat. A negative weight means it does not. We multiply each activation map by its weight and add them all together. The result is a Class Activation Map. Bright spots in this final map show exactly which pixels the network looked at to decide "this is a cat."

The mathematics here is strictly linear algebra: multiplying numbers and adding them up. There are no complex functions or hidden variables. It is just measuring similarity through multiplication and summation.


## A worked example

Let us calculate an activation map for a tiny 3x3 image using a 2x2 filter. We will slide the filter across the image without any gaps (this is called a 'valid' convolution). This means the output grid will be smaller than the input.

The Image (3x3 pixels):
Row 1: 1, 2, 1
Row 2: 0, 1, 2
Row 3: 3, 1, 0

The Filter (2x2 weights):
Top-left: 1
Top-right: 0
Bottom-left: 0
Bottom-right: -1

This filter is looking for a specific pattern: a bright pixel at the top-left and a dark pixel at the bottom-right. The '1' in the filter amplifies brightness; the '-1' amplifies darkness.

We will compute the activation for each of the four possible positions where the 2x2 filter fits inside the 3x3 image.

Position 1: Top-left corner of the image.
The patch under the filter is:
[[1, 2],
 [0, 1]]

Calculation:
Multiply corresponding numbers:
(1 * 1) = 1
(2 * 0) = 0
(0 * 0) = 0
(1 * -1) = -1

Add the results:
1 + 0 + 0 + (-1) = 0

The activation at Position 1 is 0. The pattern did not match.

Position 2: Move the filter one step to the right.
The patch under the filter is:
[[2, 1],
 [1, 2]]

Calculation:
Multiply corresponding numbers:
(2 * 1) = 2
(1 * 0) = 0
(1 * 0) = 0
(2 * -1) = -2

Add the results:
2 + 0 + 0 + (-2) = 0

The activation at Position 2 is 0. No match.

Position 3: Move the filter down one step (back to the left column).
The patch under the filter is:
[[0, 1],
 [3, 1]]

Calculation:
Multiply corresponding numbers:
(0 * 1) = 0
(1 * 0) = 0
(3 * 0) = 0
(1 * -1) = -1

Add the results:
0 + 0 + 0 + (-1) = -1

The activation at Position 3 is -1. This indicates an inverse match (the pattern was opposite to what the filter wanted).

Position 4: Move the filter one step to the right again.
The patch under the filter is:
[[1, 2],
 [1, 0]]

Calculation:
Multiply corresponding numbers:
(1 * 1) = 1
(2 * 0) = 0
(1 * 0) = 0
(0 * -1) = 0

Add the results:
1 + 0 + 0 + 0 = 1

The activation at Position 4 is 1. This is a positive match. The filter found its pattern here: a bright pixel (1) at the top-left of the patch and a dark pixel (0) at the bottom-right.

Final Activation Map (2x2 grid):
[[0, 0],
 [-1, 1]]

Interpretation:
The positive value '1' in the bottom-right corner tells us that the network detected the specific pattern it was designed to find in that location of the original image.


## From scratch, in Python

```python
# Let's visualize filters, activation maps, and a class activation map from scratch.
# We'll use plain Python lists (no libraries) to see the arithmetic.

# A tiny 5x5 grayscale image (values 0-255, but we'll use 0-3)
image = [
    [1, 0, 2, 1, 0],
    [0, 1, 1, 2, 2],
    [3, 1, 0, 1, 1],
    [1, 2, 2, 0, 0],
    [0, 0, 1, 3, 1]
]

# A simple 3x3 filter that detects edges (a known filter)
# This filter responds strongly where there is a bright-to-dark transition.
filter_weights = [
    [ 1, 0, -1],
    [ 1, 0, -1],
    [ 1, 0, -1]
]

# Step 1: Compute the activation map by sliding the filter over the image
# We'll use a stride of 1 and no padding, so output size is (5-3+1) = 3

def apply_filter(image, filter_weights):
    image_h = len(image)
    image_w = len(image[0])
    filter_h = len(filter_weights)
    filter_w = len(filter_weights[0])
    output_h = image_h - filter_h + 1
    output_w = image_w - filter_w + 1
    # Create an empty 2D list (list of lists) to hold the output
    activation_map = [[0]*output_w for _ in range(output_h)]
    # Loop over every position where the filter fits
    for i in range(output_h):
        for j in range(output_w):
            # Dot product: sum of element-wise multiplications
            total = 0
            for fi in range(filter_h):
                for fj in range(filter_w):
                    total = total + filter_weights[fi][fj] * image[i+fi][j+fj]
            activation_map[i][j] = total
    return activation_map

activation_map = apply_filter(image, filter_weights)
print('Activation map (filter response at each location):')
for row in activation_map:
    print(row)

# Step 2: Simulate a network with one convolutional layer and a final classification layer
# For interpretation, we pretend we have one feature map (the activation_map above)
# and we have weights that indicate how much this feature map contributes to each class.
# Here we have two classes: 'cat' and 'dog'.
# The weight for 'cat' is 0.8, for 'dog' is -0.5.
class_weights = {'cat': 0.8, 'dog': -0.5}

# Class activation map: multiply the activation map by the class weight
# This gives a weighted map, but a real CAM also involves upsampling and global pooling.
# We'll just compute the weighted activation map for each class.
def class_activation_map(activation_map, class_weight):
    # Multiply every element by the weight
    cam = [[cell * class_weight for cell in row] for row in activation_map]
    return cam

for class_name, weight in class_weights.items():
    cam = class_activation_map(activation_map, weight)
    print(f'Weighted map for {class_name}:')
    for row in cam:
        print(row)

# Step 3: Identify the most important region for a prediction
# In a real network, we'd upsample the CAM to the input size and find the area with max values.
# Here we simply find the position of the largest value in the activation map.
max_val = activation_map[0][0]
max_pos = (0,0)
for i in range(len(activation_map)):
    for j in range(len(activation_map[0])):
        if activation_map[i][j] > max_val:
            max_val = activation_map[i][j]
            max_pos = (i,j)
print(f'Region that most activates this filter: row {max_pos[0]}, column {max_pos[1]}')

```


## The same thing, with a framework

In practice, you'd use PyTorch to extract filters and activations. Here's how you'd get a filter's weights and a forward pass to see activations:

```python
import torch
import torch.nn as nn

# Assume you have a trained model
model = nn.Sequential(
    nn.Conv2d(in_channels=3, out_channels=4, kernel_size=3),
    nn.ReLU(),
    nn.Flatten(),
    nn.Linear(4*30*30, 2)  # just a placeholder
)

# Extract the filter weights from the first conv layer
conv_layer = model[0]
filters = conv_layer.weight.data  # shape: [out_channels, in_channels, k, k]
print('Filter 0:', filters[0])

# Forward pass with a dummy image
image = torch.randn(1, 3, 32, 32)  # batch of 1, 3 channels, 32x32
with torch.no_grad():
    activation = model[0](image)  # after conv only, before ReLU
print('Activation map shape:', activation.shape)  # [1, 4, 30, 30]

# For class activation maps, you'd use the last conv layer and the weights that connect it to the class.
# Libraries like torchcam (https://github.com/frgfm/torch-cam) implement this for you.
```


## What the code does

The from-scratch code starts with a tiny image and a filter. The `apply_filter` function does the dot product sliding. It loops over every position where the filter fits, and for each position, it loops over the filter's rows and columns, multiplying corresponding numbers and summing them. This is exactly what a convolutional layer does. Then we create a fake class weight and compute a weighted activation map, which is the idea behind CAM. Finally, we find the maximum value in the activation map to locate the most important region. The framework code shows how to access trained weights: after training, the model stores its filters in `conv_layer.weight`. To see activations, you run the input through just the conv layer. Libraries like torchcam automate CAM generation, but underneath it's the same multiplication and sum.


## Common misconceptions

**Activation maps are the same as filters** — I thought the activation map is the filter itself, but they're different.

The filter is the learned pattern (a small table of numbers); the activation map is the output after sliding that filter over the image, showing where the pattern occurs. In fact: The filter is the pattern; the activation map is the response over the image.

**Interpretability shows exactly what the network is thinking** — If I visualize the CAM, I can know exactly why the network made the prediction, like reading its mind.

Networks are complex; CAM shows a heatmap of important regions, but it does not capture all the intricate interactions between layers. In fact: Visualization gives a plausible explanation, a strong hint, but not a complete account of the network's decision process.

**Early layers learn edges only** — Early layers only ever detect straight edges and nothing else.

Early layers also detect colours, blobs, and other simple patterns; they are not limited to edges. In fact: Early layers learn a variety of simple low-level features, of which edges are one example.

**Class activation maps are enough for any model** — CAM works on any model directly without changes.

CAM requires a global average pooling layer before the final classification; many modern models use other architectures, requiring adaptation (e.g., Grad-CAM). In fact: Basic CAM needs a GAP layer; for other architectures, variants like Grad-CAM provide a general approach.

**Visualizing filters is the only way to interpret a CNN** — The only method for interpreting CNNs is to look at filters.

There are many methods: saliency, activation maximization, perturbation methods, and CAM. In fact: Filter visualization is a starting point, but many complementary tools exist.


## In short

Remember: CNNs learn filters (small pattern detectors). To interpret a network, you look at (1) the filters themselves to see what patterns they are looking for, (2) the activation maps to see where those patterns appear in an image, and (3) the class activation map (CAM) to see which parts of the image most influenced a specific prediction. Early filters detect simple things like edges, later filters detect complex composites. This is one of the most successful interpretability methods and is used for debugging, explaining, and improving trust in models.


## Where this is used

Where is CNN visualization used in practice? It is used in model debugging: when a network misclassifies an image, looking at the class activation map tells you whether it was looking at the right region. For example, if a medical model predicts 'pneumonia' but the CAM highlights the background, you know it is learning spurious cues. In model development, visualizations help decide whether to add more layers or change the convolution arithmetic. In large language models (LLMs), visualization is less directly applicable because they are not convolutional, but the idea of inspecting what drives a prediction is analogous to attention maps in transformers. However, some multimodal models that process images use CNNs for vision backbones, and visualizing those CNN features helps understand what part of an image the model is focusing on. In small models running on devices (TinyML), visualization is used to compress models: if you see that many filters are redundant, you can prune them, reducing size and latency. In retrieval-augmented generation (RAG), the relevance of a retrieved image might be judged by a CNN, and visualizing the CAM tells you why that image was considered relevant, helping debug retrieval failures. In agentic systems that plan and call tools, a vision component that inspects a screenshot or a chart may use a CNN, and interpreting its output can help the agent decide which action to take next. In everyday applied work, such as fraud detection, if the input is a time-series reshaped into an image, CNN visualization can highlight which time window drove a fraud decision. In practice, a developer reaches for PyTorch to load a pretrained model, extracts feature maps from the last convolutional layer, and uses libraries like torchcam or captum to produce saliency maps. They write code like `cam = get_cam(model, input_tensor, target_class)` and then overlay the heatmap on the original image. The common issue is that a naive implementation produces misleading maps if the model has Global Average Pooling or if the activation maps are low-resolution; developers must upsample them properly.


---

Read this on the web: https://basavapurushottam.com/learn-ai/cnn-visualization-for-interpretation
