# Keras and TensorFlow

A high-level model-building API over a production-oriented deep learning runtime, emphasising concise layer composition and a managed training loop.

**Rests on:** Computational Graphs, Tensors and Shapes

*Deep Learning · Deep Learning*


## Why learn this

Keras and TensorFlow are the two tools you will use to build and run nearly every neural network you encounter. This page is the gateway: once you can stack components in Keras and start a training run with one line, you can build image classifiers, text generators, chatbots, and recommendation systems. What this unlocks: the ability to take any architectural idea (a layer that processes images, an attention mechanism, a transformer) and turn it into working code that trains on your laptop or a cloud GPU. It also lets you read other people's models — every pretrained model you download, from BERT to GPT, is a Keras or TensorFlow model under the hood. Learning this concept is learning the keyboard of the piano; the music comes later, but without it you cannot play a note.


## The intuition

Think of building a neural network like assembling a meal from a recipe. You have a list of ingredients (layers: Dense, Conv2D, LSTM) and a sequence of steps (add this layer, then add that one, then compile). Keras is the recipe book — it gives you short, clear instructions that hide the tedious details of measuring and mixing. TensorFlow is the kitchen itself — it provides the stove, the oven, the pots, and the fuel (the GPU) that actually cooks the food. You, the cook, follow the recipe; the kitchen does the heating. When a recipe says 'preheat oven to 200°C', you do not build the oven — you just set the dial. Similarly, when Keras tells you to start the training process, you do not write the thousands of lines of math that adjust every weight; TensorFlow has already done that for you. The key insight: Keras is a *language* for describing models; TensorFlow is the *engine* that executes them. You write in Keras because it is easy; TensorFlow runs because it is fast. The separation matters: you can describe the same model in plain Python (slow, teaching) or in Keras (fast, production). The magic is that both describe the same thing.


## An analogy

Imagine you are a project manager at a construction company. Your job is to build a house. You do not personally lay bricks or pour concrete — you write a plan: 'Foundation, then Walls, then Roof.' You hand this plan to a construction crew. The crew has all the heavy machinery — cranes, mixers, hammers — and they know how to execute each step. You do not care *how* they lay each brick; you care that they follow the plan and deliver a house on time. Keras is your plan-writing tool — it lets you say, in a few lines, 'Dense layer, then ReLU, then Dropout, then a final Dense layer.' TensorFlow is the construction crew — it has the machinery (GPU kernels, backpropagation engines, memory management) to turn your plan into a trained network. The analogy breaks down in an important way: the construction crew does not just follow your plan; they also *learn* from it. After you give them the plan, they try, check the result, adjust the materials, and try again — millions of times — until the house is perfect. That learning loop (fit, evaluate, update) is also part of TensorFlow, and Keras hides it behind a single `fit()` call.


## Definition

Keras is a high-level, Python-based API for defining and training neural networks using a concise, layer-by-layer syntax, built on top of TensorFlow, which provides the low-level runtime, automatic differentiation, and GPU acceleration that actually executes the training.


## How it connects

You have not yet studied neurons, layers, or backpropagation, so think of this as the hammer and nails you will use once you learn carpentry. This page stands alone: you can understand Keras and TensorFlow as tools even before you know what a neuron is, just as you can understand what a hammer is without knowing the physics of a nail. Later, when you learn about activation functions (see your notes on that topic), you will see how Keras lets you add a `relu` or `sigmoid` activation in one word. When you learn about convolutional networks (notes in your library), Keras gives you `Conv2D` as a ready-made layer. So this concept is the bridge between ideas and implementation. It also links to your notes on 'Deep Learning' — the parent concept — because Keras and TensorFlow are the standard way that deep learning is turned from theory into practice.


## The mathematics

This concept is not about inventing new mathematics. It is about using software to perform arithmetic operations that you already know, but doing them millions of times at once.

The only mathematics involved here is multiplication and addition. In school, you learned to multiply two numbers and add the results together. A neural network does exactly this, but on a much larger scale.

Think of a single calculation in a neural network as a recipe for one number. The recipe has three parts:
1. Inputs: These are the numbers you start with. Let us call them x1 and x2.
2. Weights: These are special numbers that tell the system how important each input is. Let us call them w1 and w2.
3. Bias: This is a single number added at the end to shift the result. Let us call it b.

The calculation follows this order:
First, multiply the first input by the first weight: x1 times w1.
Second, multiply the second input by the second weight: x2 times w2.
Third, add those two products together.
Fourth, add the bias to that sum.

We can write this as a single line of arithmetic:
z = (x1 multiplied by w1) plus (x2 multiplied by w2) plus b.

The result, z, is called the 'raw score'. It is just a number. Often, we do one more step. We might pass this raw score through an 'activation function'. The most common one is called ReLU. ReLU means: if the number is positive, keep it as it is. If the number is negative, change it to zero.

A 'layer' in Keras or TensorFlow is just a group of these calculations happening together for many numbers at once. When you write code like `model.add(Dense(10))`, you are not writing new math. You are telling the computer: 'Take my inputs, multiply them by weights, add biases, and apply ReLU.' The software handles the millions of multiplications and additions automatically.

The mathematics is simple arithmetic. The power comes from doing it billions of times per second and adjusting the weights (w1, w2, etc.) to make the final answer more accurate.


## A worked example

Let us walk through the arithmetic for one single neuron. This is the exact math that happens inside a layer when you give it data.

Suppose we have two inputs:
x1 = 2
x2 = 3

We have two weights (the importance of each input):
w1 = 0.5
w2 = -1

We have one bias:
b = 0.1

Step 1: Multiply the first input by its weight.
2 multiplied by 0.5 equals 1.0.
This means the first input contributes a positive value of 1.0 to the total.

Step 2: Multiply the second input by its weight.
3 multiplied by -1 equals -3.0.
This means the second input contributes a negative value of -3.0 to the total.

Step 3: Add the two products together.
1.0 plus (-3.0) equals -2.0.

Step 4: Add the bias to that sum.
-2.0 plus 0.1 equals -1.9.
This number, -1.9, is our raw score, z.

Step 5: Apply the activation function (ReLU).
ReLU says: look at the raw score. If it is greater than zero, keep it. If it is less than or equal to zero, make it zero.
Since -1.9 is less than zero, we change it to 0.

Final Result: The output of this neuron is 0.

In a real neural network, you would have thousands of these neurons working in parallel. TensorFlow and Keras perform these exact steps for every single neuron, millions of times, without you having to write the multiplication or addition symbols yourself.


## From scratch, in Python

```python
# Plain Python: a tiny neural network with one neuron, no libraries.
# Building blocks you need:
# - A list: an ordered collection of values, e.g., inputs = [2, 3]
# - A for loop: repeats a block of code for each item in a list.
# - A function: a reusable block of code with a name.

inputs = [2, 3]           # two numbers
weights = [0.5, -1.0]     # two weights, one per input
bias = 0.1                # one bias

def neuron(inputs, weights, bias):
    raw_score = 0.0
    for i in range(len(inputs)):
        raw_score = raw_score + (inputs[i] * weights[i])   # multiply and add
    raw_score = raw_score + bias
    activation = max(0.0, raw_score)   # ReLU: keep positive, zero if negative
    return activation

print(neuron(inputs, weights, bias))   # prints 0.0

# Explanation:
# - range(len(inputs)) gives the indices 0, 1, ... up to the length-1.
# - inputs[i] gets the i-th element (e.g., inputs[0] = 2).
# - The loop goes twice: first i=0, then i=1.
# - After the loop, raw_score = 2*0.5 + 3*(-1) = 1 - 3 = -2, plus bias 0.1 = -1.9.
# - ReLU: max(0, -1.9) = 0.0.
```


## The same thing, with a framework

```python
# Now, the same thing in Keras. This is the library version.
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense

model = Sequential()
model.add(Dense(1, input_shape=(2,), activation='relu'))
model.compile(optimizer='sgd', loss='mse')

# Under the hood, this does exactly the same math as the plain Python above:
# a Dense layer with 1 neuron holds a list of 2 weights and 1 bias.
# When you call model.predict([[2, 3]]), it multiplies, adds, and applies
# ReLU, returning the same result. The difference: Keras also sets up
# automatic differentiation and a training loop so you do not write
# the gradient descent by hand.
```


## What the code does

The plain Python code shows the raw arithmetic: a loop that multiplies each input by its weight, adds a bias, and squashes with ReLU. You can run it and see the output. The Keras version replaces that loop with a single line: `Sequential()` then `add(Dense(...))`. The framework does the same arithmetic but also gives you `.compile()` and `.fit()` for training. The walkthrough: `Sequential` means layers stacked in order; `Dense` means every input connects to every neuron (in this case, one neuron). `input_shape=(2,)` tells Keras each sample has 2 numbers. `activation='relu'` applies the same `max(0, x)` function. `compile` prepares the model for training: it chooses the optimisation method (`sgd` — stochastic gradient descent) and the loss function (`mse` — mean squared error). The beauty: you do not write any of the multiplication or loop; you just describe the architecture.


## Common misconceptions

**Keras is a different thing from TensorFlow** — Keras and TensorFlow are two separate tools that you pick between. I should choose one.

Keras is a high-level API that *runs on* TensorFlow. You cannot use Keras without TensorFlow (in practice, though there are other backends, the standard is TensorFlow). They are not alternatives; they are layers — Keras is the user-friendly surface, TensorFlow is the engine underneath. In fact: Keras is the interface you write your model in; TensorFlow is the runtime that executes it. They work together: every Keras call secretly invokes TensorFlow operations.

**TensorFlow is just a Python library** — TensorFlow is like NumPy — a Python package I import and use, and it runs all in Python.

TensorFlow is a C++-based runtime with a Python front-end. The heavy computation (the multiplications) happens in compiled kernels that run on GPUs, not in Python. That is why it can handle billions of operations quickly. In fact: TensorFlow is a high-performance engine with a Python API. The Python code you write describes a computational graph; TensorFlow executes it efficiently, often on GPUs.

**Keras is only for beginners** — Once I am a serious researcher, I will stop using Keras and write raw TensorFlow instead.

Many production models, including some cutting-edge research, are built with Keras. It is the standard in industry because it reduces boilerplate, and the flexibility (via the functional API) supports complex models like transformers. In fact: Keras is not a toy. It is used in industry and academia for a wide range of models. Its concise syntax is an advantage at every level, not a crutch.

**The 'fit()' method does nothing important** — I can just write the forward pass myself and loop over the data manually; fit() is just a convenience.

fit() orchestrates the entire training loop: it batches data, runs the forward pass, computes the loss, calculates gradients via backpropagation, updates weights using an optimizer, and tracks metrics. Recreating that manually is hundreds of lines of error-prone code. In fact: fit() is a powerful, battle-tested training loop. It handles all the steps, and it also does them efficiently on GPUs, with progress bars and built-in validation.

**I need to learn TensorFlow before Keras** — Keras is the easy version, but to understand how it works, I should first learn the low-level TensorFlow API (variables, sessions, graphs).

Modern Keras is the recommended way to use TensorFlow. Learning low-level TensorFlow is not a prerequisite; it can be confusing and is rarely needed unless you are writing custom layers or research code. In fact: Start with Keras. The underlying engine is there to make your life easier, not to be a hurdle. You can always delve deeper later if needed.


## In short

Keras is the recipe book; TensorFlow is the kitchen. You describe your neural network in Keras with a few lines (Sequential, add layers, compile, fit). TensorFlow executes the training on fast hardware. The math underneath is simple multiplication and addition, repeated billions of times. Remember the three-step incantation: define the model, compile it (choose optimizer and loss), then fit it to your data. That is how nearly every deep learning project starts.


## Where this is used

**In large language models:** Keras is how many researchers prototype new models before scaling them. Even though production LLMs are trained with custom frameworks, the *idea* of composing layers — embedding, attention, feed-forward — is exactly what Keras does for smaller models. You can build a mini-GPT with Keras in a few dozen lines. **On small devices (TinyML):** TensorFlow Lite is a variant that converts trained Keras models into compact files that run on phones and microcontrollers. A developer trains a model in Keras, then exports it to TensorFlow Lite, which optimises it for limited memory. **In retrieval-augmented generation (RAG):** RAG itself does not use Keras for the retrieval part, but the embedding model that converts text into vectors is almost certainly a Keras/TensorFlow model. The developer loads a pretrained embedding model (e.g., a universal sentence encoder) via TensorFlow Hub, uses it to embed documents and queries, and then uses cosine similarity (a simple dot product) to find relevant text. **In agentic systems:** Agents that plan and call tools are typically built on top of LLMs, and the LLM itself is served by TensorFlow Serving (the production serving engine). Developers write the agent loop in Python, but the neural network that powers the language model is a TensorFlow model served through an API. **In everyday applied ML:** Image classifiers for defect detection in factories, recommender systems in e-commerce, fraud detection in banks, speech-to-text in voice assistants — all of these are built with Keras. A developer writes a `Sequential` model with a few `Conv2D` and `Dense` layers, calls `fit()` on a dataset, and deploys with TensorFlow Serving or TensorFlow Lite. **What breaks when done badly:** If a developer ignores the `input_shape`, the model may not compile. If they use too many layers without proper activation, gradients vanish (see your notes on 'Activation Functions' and 'Gradient flow'). If they forget `compile` or use the wrong loss for the task, training will not converge. And if they try to deploy a 300 MB model to a phone, it will be too slow — hence TensorFlow Lite's pruning and quantization.


---

Read this on the web: https://basavapurushottam.com/learn-ai/keras-and-tensorflow
