# Edge and Fog Computing

Moving computation toward where data is produced — on devices or at intermediate nodes — instead of sending everything to a central cloud.

**Rests on:** Data Compression

**Leads to:** Code Generation for Deployment

*MLOps and Platform · Cloud Computing*


## Why learn this

Mastering edge and fog computing is the key to understanding how modern AI systems actually run in the world. When a smart speaker answers instantly, a self-driving car brakes without waiting for a signal, or a factory robot adjusts its arm in milliseconds, that is edge computing in action. Learning this unlocks the next step in your AI mastery path: designing and troubleshooting systems where intelligence must live near the data, not in a distant cloud. You will be able to reason about why some AI tasks cannot afford to wait a round-trip to a server, how to balance speed, privacy, and cost, and what it takes to manage thousands of devices running models on their own. This is the foundation for advanced topics like TinyML, distributed training, and real-time inference at scale.


## The intuition

Think of the internet as a highway. The cloud is a giant warehouse far away, where all the heavy work is done. If a sensor on a factory floor wants to know if a machine is about to fail, sending the data all the way to the warehouse takes time — the data travels over miles of cables, hops through many routers, and then the answer has to come back. That round trip might take hundreds of milliseconds — an eternity for a machine that needs to stop instantly. Edge computing says: do the computation right next to the sensor, on a small computer attached to the machine. The sensor sends data to that little computer, which runs a model and gives an immediate verdict. Fog computing is a middle step: instead of each tiny device doing everything, there is a slightly bigger computer in the local network (like a gateway in the office or factory) that gathers data from many sensors, does some processing, and only sends summaries to the cloud. The key insight is that cloud, edge, and fog are not competitors — they are steps on a continuum from the data source to the central cloud. You decide how far up the ladder computation moves based on how fast you need results, how much bandwidth you have, and how sensitive the data is.


## An analogy

Imagine you are cooking a big family meal in your kitchen. The cloud is a legendary chef in a restaurant across town. You could ask that chef to chop all your vegetables, but you would have to box them up, drive across town, wait, and drive back. That takes too long if you need dinner in twenty minutes. So, you chop the vegetables yourself — that is edge computing. You are doing the work right where the ingredients are. Fog computing is like asking a neighbour who lives close by to help with the cooking. The neighbour is not as skilled as the famous chef, but they are much closer, so you can send them a dish to taste, they adjust the seasoning, and bring it back quickly. You still call the chef for the tricky sauce recipe — that is the cloud, used for things that need deep expertise and where speed is less critical. Where does the analogy stop working? In cooking, you can always talk to the chef quickly if you need to; communication is nearly instant. In computing, the network to the cloud is not instant — it can be slow or unreliable, and that is the whole reason we move work to the edge. Also, the famous chef is free to use; in computing, the cloud costs money per hour, while edge devices are already bought and running. So the analogy captures the core idea of moving work closer to the source for speed and convenience, but breaks down on the costs and communication constraints that drive real engineering decisions.


## Definition

Edge and fog computing is the practice of moving computation toward where data is produced — onto devices themselves or onto intermediate nodes in the local network — rather than sending everything to a central cloud, in order to reduce latency, save bandwidth, and protect privacy.


## How it connects

This concept builds on your understanding of cloud computing, where you learned that the cloud provides on-demand compute, storage, and managed services. Edge and fog is not a replacement but a complement: you still have the cloud for heavy training, model updates, and storing historical data. Now, instead of every device talking to the cloud, you push some intelligence down to the edge. This is like moving from a single central market (cloud) to having local stalls (edge) that can serve customers faster. In your AI mastery path, this is the first systems topic that directly affects how AI models are deployed — you will see this again when you study model optimisation, TinyML, and real-time inference. The same three drivers — latency, bandwidth, and privacy — will recur in every deployment decision you make.


## The mathematics

This concept does not require heavy mathematics; it is mostly a systems and networking topic. The only mathematics involved is simple arithmetic to understand the trade-offs. We will think about latency — how long it takes for a message to travel from a device to a server and back. We will compute round-trip time, average latency across multiple requests, and the percentage of data you can keep off the network when you process it at the edge. These are all basic ratio and average calculations you can do with a calculator. For example, if a round trip to the cloud takes 200 milliseconds and a local computation takes 5 milliseconds, you can compute that it is 40 times faster to do it locally. The mathematics here is about comparing numbers to make design decisions, not about algebra or calculus.

To compare these two approaches fairly, we need to define exactly what time we are measuring for each one. We will use letters to stand for specific time intervals. This allows us to write down the rules once and then plug in any numbers later.

First, let us look at the Cloud approach. When a device sends data to a cloud server, three things happen in sequence:
1. The data travels from the device to the server. We call this time d_send. It is measured in milliseconds (ms).
2. The server processes the data. We call this time d_process. It is also measured in milliseconds.
3. The result travels back from the server to the device. We call this time d_reply. It is measured in milliseconds.

The total time for the cloud approach, which we will call T_cloud, is the sum of these three parts:
T_cloud = d_send + d_process + d_reply

Next, let us look at the Edge approach. An edge computer is located very close to the device, often on the same local network or even inside the device itself. Because it is so close, we can assume there is no significant network delay for sending data back and forth. The only time that matters is how long the edge computer takes to do the work. We call this time d_edge_process.

The total time for the edge approach, which we will call T_edge, is simply:
T_edge = d_edge_process

Now that we have two total times, we can compare them in two useful ways: by speedup and by percentage reduction.

Speedup tells us how many times faster the edge computer is compared to the cloud. To find this, we divide the slower time (cloud) by the faster time (edge):
Speedup = T_cloud divided by T_edge

Percentage reduction tells us how much less time the edge approach takes, expressed as a part of 100. To find this, we first find the difference in time between the two methods (T_cloud minus T_edge). Then we divide that difference by the original cloud time (T_cloud) to see what fraction of the total time was saved. Finally, we multiply by 100 to turn that fraction into a percentage:
Latency reduction percentage = ((T_cloud - T_edge) divided by T_cloud) times 100 percent


## A worked example

A smart factory sensor sends a data packet to the cloud for analysis. The one-way network latency to the cloud is 150 milliseconds. The cloud processes the packet in 10 milliseconds. The reply then travels back another 150 milliseconds. Alternatively, an edge computer next to the sensor processes the same packet in 5 milliseconds, and the result is used instantly, no network round trip.

Step 1: Calculate the total cloud time (T_cloud).
We add the send time, the process time, and the reply time together.
d_send = 150 ms
d_process = 10 ms
d_reply = 150 ms
T_cloud = 150 + 10 + 150
T_cloud = 310 ms
This is the full time from the sensor sending data to receiving a response.

Step 2: Calculate the total edge time (T_edge).
The edge computer processes the packet locally. We assume no network delay for the return trip because the result is used instantly on-site.
d_edge_process = 5 ms
T_edge = 5 ms
This is the total time from the sensor sending data to receiving a response via the edge.

Step 3: Calculate the Speedup.
We divide the cloud time by the edge time to see how many times faster the edge is.
Speedup = T_cloud divided by T_edge
Speedup = 310 divided by 5
Speedup = 62
The edge computing approach is 62 times faster than the cloud approach for this one request.

Step 4: Calculate the Latency Reduction Percentage.
First, find the time saved: T_cloud - T_edge = 310 - 5 = 305 ms.
Next, divide the time saved by the original cloud time: 305 divided by 310 ≈ 0.98387.
Finally, multiply by 100 to get the percentage: 0.98387 times 100 = 98.39%.
The edge approach reduces latency by approximately 98.4%.


## From scratch, in Python

This concept is about systems architecture, not a specific algorithm. However, we can write a tiny Python simulation to show how a device decides whether to process locally or send to the cloud, based on latency and a threshold.

First, we need two Python basics:
- A **list** is an ordered collection of values, written in square brackets. E.g., `[150, 10, 150]` holds three numbers.
- A **function** is a reusable block of code that takes inputs and returns a result. We define one with `def function_name(argument):` and use `return` to give back a value.

Here is a simulation:

```python
def should_use_edge(cloud_latency_ms, edge_latency_ms, threshold_ms):
    # Compare the latencies and decide
    if cloud_latency_ms > threshold_ms:
        decision = "edge"
    else:
        decision = "cloud"
    return decision

# Example: cloud round trip 310 ms, edge 5 ms, threshold 50 ms
result = should_use_edge(310, 5, 50)
print(result)  # prints 'edge'

# Let's also compute the speedup
def speedup(cloud_time, edge_time):
    return cloud_time / edge_time

print(speedup(310, 5))  # prints 62.0
```

The function `should_use_edge` checks if the cloud latency is above a threshold; if so, it recommends edge processing. The function `speedup` computes the factor by which edge is faster. This is a simple decision rule that a real device might use.


## What the code does

The code defines two functions. The first, `should_use_edge`, takes three numbers: the cloud latency, the edge latency, and a threshold. If the cloud latency is greater than the threshold, it returns the string `'edge'`, meaning we should process locally; otherwise it returns `'cloud'`. In the example, cloud latency is 310 ms, which is greater than the threshold 50 ms, so the decision is edge. The second function, `speedup`, simply divides the cloud time by the edge time to give the factor of improvement (62.0). This illustrates the key trade-off: when the cloud is too slow relative to a threshold, you move computation to the edge.


## Common misconceptions

**Edge replaces the cloud** — Once I move computation to the edge, I don't need the cloud at all.

The cloud is still used for training models, storing large datasets, and handling complex queries that need more compute than a phone or sensor can provide. Edge and fog are not replacements; they are tiers in a continuum. You still need the cloud for the heavy lifting. In fact: Edge and fog reduce the amount of data sent to the cloud by processing some of it locally, but the cloud remains essential for model training, global coordination, and tasks that require extensive resources.

**Fog is just another name for edge** — Fog computing and edge computing are the same thing.

Edge is on the device itself. Fog is a layer between the device and the cloud, often a local server or gateway. Fog aggregates data from many devices, does some processing, and sends only summaries to the cloud. It's a different point on the continuum. In fact: Fog is an intermediate layer that sits between edge devices and the cloud, providing a balance of latency and capacity — it is one step closer to the cloud than the edge is.

**Lower latency is the only reason** — I only need edge computing when I need speed.

Latency is one driver, but bandwidth and privacy are equally important. Sending massive amounts of sensor data over a cellular network could be prohibitively expensive, and sending sensitive health or financial data to the cloud raises privacy concerns. Edge computing can reduce traffic and keep data local. In fact: Edge computing is driven by three factors: latency (response time), bandwidth (how much data you ship over the network), and privacy (keeping sensitive data on the device).

**Edge devices are weak and useless** — Edge devices can't run real AI models; they're too small.

Modern edge devices like smartphones and dedicated AI chips (e.g., Google Edge TPU, NVIDIA Jetson) are powerful enough to run small to medium models efficiently, sometimes with quantisation to reduce memory. You wouldn't run a 70-billion-parameter model on them, but you can run a distilled small language model or a YOLO object detector. In fact: Edge devices can run many useful AI models, especially after optimisation like quantisation, pruning, and knowledge distillation. They excel at small, fast, on-device tasks.

**Security is automatic** — Edge computing is more private because data doesn't leave the device, so security is automatically better.

Edge devices are physically accessible, can be stolen or tampered with, and often lack strong security updates. The data is still at rest on the device, which is a risk. Also, models deployed on edge can be reverse-engineered. In fact: Edge computing improves privacy by avoiding transmission, but it introduces new security challenges, like secure storage on the device, update mechanisms, and resistance to physical tampering. 'Private at rest' is not automatically 'secure'.


## In short

Edge and fog computing are not alternatives to the cloud but an extension of it, moving computation closer to the data source to reduce latency, save bandwidth, and protect privacy. The continuum goes: device (edge) → local gateway (fog) → cloud. Learn to decide where to run each part of your AI system. Practical decisions require simple arithmetic: compare round-trip times, and use thresholds. In practice, you'll use frameworks like TensorFlow Lite or PyTorch Mobile to run models on devices, and message brokers to manage the flow. Master this and you'll be able to design AI systems that actually work in the real world, where milliseconds and privacy matter.


## Where this is used

Edge and fog computing are everywhere in modern AI, especially where speed, bandwidth, or privacy matter. Here are the concrete ways developers use it:

- **Large Language Models (LLMs)**: LLMs like ChatGPT run in the cloud because they need enormous compute. However, there is movement toward running small language models on-device (the edge) for quick tasks like autocomplete or voice dictation. Apple's on-device models for Siri are an example. Developers use frameworks like CoreML or TensorFlow Lite to convert a model to run on a phone, avoiding sending every keystroke to the cloud. The trade-off is a smaller, less capable model but instant response and full privacy.

- **Small Language Models on devices**: When a model must run on a phone or a laptop, you cannot use a 70-billion-parameter model. You use a distilled version, maybe with only a few hundred million parameters, and you quantize it to reduce size. The edge device does the inference. This is used in offline translation, keyboard autocorrect, and smart replies. Developers use TensorFlow Lite, ONNX Runtime, or PyTorch Mobile to deploy these models.

- **Retrieval-Augmented Generation (RAG)**: RAG combines a retrieval system with an LLM. The retrieval step can happen at the edge: a device identifies relevant documents locally, then sends only those snippets to the cloud LLM. This reduces bandwidth and keeps private data on the device. For example, a medical app could search a local medical database (on the edge) and then ask the cloud LLM to synthesise an answer, without sending the entire patient record.

- **Agentic systems that plan, call tools, and act in loops**: Agents often need to react to real-time events. In a smart home, an agent might process sensor data locally to decide whether to raise an alarm, and only alert the cloud for complex decisions. The edge handles time-critical actions (closing a valve, turning off a motor) while the cloud does long-horizon planning. Developers use message brokers like MQTT to communicate between edge nodes and the cloud.

- **Everyday applied machine learning**: In recommendation systems for retail, edge devices track user interactions and run a model to suggest products without sending every click to the cloud. In industrial forecasting, sensors on machinery use edge models to predict failures locally. In fraud detection, a point-of-sale terminal checks a transaction on-device against a local model to approve or deny in milliseconds, even if the internet is down. In vision systems for autonomous vehicles, edge computers (like NVIDIA Jetson) run object detection models in real time; a delay would be fatal. In speech recognition, smart speakers process wake words on-device, only streaming the full request to the cloud when needed.

**What developers write**: For edge inference, they typically write code in Python using TensorFlow Lite or PyTorch Mobile. For example, in PyTorch Mobile:

```python
import torch
model = torch.jit.load('model.pt')
model.eval()
with torch.no_grad():
    output = model(torch.tensor(data))
```

underneath, this loads the model into memory and runs the forward pass entirely on the device, without sending data to a server. For fog nodes that aggregate data from many sensors, developers often use cloud-like frameworks but run them on local servers, e.g., using Docker containers on an on-premises gateway. The code could collect sensor readings, compute summary statistics, and only upload those summaries to the cloud.

**What breaks when done badly**: If you ignore edge computing and send all data to the cloud, you get high latency (a robot that misses a deadline), high bandwidth costs (streaming video from every security camera), and privacy breaches (sending sensitive health data to a remote server). Conversely, if you move too much to the edge, you might overburden low-powered devices, drain their batteries, or make it hard to update models consistently. Poorly managed edge networks become a chaos of disconnected devices with conflicting versions, which is a security nightmare. A well-designed system carefully chooses what runs where, monitors the fleet, and updates models over the air.


---

Read this on the web: https://basavapurushottam.com/learn-ai/edge-and-fog-computing
