# Unstructured Data

Text, image, audio and video content lacking a predefined schema, which now dominates data volume.

**Rests on:** Data Cleaning

**Leads to:** Found Data

*Data Engineering and Analytics · Data Quality*


## Why learn this

Unstructured data holds most of the value in modern AI systems. If you can process text, images, audio, and video — information that does not fit into neat rows and columns — you can build tools that read documents, recognize faces, transcribe speech, and answer questions. This idea is the starting point for everything else: cleaning data, changing its format, preparing it for analysis, and eventually training computer programs to learn from it. Without grasping what unstructured data is and why it is difficult, you cannot follow discussions about turning text into numbers, using past information to guide new answers, or large language models. Mastering this first makes every later page in this course easier.


## The intuition

Imagine your office has two kinds of files. One shelf holds neatly labeled folders, each with the same pre-printed forms: name, date, amount, signature. Every form has the same boxes in the same order. That is structured data — like a spreadsheet, where every row has the same columns and every cell holds one specific type of thing. Now look at the other shelf. It is a pile of loose papers: handwritten notes, printed emails, newspaper clippings, photographs, audio cassettes. There is no consistent shape. One page might be a letter with a date at the top, another is a receipt with no date at all, a third is a scribbled diagram. To extract anything useful, you have to read each one and decide what matters. That is unstructured data — text, images, audio, and video that do not come in a predefined table. It has no built-in structure. The information is there, but it is embedded in human language and perception, not in rows and columns. This is the data that now dominates the world: social media posts, customer reviews, support tickets, medical images, security footage, voice messages. And because it is unstructured, it is the hardest to process — but also the richest. Every time you type a question into a search engine and get a sensible answer from a web page, or ask a voice assistant to play a song, you are relying on a system that has somehow extracted structure from unstructured data. The extraction is the hard part, and it is where the value of AI really lies.


## An analogy

Think of structured data as a library catalogue and unstructured data as the books themselves. The catalogue has fixed fields — title, author, ISBN, shelf location — every card looks the same, and you can search it quickly with simple rules. But the actual knowledge is in the books. The books have no consistent structure: a history book and a novel and a cookbook all look different, have different chapters, different fonts, different layouts. To answer a question like 'What did the author think about freedom?', you cannot just look at the catalogue — you have to read the book. Reading is hard for a computer: it must parse prose, understand context, recognize irony, and pick out the relevant passages. Now push the analogy further: the books are not even paper. Some are photographs of people, others are audio recordings of conversations, others are video clips of a machine in a factory. To extract any structure from these, you need computer vision to recognize faces and objects, speech recognition to convert sound to words, and natural language processing — the method for interpreting meaning in human text — to understand the content. The analogy breaks down here, because a library catalogue is still a structured summary — but real unstructured data is raw and messy, with no summary already made. You have to build the summary yourself, and that is the essence of the challenge. The AI techniques you will learn later are all ways of automatically extracting that structure from unstructured data.


## Definition

Unstructured data is information that does not have a predefined, fixed schema or data model, and therefore cannot be easily stored in traditional rows-and-columns tables; it includes text, images, audio, and video, and extracting usable structure from it requires significant processing.


## How it connects

This concept is the first in the Data Quality branch of the Personal AI Mastery System. It builds directly on the idea of data quality, which concerns whether data is accurate, complete, timely, and consistent — but for unstructured data, quality is harder to judge because there is no built-in structure to validate against. It connects to Data Cleaning, because cleaning unstructured data is very different from cleaning a clean table of numbers. It connects to Data Transformation, because changing unstructured data into structured form (like extracting numbers from a document) is a major step in many workflows.


## The mathematics

This concept does not itself require mathematical operations — it is about a category of data. However, to understand how computers handle unstructured data, you need one key idea: converting messy text or images into numbers. That conversion uses a branch of mathematics called **linear algebra**, which is the study of lists of numbers and how to combine them. Later you will meet **vectors** (which are just ordered lists of numbers, like `[0.5, 1.2, -3.0]`), and operations like the **dot product** (which multiplies corresponding numbers and adds the products). For now, the only mathematical idea you need is that a computer can represent any piece of unstructured data as a long list of numbers. For text, this might be counting how many times each word appears. For images, it is the brightness of every pixel. For audio, it is the sound amplitude at each moment. The numbers are the **features** — the measurements that describe the data. Once you have numbers, you can apply mathematical techniques to compare, cluster, and classify the data. But the conversion itself is not mathematics — it is a design choice about what to measure.

There is no single formula for unstructured data. The key idea is that any piece of unstructured data can be mapped to a vector of numbers via a feature extraction function:  data → vector (e.g., word counts, pixel brightness, audio amplitudes). The vector lives in a high-dimensional space, and similarity between two pieces of data can be measured using a distance formula (e.g., Euclidean distance = sqrt((x1-y1)^2 + (x2-y2)^2 + ...)).


## A worked example

```
Imagine you have three short movie reviews: 'I loved it', 'It was boring', 'The plot was great but acting poor'. You decide to represent each review by counting how many times the words 'love', 'boring', 'great', 'poor' appear.

Step 1: Review 1: 'I loved it' -> [love=1, boring=0, great=0, poor=0]   # The word 'loved' appears once, the others zero.
Step 2: Review 2: 'It was boring' -> [love=0, boring=1, great=0, poor=0]   # The word 'boring' appears once.
Step 3: Review 3: 'The plot was great but acting poor' -> [love=0, boring=0, great=1, poor=1]   # Both 'great' and 'poor' appear once.

Result: Each review is now a list of four numbers. This is a 'vector' — a way to turn unstructured text into structured numbers that a computer can work with. The choice of which words to count is a 'feature extraction' decision.
```


## From scratch, in Python

```python
# Represent unstructured data as numbers: word counts for a tiny set of reviews

# Step 1: Define the set of words we care about (our 'vocabulary')
vocab = ['love', 'boring', 'great', 'poor']

# Step 2: Write the reviews as plain strings (lists of characters)
reviews = [
    'I loved it',
    'It was boring',
    'The plot was great but acting poor'
]

# Step 3: Convert each review to a vector of word counts
# We'll use a list of lists: each inner list is one review's vector
vectors = []

for review in reviews:
    # Start a vector of all zeros, one for each word in vocab
    vector = [0, 0, 0, 0]
    # Split the review into lowercase words
    words = review.lower().split()
    # For each word, if it's in our vocab, increment its count
    for word in words:
        if word in vocab:
            index = vocab.index(word)
            vector[index] += 1
    vectors.append(vector)

# Step 4: Compute a simple similarity between two reviews using dot product
# Dot product: multiply corresponding numbers and add
review1 = vectors[0]
review2 = vectors[2]
similarity = 0
for i in range(len(vocab)):
    similarity += review1[i] * review2[i]

print('Vectors:', vectors)
print('Similarity between review 1 and review 3:', similarity)
```


## The same thing, with a framework

```python
In real applications, you wouldn't write loops yourself. You'd use libraries like NumPy to handle numbers efficiently, or PyTorch for deep learning. For example, with NumPy: `import numpy as np`, `vectors = np.array([...])`, `similarity = np.dot(vectors[0], vectors[2])`. The `np.dot` function does exactly the same multiply-and-add loop we wrote, but in compiled C under the hood, making it much faster. You could also use `sklearn.feature_extraction.text.CountVectorizer` to convert text to word-count vectors automatically, but it's important to know what it's doing underneath.
```


## What the code does

This code is the plain-Python foundation for handling unstructured text. First, we define a vocabulary: a list `vocab` that holds the four words we decided to count. A **list** in Python is an ordered collection that can be changed; here it holds strings. Then we have `reviews`: a list of three strings, each a whole review. Strings are just text between quotes. The heart of the script is the loop over each review. The line `review.lower().split()` converts the review to lowercase and splits it into a list of words on spaces. For example, 'I loved it' becomes `['i', 'loved', 'it']`. Then for each word, we check if it is in our vocabulary. If yes, we find its position in the vocab list with `vocab.index(word)` and increment the corresponding slot in the vector. The vector starts as `[0, 0, 0, 0]` and gets updated. After processing all reviews, we have `vectors`, a list of lists — a table of numbers. Finally, we compute a similarity score between the first and third reviews using a **dot product**: we multiply matching numbers and add them up. This is a measure of how much the two reviews share the same vocabulary words. The same idea underlies much more sophisticated text-matching in real AI.


## Common misconceptions

**Unstructured means it's random or meaningless** — If data is unstructured, it's just a pile of noise with no value.

Unstructured data often contains high-value information — a customer review, a medical scan, a voice recording — but it's encoded in human language or perception, not in a fixed table. The structure is implicit, not absent. In fact: Unstructured data has rich internal structure (grammar, visual patterns, audio rhythms), but that structure is not pre-defined for the computer — it must be extracted, which is exactly what AI models do.

**Unstructured data can't be stored in a database** — Because it doesn't fit in rows and columns, you can't use a database for it.

Modern databases (NoSQL like MongoDB) store JSON documents, and even relational databases can store text blobs or binary objects. The point is not that it's un-storable, but that querying it in its raw form is limited. In fact: Unstructured data can be stored in any file system or database, but to search it meaningfully, you need to convert it to structured form (e.g., embeddings) and then index that.

**Structured data is always better** — Structured data is 'cleaner' and more useful, so we should always prefer it.

Not all information can be captured in a predefined schema. The nuance, tone, and context of a text or image would be lost if forced into a table. Unstructured data often contains the richest signal. In fact: Structured data is easy to compute with, but unstructured data is where most of the real-world information lives. Good systems combine both: extract structure from unstructured data where possible, and keep the original for deeper analysis.

**All text files are unstructured data** — Everything that's not in a spreadsheet is unstructured, including CSV files or JSON.

CSV files have a delimiter structure; JSON has a key-value schema. They are self-describing, even if not in a relational table. In fact: Semi-structured data (JSON, XML, CSV) has some organisational properties, but still often requires parsing. Fully unstructured data (raw text, images, audio) has no such built-in structure.

**Image data is structured because it's a grid of pixels** — An image is just a matrix of numbers, so it's structured data.

While pixels are numeric, the meaning of the image (a cat, a face) is not embedded in the pixel grid in any simple way. The same arrangement of numbers can represent totally different concepts. In fact: The raw pixel values form a structured grid, but the semantic content is unstructured. Extracting that semantic meaning requires complex models (CNNs) — that's why images are treated as unstructured data.

**Once you vectorise the data, it becomes structured for good** — After converting text to numbers, it's now structured data and you can treat it like a spreadsheet.

The vector is just a representation; the underlying data is still unstructured. The vector may embed meaning but you still need further processing to interpret it. In fact: Vectorisation is a transformation step that makes unstructured data computable, but it is not the same as structuring it into a fixed schema with clear field semantics.

**Unstructured data can't be used for machine learning** — You need a clean table of numbers to train a model; unstructured data is useless.

Machine learning models, especially deep learning, work directly on images, text, and audio. The model itself extracts the structure (e.g., convolutions for images, embeddings for text). In fact: Modern AI is designed to handle unstructured data — it learns the features during training, rather than you supplying them manually. This is why unstructured data is now so valuable.

**Unstructured data is a small part of the data world** — Most data in companies is structured, in tables.

Surveys estimate that 80-90% of data generated today is unstructured (text, video, images, social media). The structured portion is a small slice. In fact: Unstructured data dominates in volume. The challenge is not scarcity but making it useful — that's where AI techniques come in.


## In short

Unstructured data — text, images, audio, video — has no predefined schema, yet contains most of the world's information. To use it with computers, you convert it to numbers (vectors) using feature extraction or learned embeddings. This conversion is the cornerstone of everything from search engines to ChatGPT. Mastering this idea sets you up for every subsequent topic in this system: data cleaning, transformation, and ultimately machine learning.


## Where this is used

Unstructured data is the raw material for almost every modern AI system. Here is how it shows up, and what the developer actually does. 

**Large Language Models (LLMs).** LLMs like ChatGPT are trained on vast amounts of unstructured text — books, web pages, forums, Wikipedia. The model's job is to predict the next word given the previous words. To do that, the raw text, which is a sequence of characters, must be turned into numbers. The developer splits the text into tokens (small pieces, like words or subwords) and converts each token into a vector of numbers using an embedding matrix — this is the conversion from unstructured to structured we described. Without this step, the neural network could not compute anything. During training, the model learns patterns in these sequences, and that knowledge is stored in its billions of weights. When you use an LLM, you give it a prompt (a sequence of tokens), and it generates new tokens one by one. This is where a huge amount of engineering goes: tokenisation, batching, and memory management.

**Small Language Models (on a phone or laptop).** When a model has to run on a device with limited memory, the challenge is dealing with unstructured data efficiently. The developer must compress the model (e.g., via quantization) so that the token-to-vector conversion and all the subsequent matrix multiplications fit in the device's RAM. They may also pre-process the input text to strip out formatting before tokenising. The same fundamental technique applies, but the implementation is optimised for speed and size.

**Retrieval-Augmented Generation (RAG).** This is how an LLM can answer questions about documents it wasn't trained on. The developer builds a pipeline that takes a corpus of unstructured documents (PDFs, web pages, emails) and converts each document into a vector using an embedding model. These vectors are stored in a database. When a user asks a question, the question is also converted into a vector, and the system finds the documents whose vectors are most similar to the question's vector — using a distance metric like the dot product or cosine similarity. This is literally the same operation we did in the code section: representing text as numbers and comparing them. The retrieved documents are then fed to the LLM along with the question, so the model can answer based on that context. A developer writes code that calls API endpoints for embedding, stores vectors in a vector database (like Pinecone or FAISS), and implements a search function.

**Agentic Systems.** AI agents that plan, call tools, and act in loops also depend on unstructured data. The agent receives instructions as natural language (unstructured text), which must be turned into a machine-understandable plan. It may need to read a document, parse its content, and decide what to do. For example, an agent that answers emails must parse incoming messages (text), extract the intent, and draft a reply. The developer uses LLMs to do this parsing, but also writes auxiliary code to validate the agent's output, call external APIs, and manage state. The unstructured data handling is often the bottleneck: extracting reliable facts from messy text is error-prone, and the agent must handle those errors gracefully.

**Everyday applied machine learning.** Beyond LLMs, unstructured data is everywhere. 
- **Recommendation systems:** They often use user behaviour (structured) but also product descriptions, images, and reviews (unstructured). To recommend a movie, the system might embed the plot summary and poster image into vectors and compare them with user preferences. 
- **Vision:** Image recognition models take raw pixel values (a 2D grid of numbers) as input. A developer preparing a dataset of photos must resize, normalise, and augment images before feeding them to a CNN. 
- **Forecasting:** Time-series data (e.g., sensor readings) is technically structured, but often comes with unstructured notes or logs that are used for explanation. 
- **Fraud detection:** Models use structured transaction data but also email transcripts, chat logs, and claim documents (all unstructured) to detect patterns of fraud. 
- **Speech:** Voice assistants convert audio waveforms (a long sequence of amplitude values — unstructured) into text via automatic speech recognition (ASR), then process that text with NLP. 

Across all these, the developer writes code that pre-processes the unstructured data into a structured form (vectors, numbers), feeds it into a model, and post-processes the output. The failures often come from poor handling of unstructured input: missing values, inconsistent formats, or data that doesn't fit the expected schema. That's why understanding unstructured data is the first step to building reliable AI.


---

Read this on the web: https://basavapurushottam.com/learn-ai/unstructured-data
