# Sentiment Analysis

Classifying the polarity or attitude expressed in text, the most widely deployed text classification task.

**Rests on:** Two-Class and Multiclass Classifiers

**Leads to:** Emotional Intelligence in NLP, Financial Sentiment, NLP for Mobility and Transportation, Sentiment and Emotion Bias

*Deep Learning · Natural Language Processing*


## Why learn this

This concept is the most widely deployed text classification task, and it will be your gateway into processing language with computers. Once you master this skill, you'll have the foundation to understand more complex tasks: identifying specific emotions (like joy or anger), determining a person's position on an argument, or understanding what they want you to do (intent detection). You will also need these patterns for tools that power search results and for spotting fake news. In practice, this is the first thing you'll build when you work with text data, making it the perfect entry point. For example, consider a customer review: 'I loved this movie' has 1 positive word and 0 negative words, while 'The service was terrible' has 1 negative word and 0 positive words. Simple counting works here, but real life is messier.


## The intuition

Think of sentiment analysis as teaching a computer to read a review and tell you if the writer was happy or unhappy. It's like a quick poll of opinion: 'I loved this movie' → positive, 'The service was terrible' → negative. But the computer doesn't understand meaning like you do. It only sees numbers. So we must convert words into numbers in a way that captures their feeling. The simplest way is to count how many positive words and negative words appear. If more positives, it's positive. If more negatives, it's negative. Of course, real sentences are trickier: 'not good' is negative, but a simple count of 'good' would say positive. That's where the magic of machine learning comes in — we can train a model to learn these patterns. For now, let's start with the basics: counting positive and negative words.


## An analogy

Imagine you're a teacher grading short answer essays. You don't read every word; you scan for keywords that signal the student's attitude. You circle 'excellent', 'great', 'clear' with a green pen (positive), and 'confusing', 'boring', 'wrong' with a red pen (negative). At the end, you count the green marks and the red marks. If green outnumber red, the essay gets a positive note. That's sentiment analysis in a nutshell.


## Definition

Sentiment analysis is the task of automatically determining the direction of feeling expressed in a piece of text, typically classifying it as positive, negative, or neutral.


## How it connects

You haven't studied any other concepts yet, so this is your starting point. It belongs to the field of Natural Language Processing (NLP), which is the computational processing of human language. Your notes say that sentiment is 'the easy case' relative to emotion, stance, and intent — those are harder because they involve more nuance. But you'll build on sentiment analysis to understand more advanced ideas like tools that convert text into numbers for search retrieval and systems that score how well two pieces of text match each other. Your library also mentions that 'aspect-level sentiment is far more actionable than document-level' — that's a variation we'll touch on.


## The mathematics

The mathematics behind the simplest sentiment classifier is just arithmetic: counting and comparing. You only need addition and comparison.

To classify a text, you count how many positive words it contains and how many negative words it contains. If the positive count is greater, the text is positive. If the negative count is greater, it's negative. If they are equal, it's neutral. That's it. There's no algebra, no functions, nothing beyond what you've done in school.

More advanced methods use algebra and probability, but the core idea remains this simple comparison. Even machine learning models that use many numbers ultimately output a single number (a score) that you compare to a threshold to decide the sentiment. So the branch of mathematics here is basic arithmetic and comparison — nothing more.

Let P be the number of positive words. Let N be the number of negative words. The rule is:
- Sentiment is positive if P is greater than N.
- Sentiment is negative if N is greater than P.
- Sentiment is neutral if P equals N.


## A worked example

We have a text: 'The movie was fantastic, but the ending was disappointing.' We have a small dictionary of positive words: ['fantastic', 'good', 'great'], and negative words: ['bad', 'disappointing', 'terrible'].

Step 1: Positive count. Look at each word in the text. 'Fantastic' is in the positive list. Count it as 1. No other words are in the positive list. Total positive count P = 1.

Step 2: Negative count. Look at each word in the text. 'Disappointing' is in the negative list. Count it as 1. No other words are in the negative list. Total negative count N = 1.

Step 3: Compare. We have P = 1 and N = 1. Is 1 greater than 1? No. Is 1 less than 1? No. They are equal.

Result: The text is classified as neutral, even though a human might see it as mixed. This shows the simplicity of the counting method.


## From scratch, in Python

```python
# Simple sentiment classifier using a word count approach

text = "The movie was fantastic, but the ending was disappointing."

# Define a list of positive and negative words
positive_words = ["fantastic", "good", "great", "excellent"]
negative_words = ["bad", "terrible", "disappointing", "awful"]

# Convert text to lowercase and split into words
words = text.lower().split()  # ['the', 'movie', 'was', 'fantastic,', 'but', 'the', 'ending', 'was', 'disappointing.']

# Count positive and negative words
positive_count = 0
negative_count = 0

for word in words:
    # Remove punctuation (just period and comma)
    word = word.strip(",.!?")
    if word in positive_words:
        positive_count += 1
    elif word in negative_words:
        negative_count += 1

# Decide sentiment
if positive_count > negative_count:
    sentiment = "positive"
    score = positive_count - negative_count
elif negative_count > positive_count:
    sentiment = "negative"
    score = negative_count - positive_count
else:
    sentiment = "neutral"
    score = 0

print("Positive count:", positive_count)
print("Negative count:", negative_count)
print("Sentiment:", sentiment)
print("Score (magnitude):", score)
```


## The same thing, with a framework

```python
# Using scikit-learn's CountVectorizer and LogisticRegression
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline

# Example training data (text, label)
train_texts = ["great movie", "terrible movie", "good acting", "bad plot"]
train_labels = [1, 0, 1, 0]  # 1 = positive, 0 = negative

# Create a pipeline that turns text into word counts, then trains a logistic regression
model = make_pipeline(CountVectorizer(), LogisticRegression())
model.fit(train_texts, train_labels)

# Predict sentiment on new text
new_text = "fantastic film"
prediction = model.predict([new_text])[0]
print("Prediction:", "positive" if prediction == 1 else "negative")
```


## What the code does

The from-scratch code does exactly what we've been doing by hand. First, we define a list of positive and negative words. A list in Python is an ordered collection of items, like a shopping list. We write it with square brackets, and separate items with commas. Then we convert the text to lowercase (so 'Fantastic' becomes 'fantastic') and split it into individual words. The split() method returns a list of words. Then we loop through each word using a for loop — a for loop repeats the same block of code for each item in a list. Inside the loop, we remove punctuation (like commas and periods) so that 'fantastic,' becomes 'fantastic'. Then we check if the word is in the positive_words or negative_words list using the 'in' keyword. If it is, we increment (add 1 to) the respective count. After the loop, we compare the counts and print the result. The library version with scikit-learn is much more powerful: it automatically creates a matrix of word counts (that's what CountVectorizer does) and trains a LogisticRegression model, which can learn weights for each word instead of just counting equally. This is what a developer would actually use in practice, because it handles nuances better and can be trained on large datasets.


## Common misconceptions

**Sentiment analysis is always accurate** — AI can read sarcasm and understand nuance, so it never gets it wrong.

Sarcasm, negation, and domain-specific language are persistent failure cases. If I say 'Oh great, another meeting', a simple model sees the word 'great' and thinks positive, but the tone is negative. Even advanced models struggle with this without context. In fact: Sentiment analysis is a statistical prediction, not a reading of mind. It can be fooled by irony, ambiguity, and out-of-vocabulary words. Accuracy varies by domain and data quality.

**More words means better accuracy** — The longer the text, the more accurate the sentiment prediction will be.

Long text can contain mixed sentiments, sarcasm, and many entities. A long review can be positive about one aspect and negative about another, but document-level sentiment would flatten that into a single label, losing nuance. In fact: Aspect-level sentiment, which looks at specific aspects (like 'camera' or 'battery'), is far more actionable and accurate for long reviews. Document-level is a simplification.

**Negative words always indicate negative sentiment** — Any sentence containing 'bad' or 'terrible' must be negative.

Consider 'The movie was not bad'. The word 'bad' is present, but the meaning is positive (or at least neutral). Negation flips the polarity. Also, words like 'sick' (in slang) can mean good. In fact: Context and negation matter. A word's polarity depends on its surrounding words. Modern models learn this from data, but simple word counting cannot.

**Sentiment analysis is only about positive/negative** — Sentiment analysis just tells you if a text is happy or sad, that's it.

Sentiment analysis can be more nuanced: there's neutral sentiment, and there are finer-grained scales (e.g., 1 to 5 stars). There's also emotion detection (anger, joy, sadness), stance detection (for/against), and intent detection (what the user wants). Your library says sentiment is the easy case; emotion, stance, and intent are harder. In fact: Sentiment is a simple polarity. Emotion, stance, and intent are more detailed and difficult tasks that build on similar techniques.

**The model stores the training examples** — The sentiment model memorizes all the reviews it was trained on and just looks up the answer.

A model compresses patterns into numbers (weights). It doesn't store the text. For example, a logistic regression model stores a weight for each word, not the entire dataset. This is why it can generalize to new, unseen sentences. In fact: The model learns a function that maps text to a sentiment score based on the patterns in the training data, but it does not retain the input text itself.


## In short

Sentiment analysis is the task of automatically classifying the polarity (positive, negative, neutral) of a piece of text. At its heart, it's a simple count and comparison of positive vs. negative words, but real-world challenges like sarcasm, negation, and mixed aspects require more sophisticated models. It is the easiest of the NLP classification tasks, but it's the foundation for understanding more advanced topics like emotion detection, stance detection, and retrieval-based systems. Remember that a machine doesn't 'understand' language; it learns patterns from data.


## Where this is used

Sentiment analysis is everywhere, and here's how it's used in building AI systems. In large language models (like GPT-4), sentiment analysis is not a separate component, but the model can be fine-tuned to perform it. More commonly, LLMs are used in a zero-shot way: you ask the model to classify text by providing a prompt like 'Classify the sentiment of this review as positive, negative, or neutral.' The model's training already includes sentiment reasoning, so it can handle sarcasm and negation better than a simple count. Developers use tools like Hugging Face's `transformers` library, and they might call `pipeline('sentiment-analysis')` to get a sentiment score. For small language models that run on your phone (like TinyML keyword spotters), the approach is more constrained. You can train a lightweight model (e.g., a small neural network) on word counts or embedded features, and it will output a sentiment score. The key is that these models must be very small and fast, so they use simplified architectures. In retrieval-augmented generation (RAG), sentiment analysis can be used to filter or rank documents based on sentiment. For example, when answering a question about a product, you might first retrieve reviews and then the LLM uses sentiment to summarize whether people liked it. The embedding and similarity machinery underneath is separate, but sentiment is a useful feature to add. In agentic systems that plan and call tools, sentiment analysis might be a tool the agent calls to gauge user feedback. For instance, an agent monitoring customer feedback might call a sentiment analysis API on new reviews and raise an alert if sentiment turns negative. In everyday applied machine learning, sentiment analysis is the workhorse for social media monitoring, brand tracking, customer service automation, and product analytics. A developer might use a pre-trained model from `scikit-learn` or `fastai`, or turn to commercial APIs like AWS Comprehend or Google Cloud NLP. What they write depends on the libarary: with scikit-learn they build a pipeline, with fastai they might fine-tune a language model, and with a commercial API they just send a POST request. What breaks when it's done badly? Sarcasm, negation, and domain-specific terms (e.g., 'sick' meaning 'good' in slang) cause frequent errors. Also, if the data is biased (e.g., training on reviews from one region), the model will not generalize. A developer must be careful with preprocessing, handle class imbalance, and evaluate on a held-out test set that reflects the real input distribution. As your library notes, aspect-level sentiment (evaluating sentiment for a specific aspect like 'camera quality' vs 'battery life') is far more actionable than document-level. This is a more granular form of sentiment analysis that extracts sentiment for each entity or aspect in a sentence.


---

Read this on the web: https://basavapurushottam.com/learn-ai/sentiment-analysis
