Coder Legion - Recent posts tagged sentimentanalysis

Smart Movie Recommender using Linear SVC

Sun, 19 Apr 2026 14:49:45 +0000

Project Overview

This project focuses on building an end-to-end Machine Learning pipeline to analyze user sentiment from IMDB movie reviews. Unlike standard binary classifiers, this system incorporates Decision Logic to identify neutral feedback, providing a more nuanced recommendation: "Recommend," "Do Not Recommend," or "Maybe Watch."

How I Built It (Technical Workflow)

Advanced Text Preprocessing:
Raw text is often noisy. I developed a cleaning pipeline using NLTK and Regex:
- Noise Removal: Stripped HTML tags and non-alphabetic characters.
- Normalization: Converted all text to lowercase for consistency.
- Stopword Removal & Lemmatization: Removed common fillers (e.g., "the", "a") and reduced words to their dictionary root (e.g., "watched" becomes "watch") to reduce dimensionality.
Feature Engineering (TF-IDF):
To translate text into a language machines understand, I used TF-IDF (Term Frequency-Inverse Document Frequency). I limited the vocabulary to the top 5,000 features to ensure the model focuses on the most impactful words while ignoring rare noise.
Model Implementation:
I chose Linear SVC (Support Vector Classification). It is highly efficient for high-dimensional text data and provides a clear decision boundary between sentiments.

Challenges & Solutions

1. The Neutral Data Gap

The Challenge: The IMDB dataset is strictly binary (Positive/Negative). This meant the model had never "seen" a neutral review, causing it to force-classify "average" movies as either good or bad.
The Solution: Instead of using model.predict(), I utilized model.decision_function(). By calculating how close a review lies to the decision boundary (Thresholding), I was able to manually trigger a Neutral (0) classification for reviews that lacked strong positive or negative conviction.

2. Balancing Overfitting

The Challenge: Initially, the model showed high training accuracy but lower performance on new data.
The Solution: I leveraged L2 Regularization (inherent in Linear SVC). By monitoring the gap between training and testing scores, I ensured the model generalized well. Limiting the TF-IDF features also helped prevent the model from "memorizing" specific review noise.

How To Use This Project

Setup: Ensure scikit-learn, pandas, and nltk are installed in your Python environment.
Input: Pass a raw string (movie review) into the get_recommendation function.

Inference Example:

review = "The visuals were stunning, but the pacing was a bit slow and mediocre."
print(get_recommendation(review)) 
    
# Result -> Score: 0.04 | Sentiment: 0 | Action: Maybe watch (Neutral)

Final Performance

Test Accuracy: 88%
Generalization: The model maintains a balanced F1-score across both classes, proving it is robust against bias and ready for real-world unseen data.

Movie Review Sentiment Analysis

Sun, 19 Apr 2026 14:48:21 +0000

Project Overview

How I Built It (Technical Workflow)

Advanced Text Preprocessing:
Raw text is often noisy. I developed a cleaning pipeline using NLTK and Regex:
- Noise Removal: Stripped HTML tags and non-alphabetic characters.
- Normalization: Converted all text to lowercase for consistency.
- Stopword Removal & Lemmatization: Removed common fillers (e.g., "the", "a") and reduced words to their dictionary root (e.g., "watched" becomes "watch") to reduce dimensionality.
Feature Engineering (TF-IDF):
To translate text into a language machines understand, I used TF-IDF (Term Frequency-Inverse Document Frequency). I limited the vocabulary to the top 5,000 features to ensure the model focuses on the most impactful words while ignoring rare noise.
Model Implementation:
I chose Linear SVC (Support Vector Classification). It is highly efficient for high-dimensional text data and provides a clear decision boundary between sentiments.

Challenges & Solutions

1. The Neutral Data Gap

The Challenge: The IMDB dataset is strictly binary (Positive/Negative). This meant the model had never "seen" a neutral review, causing it to force-classify "average" movies as either good or bad.
The Solution: Instead of using model.predict(), I utilized model.decision_function(). By calculating how close a review lies to the decision boundary (Thresholding), I was able to manually trigger a Neutral (0) classification for reviews that lacked strong positive or negative conviction.

2. Balancing Overfitting

The Challenge: Initially, the model showed high training accuracy but lower performance on new data.
The Solution: I leveraged L2 Regularization (inherent in Linear SVC). By monitoring the gap between training and testing scores, I ensured the model generalized well. Limiting the TF-IDF features also helped prevent the model from "memorizing" specific review noise.

How To Use This Project

Setup: Ensure scikit-learn, pandas, and nltk are installed in your Python environment.
Input: Pass a raw string (movie review) into the get_recommendation function.

Inference Example:

review = "The visuals were stunning, but the pacing was a bit slow and mediocre."
print(get_recommendation(review)) 
    
# Result -> Score: 0.04 | Sentiment: 0 | Action: Maybe watch (Neutral)

Final Performance

Test Accuracy: 88%
Generalization: The model maintains a balanced F1-score across both classes, proving it is robust against bias and ready for real-world unseen data.

Sentiment Analysis Using NLP: Visualizing Emotions in Text with Python and Power BI

Fri, 17 Apr 2026 00:26:59 +0000

Introduction

Sentiment analysis is one of the most practical applications of Natural Language Processing (NLP).

In this project, I explore how to perform sentiment analysis on a short story and visualize emotional patterns using Python and Power BI.

The goal is simple: transform raw text into meaningful insights.

In this guide, we’ll walk through how to perform sentiment analysis using NLP step-by-step.

What is Sentiment Analysis in NLP?

Sentiment analysis is a technique used in NLP to determine whether a piece of text expresses a positive, negative, or neutral emotion.

It is widely used in:

Customer feedback analysis
Social media monitoring
Product reviews

In this case, I applied sentiment analysis to a literary text to understand how emotions evolve throughout a story.

Project Idea

Most people read stories…

But what if we could analyze how a story feels?

I used “The Tell-Tale Heart” by Edgar Allan Poe and broke it into smaller segments, then applied sentiment analysis to each part.

This allowed me to track emotional changes across the narrative.

How to Analyze Text Using NLP

The workflow for this project was:

Load and clean the text
Split the text into segments
Apply sentiment analysis
Generate a structured dataset
Visualize the results

Here’s a simplified Python example:

from textblob import TextBlob

text = "I felt nervous and terrified."
blob = TextBlob(text)

sentiment = blob.sentiment.polarity
print(sentiment)

This returns a sentiment score that helps classify the emotional tone of the text.

Building a Sentiment Analysis Dataset

After processing the text, I created a dataset like this:

   Segment  | Label   | Score
   1        | Positive| 0.91
   2        | Negative| 0.45

This step is critical because it transforms unstructured text into structured data that can be analyzed and visualized.

Visualizing Sentiment Analysis Results Using Power BI

Once the dataset was ready, I built a dashboard using Power BI to visualize:

Emotional trends over time
Distribution of positive vs negative segments
Key moments of emotional intensity

Sentiment analysis dashboard using NLP and Power BI

This helped turn raw numbers into a clear emotional narrative.

Key Insights from the Analysis

The story appears mostly positive overall (~56%)
However, it contains frequent sharp emotional drops
These drops align with moments of tension and psychological intensity

This shows that even if a story seems balanced overall, emotional spikes reveal its most impactful moments.

What I Learned

How to apply sentiment analysis using NLP in a real project
How to transform text into structured datasets
How to visualize insights using Power BI
The importance of storytelling in data analysis

Conclusion

This project demonstrates how sentiment analysis using NLP can go beyond traditional use cases and be applied to creative domains like literature.

By combining NLP with visualization tools, we can better understand how emotions evolve in any text.

Project Links

GitHub Repository: https://github.com/Fadydesoky/ai-story-sentiment-analysis

LinkedIn: https://www.linkedin.com/in/fadydesokysaeedabdelaziz

If you're interested in sentiment analysis using NLP or building real-world data projects, feel free to explore the full project on GitHub or connect with me on LinkedIn.