Coder Legion - Recent posts tagged deeplearning

The New ML Perspective

Wed, 28 Oct 2026 00:13:09 +0000

Artificial intelligence seems like a very complex thing packaged in a simple box. It's this magic concept of creating intelligence from something that otherwise wouldn't have any intelligence at all. Giving machines, just a bunch of 1s and 0s, electrical impulses with no real knowledge, the ability to make informed, complex decisions just like we do.

But what if we've been looking at AI from the wrong perspective?

Today, we're talking about the new AI perspective. Which is:

Drum roll please…

AI, DL, ML engineers, and all the like are the new teachers.

On a random Tuesday, I realized that the very first people who thought of Artificial Intelligence weren't trying to create magic. They were just trying to take the idea of human intelligence, one of the most fascinating abilities we have, and give it to machines.

And if learning is the goal, then someone has to teach.

The Classroom

Think about what happens when we build a machine learning model, ranging from linear regression models to complex neural networks.

After we have the data and have cleaned it, the very first thing we do is create the "brain." A blank slate walking into a classroom on the first day of school. It doesn't understand house prices. It doesn't know what a cat looks like. It has no idea whether a passenger on the Titanic survived or not.

It simply has the potential to learn anything and everything.

It's the same thing with humans. When a teacher receives a student for the very first time, they see a blank slate that knows almost nothing about the subject this teacher is going to teach. And this teacher's goal is to make sure that by the end of the school year, the student is basically a master at this subject.

In this case, the data is the textbook.

The Teaching Process

Now our classroom has three things:

The engineer (the teacher)
The model (the student)
The data (the textbook)

As the engineer, our job is to feed the textbook to the student over and over again. The student makes mistakes, lots of them, learns from those mistakes, slowly discovers patterns hidden in the data, and eventually gets better at making predictions.

A good teacher doesn't just hand students a textbook and hope for the best. They organize the lessons, remove confusion, provide examples, and make sure the student learns the underlying concepts instead of just memorizing answers. So, the responsibility shifts back to the teacher. If the teacher fails to frame the knowledge in a way the student can understand, or lets the student learn shortcuts, then the teacher has failed at their objective.

Machine learning engineers do exactly the same thing. We can't have models with leakage, or models that can't capture anything from the data. So it's up to us to take the data and use the best techniques we know to make sure the model captures actual relationships and learns from it. In engineering terms, this means encoding data, removing unnecessary columns, engineering new features, resizing images, handling missing values, and countless other techniques that help the model focus on what actually matters.

If a student memorizes answers instead of understanding the material, we question the teaching. If a model memorizes the training data, suffers from leakage, or fails to generalize to new examples, we should question the training process the exact same way.

Exam Time

Every student eventually sits for a TEST. A teacher gauges how well a student performs by how well they do on it.

Now, you might ask, "Ruva, how does this equate to my Titanic Survival Rate Prediction model?" Well, the same way we have those stressful exams, models are tested on a small portion of unseen test data, data that's locked away and only brought out when it's time to test the model. One common way to do this is by using scikit-learn's train_test_split() function.

The model is then "graded" using evaluation metrics such as:

R² score
Mean squared error (MSE)
ROC AUC
Accuracy
Precision and recall

All of these different metrics let us evaluate the model's performance and figure out whether it performed well, is just guessing, or is as confused as it can possibly get.

A Different Way to See AI

Thinking about AI this way changed how I see machine learning.

We aren't creating intelligence out of thin air. We're creating students. And our responsibility isn't just to build models, it's to teach them well. Just like in a real classroom, the quality of the student depends on the quality of the teaching, the lessons, and the textbook.

Maybe AI was never about building intelligent machines. Maybe it's always been about learning how to teach them.

That's all for today. Next time, we'll talk about the magic behind Artificial Intelligence, and how AI is just the child of math and programming.

See you next time!

Machine Learning VS Deep Learning: What’s the Difference?

Fri, 23 Oct 2026 22:07:20 +0000

Artificial Intelligence (AI) is the new thing on the street. Everyone is talking about it, and everyone wants to be part of it in one way or another. From chatbots that can hold conversations to self-driving cars and systems that can help detect diseases, AI is becoming increasingly integrated into our everyday lives.

But have you ever stopped to ask yourself: What actually makes Artificial Intelligence intelligent?

I like to think that the answer to this question lies in its subfields.

Artificial Intelligence is a broad field of computer science focused on building systems that can perform tasks that typically require human intelligence. Like deciding whether or not that apple is still safe to eat or if it has gone bad. These tasks include reasoning, problem-solving, learning, and understanding language. Yep, it's really cool.

Within this broad field, we have several related areas, including Natural Language Processing (NLP), Computer Vision, Robotics, and Expert Systems, among others. Each of these explores different approaches to building intelligent systems.

Today, however, we're focusing on two terms that are often used interchangeably, sometimes even by people working in tech: Machine Learning (ML) and Deep Learning (DL).

Deep Learning took off in the 2010s because three things came together: much more data, powerful GPUs, and better training techniques. A big moment was 2012, when a deep neural network called AlexNet won the ImageNet image-recognition competition by a wide margin. After that, Deep Learning kept improving results in speech, language, and image-related tasks. This is why Deep Learning is such a huge topic in the tech space.

Are they the same thing? If not, what makes them different? And why does everyone seem to be talking about Deep Learning?

Let's break it down.

First, we need to define Machine Learning and Deep Learning.

What Is Machine Learning?

Machine Learning is a subfield of Artificial Intelligence that allows computers to learn patterns from data and use those patterns to make predictions or decisions without being explicitly programmed with fixed rules for every possible situation.

Basically, we're giving a computer a bunch of examples of, for example, houses, their details, and their prices, and allowing it to look for patterns in that data.

Like the example I mentioned, predicting house prices based on features such as location, size, and number of bedrooms.

Actually programming this, including all the rules, would be a lot of work. Imagine the number of conditions and scenarios you would have to account for just to produce a model that is somewhat accurate. It would be a lot of work, and that's one of the reasons ML exists.

Instead of manually writing every rule, we allow an algorithm to learn relationships from examples. The resulting model can then use what it has learned to make predictions about houses it hasn't encountered before.

Pretty cool, right?

What Is Deep Learning?

Deep Learning is a specialized branch of Machine Learning that uses artificial neural networks with multiple layers to learn increasingly complex patterns from data.

Each layer transforms its input into a representation that helps the network learn useful features. For example, in a Convolutional Neural Network (CNN) trained on images, earlier layers might learn to detect edges and simple textures, while deeper layers may learn to recognize more complex shapes and object parts.

It's called Deep Learning because of the multiple layers in these neural networks.

Deep Learning was loosely inspired, in part, by the structure and functioning of the human brain, though artificial neural networks are a hugely simplified version of the real thing.

HOW COOL IS THAT? Honestly, I need a second to appreciate Deep Learning.

Anyway, now that I've finished fangirling, let's get back to business.

Deep Learning uses these artificial neural networks to learn patterns from data, sometimes with very little manual feature engineering.

A good example is classifying fish species from photographs. Imagine having thousands of photographs of different fish species.

We want to build a model that can identify the species in a photograph it has never seen before.

With Deep Learning, we can train a neural network to learn useful visual features from the images and use them to classify the fish.

In practice, you'd usually start from a pretrained model rather than training from scratch.

Instead of manually telling the model which edges, colours, textures, or shapes to look for, the network can learn useful combinations of these features during training.

And that is one of the things that makes Deep Learning so interesting!

Of course, it doesn't mean we can simply throw data at a neural network and expect magic. We still need suitable data, a carefully designed model, training, and evaluation. The model's performance depends on several factors, including the quality of the data and how well the model is trained.

What's the Difference?

Now that we understand both concepts, let's talk about their differences.

First, remember this: Deep Learning is a type of Machine Learning. They aren't two completely separate fields competing against each other.

Think of Artificial Intelligence as a big umbrella. Under that umbrella, we have Machine Learning, and under Machine Learning, we have Deep Learning.

Yeah, we have an umbrella in an umbrella which is also in a bigger umbrella.

But before we dive in, we need to define feature engineering.

Feature engineering means choosing or creating the inputs a model learns from. For house prices, that could be turning year built into age of house, or calculating price per square metre.

Traditional Machine Learning often relies heavily on this step. Deep Learning can reduce the need for manual feature engineering when working with data like images, audio, and text. However, on tabular data, well-engineered features combined with simpler models can often still perform very well.

So, what actually makes them different?

A. How They Learn From Data

One major difference is how they learn useful features from data.

Traditional Machine Learning approaches often rely on people to select or engineer the features that a model uses.

For example, if we wanted to predict house prices, we might provide a model with the house's size, location, number of bedrooms, and age.

We decide which information is relevant and give the model that information to learn from.

The house-price example is called supervised learning, because the model learns from examples with known answers. There are other types, but we'll leave those for another post.

Deep Learning can go a step further by learning useful features directly from less-processed data.

Remember our fish classification example? Instead of manually extracting features from each photograph, a Deep Learning model can learn useful visual representations from the images themselves.

This doesn't mean traditional Machine Learning cannot process images, or that Deep Learning never needs feature engineering. The difference is in how much of the feature-learning process the model can handle.

B. Data Requirements

Another difference is the amount of data they typically benefit from.

Many traditional Machine Learning algorithms can perform well on relatively small datasets, particularly when the data is structured and the relevant features are available.

Deep Learning models often benefit from larger datasets because they may need many examples to learn useful representations and generalize well to new data.

However, this isn't a strict rule. The amount of data required depends on the problem, the model, and other factors.

Techniques such as transfer learning can also allow Deep Learning models to perform well when the available dataset is relatively small.

C. Computational Resources

Let's talk about computing power.

Traditional Machine Learning models, such as decision trees and logistic regression, can often be trained with relatively modest computing resources.

Deep Learning models can be much more computationally expensive, particularly when they have many parameters or are trained on large datasets.

This is one reason why GPUs are commonly used to train Deep Learning models. They can perform the large number of mathematical operations involved in neural network training efficiently.

This is because GPUs are good at running many calculations in parallel, especially the matrix mathematics neural networks rely on.

And yes, this can also mean longer training times and higher costs.

D. Examples of Algorithms

Machine Learning includes several approaches, such as:

Linear and logistic regression
Decision trees
Random forests
Support Vector Machines (SVMs)
Gradient Boosting, such as XGBoost

Deep Learning uses neural network architectures such as:

Convolutional Neural Networks (CNNs): Commonly used in image-related tasks.
Recurrent Neural Networks (RNNs): Designed to process sequential data.
Transformers: Widely used in modern language models and other applications.

There is a lot more to explore within these categories, but we'll save that for another day.

For now, here's a quick comparison:

Feature	Traditional Machine Learning	Deep Learning
Relationship	A subfield of AI	A subfield of ML
Feature extraction	Often involves manually selecting or engineering features	Can learn features directly from raw or minimally processed data
Data requirements	Can work well with smaller datasets	Often benefits from larger datasets
Computing power	Often requires fewer resources	Can require substantial computing resources
Interpretability	Some models are relatively easy to interpret, such as linear models and small decision trees	Complex neural networks can be difficult to interpret
Examples	Decision trees, random forests, logistic regression	CNNs, RNNs, Transformers

These are general tendencies rather than absolute rules. The best approach depends on the problem you're trying to solve.

Can Machine Learning Work Without Deep Learning?

YES. Absolutely!

Remember when I said that Deep Learning is a branch of Machine Learning? That means Machine Learning existed as a field of study long before modern Deep Learning became popular.

You can build useful Machine Learning systems without using a single neural network.

For example, imagine you're building a system to predict whether a customer is likely to leave a service.

You could use logistic regression, a decision tree, or a random forest to learn patterns from historical customer data and predict which customers are at risk of leaving.

You don't necessarily need a Deep Learning model for this.

In fact, using a complex neural network when a simpler model can solve the problem effectively may introduce unnecessary computational costs and complexity.

The goal isn't to use the most complicated algorithm available. The goal is to solve the problem effectively.

Deep Learning is powerful, but it isn't automatically the best choice for every Machine Learning task.

Sometimes, a simpler model is all you need.

When Should You Use Each?

So, how do you decide whether to use traditional Machine Learning or Deep Learning?

It depends on the problem, the data you have, the resources available to you, and the results you need.

Consider Traditional Machine Learning When:

You're working with structured data, such as spreadsheets or databases.
You have a relatively small dataset.
You need a model that is easier to interpret.
You have limited computational resources.
Simpler algorithms can achieve the performance you need.

Consider Deep Learning When:

You're working with complex data, such as images, audio, or natural language.
You have enough data or access to a suitable pretrained model.
The problem may benefit from learning complex representations.
You have the computational resources needed to train or run the model.
Traditional approaches aren't achieving the performance you need.

Again, these are guidelines, not strict rules. Traditional Machine Learning can work with images and text, while Deep Learning can work with structured data.

The important thing is to understand the problem before deciding which approach to use.

And please, don't use Deep Learning just because it sounds more impressive. Nobody gets bonus points for making a simple problem unnecessarily complicated!

Final Thoughts

Artificial Intelligence is a huge field, and there's so much to explore within it.

Machine Learning allows us to build systems that learn patterns from data, while Deep Learning takes this further by using multilayered neural networks to learn complex representations.

Neither is universally better than the other. They have different strengths, limitations, and use cases.

If you're just getting started with AI, my advice is to build a solid foundation in Machine Learning before diving deep into Deep Learning.

Understanding concepts such as training, testing, overfitting, loss functions, and model evaluation will make your Deep Learning journey much easier.

And remember, you don't have to use the most complex model to solve a problem well. You just need to understand your problem well enough to choose the right tool.

Now, go forth and build something cool.
See you next time!

Where do squared error and cross-entropy actually come from?

Wed, 14 Oct 2026 05:20:00 +0000

Short answer: maximum likelihood. Choose a distribution for the output, then:

Read its parameter as the network's output → output function
Take −log of it → error function

Gaussian (regression)

Output: ŷ = Wᵀh + b (the mean)
Error: −log N(y; ŷ, σ²) → squared error (constants drop out)

Bernoulli (binary classification)

Output: exponentiate + normalize → ŷ = σ(z)
Error: −log P(y) → cross-entropy = softplus((1 − 2y)z)

Why the pairing matters: with cross-entropy, the log cancels the sigmoid's exponential, so ∂J/∂z = σ(z) − y. With squared error, σ′(z) stays in the gradient and goes to ~0 when the model is confidently wrong, so learning stalls.

Full derivations: https://husseinmahdi.xyz/writing/series/math-behind-neural-networks/

Next up: softmax + cross-entropy for k classes.

Neural Networks: Inspired by the Brain, Built for Data

Wed, 23 Sep 2026 14:01:57 +0000

When I first started learning about neural networks, I found the idea both fascinating and slightly intimidating.

The phrase “neural network” makes it sound like something extremely complex, and honestly, once you start looking at layers, weights, activation functions, forward propagation, backpropagation, and optimization, it can become overwhelming pretty quickly.

But beneath all of that mathematics and code is a surprisingly simple idea:

A neural network learns patterns from data by adjusting itself based on its mistakes.

That idea is what made neural networks interesting to me.

As I continue exploring data science and machine learning, understanding neural networks has helped me see machine learning from a different perspective. Instead of simply telling a computer what rules to follow, we can give it data and allow it to learn patterns that may be difficult for us to define manually.

In this article, I'll break down where the inspiration for neural networks came from, what they are, and the major components that make them work.

Where Did Neural Networks Come From?

The inspiration behind artificial neural networks comes from something we already have:

the human brain.

Our brains contain billions of neurons that communicate with one another. A biological neuron receives signals, processes them, and passes a signal forward when certain conditions are met.

Artificial neural networks don't replicate the human brain perfectly. Instead, they borrow a simplified idea from how biological neurons work.

A biological neuron can be thought of roughly as:

receive signals → process them → produce a response

An artificial neuron follows a similar conceptual process:

receive inputs → calculate a weighted sum → apply an activation function → produce an output

This was one of the things I found interesting when learning about neural networks.

The goal isn't to recreate the brain inside a computer.

The goal is to take a useful idea from biology and turn it into a mathematical model that can learn from data.

So, What Exactly Is a Neural Network?

A neural network is a machine learning model made up of interconnected nodes, commonly called neurons.

These neurons are organized into layers.

A basic neural network usually contains:

Input layer
One or more hidden layers
Output layer

A simple representation looks like this:

Input Layer        Hidden Layer        Output Layer

   x₁  ─────────►    ○
                    │
   x₂  ─────────►    ○ ─────────►     ŷ
                    │
   x₃  ─────────►    ○

The input layer receives the data.

The hidden layers transform the data and learn patterns.

The output layer produces the final prediction.

The interesting part is that we don't manually program the network to recognize every possible pattern.

Instead, the network learns the parameters needed to recognize those patterns from examples.

1. The Input Layer

Everything starts with the input.

Suppose we want to predict whether a student is likely to pass an exam.

Our dataset might contain:

Hours studied
Attendance
Previous test scores
Assignment completion

These become our input features.

For example:

Hours Studied     = 6
Attendance        = 85%
Previous Score    = 72
Assignments       = 9

The neural network receives these values through the input layer.

One important thing I learned is that neural networks generally work best when the input data has been appropriately prepared.

That can involve:

Handling missing values
Encoding categorical variables
Scaling numerical features
Removing irrelevant features
Splitting data into training and testing sets

So even though neural networks are powerful, data preparation still matters.

Garbage in can still mean garbage out.

2. Neurons

The basic building block of a neural network is the artificial neuron.

A neuron takes several inputs, assigns different importance to them using weights, adds a bias, and then passes the result through an activation function.

Mathematically, we can represent this as:

\[ z = w_1x_1 + w_2x_2 + ... + w_nx_n + b \]

Where:

$x$ = input
$w$ = weight
$b$ = bias
$z$ = weighted sum

The neuron then applies an activation function:

\[ a = f(z) \]

The result becomes the neuron's output.

This might look complicated initially, but the basic idea is straightforward:

The neuron combines information from its inputs and decides how strongly to activate.

3. Weights

Weights determine how important different inputs are to a neuron.

Imagine we're predicting whether a student will pass.

Perhaps hours studied is more predictive than the number of assignments completed.

The network can learn this by assigning different weights to those features.

For example:

Hours studied       → weight = 0.8
Attendance          → weight = 0.5
Previous score      → weight = 0.9
Assignments         → weight = 0.3

These numbers are only examples.

During training, the neural network continuously adjusts its weights to improve its predictions.

This is one of the key ideas behind learning in neural networks.

The model isn't explicitly told what the correct weights are. It learns them from data.

4. Bias

Another component is the bias.

Bias gives the neuron some flexibility in determining when it should activate.

Without going too deep into the mathematics, you can think of the bias as an additional parameter that allows the activation function to shift.

The basic equation becomes:

\[ z = wx + b \]

The weights control the influence of the inputs, while the bias helps adjust the overall output.

Both weights and biases are learned during training.

5. Activation Functions

This was one of the concepts that really helped me understand why neural networks can learn complex patterns.

After calculating the weighted sum, a neuron applies an activation function.

Why?

Because without activation functions, stacking multiple layers of neurons would still result in a fundamentally linear transformation.

Activation functions introduce non-linearity.

And real-world data is rarely perfectly linear.

Some commonly used activation functions include:

Sigmoid

The sigmoid function produces values between 0 and 1.

\[ \sigma(x) = \frac{1}{1 + e^{-x}} \]

This makes it useful in certain binary classification problems, particularly as an output activation.

For example:

0.91 → high probability of class 1
0.12 → low probability of class 1

ReLU

ReLU stands for Rectified Linear Unit.

It is defined as:

\[ ReLU(x) = \max(0,x) \]

So:

x = -3 → 0
x =  2 → 2
x =  7 → 7

ReLU became particularly important in deep learning because it is simple and computationally efficient and helps neural networks model nonlinear relationships.

Softmax

Softmax is commonly used in the output layer for multi-class classification.

Suppose we're classifying an image as:

Cat
Dog
Bird

Softmax can convert the model's outputs into probabilities that sum to 1.

For example:

Cat  → 0.10
Dog  → 0.82
Bird → 0.08

The model would predict Dog.

6. Hidden Layers

Now we get to one of the most important parts of a neural network.

The hidden layers.

These layers sit between the input and output layers.

A network with only a few layers might look like:

Input → Hidden → Output

A deeper network could look like:

Input
  ↓
Hidden Layer 1
  ↓
Hidden Layer 2
  ↓
Hidden Layer 3
  ↓
Hidden Layer 4
  ↓
Output

Each layer can learn different representations of the data.

This is where the term deep learning comes from.

A neural network with many hidden layers is generally described as a deep neural network.

The interesting part is that the network can learn increasingly complex representations as information moves through the layers.

For example, in an image recognition task:

Pixels
  ↓
Edges
  ↓
Shapes
  ↓
Patterns
  ↓
Objects

The network isn't necessarily programmed with these exact rules.

It learns useful representations from the training data.

7. Forward Propagation

So how does a neural network actually make a prediction?

This happens through forward propagation.

The input data moves through the network from the input layer toward the output layer.

At each neuron:

Inputs are multiplied by weights.
The results are added together.
A bias is added.
An activation function is applied.
The result is passed to the next layer.

Eventually, the network produces a prediction.

Conceptually:

Input
  ↓
Weighted Sum
  ↓
Activation
  ↓
Next Layer
  ↓
Weighted Sum
  ↓
Activation
  ↓
Prediction

This happens extremely quickly when implemented computationally.

8. Loss Functions

The network makes a prediction.

But how does it know whether the prediction is good or bad?

This is where the loss function comes in.

A loss function measures the difference between the model's prediction and the actual target.

For example:

Actual value      = 1
Predicted value   = 0.3

There is a significant difference.

The loss function quantifies that error.

Different problems use different loss functions.

Some common examples include:

Mean Squared Error (MSE)
Binary Cross-Entropy
Categorical Cross-Entropy

The general goal during training is simple:

Minimize the loss.

9. Backpropagation

This is probably one of the most important concepts in neural networks.

After making a prediction and calculating the loss, the network needs to figure out:

Which weights contributed to the error, and how should they change?

This is where backpropagation comes in.

The error is propagated backward through the network.

Using calculus—particularly the chain rule—the network calculates gradients that indicate how changes in the parameters would affect the loss.

Conceptually:

Prediction
    ↓
Calculate Loss
    ↓
Propagate Error Backward
    ↓
Calculate Gradients
    ↓
Update Weights

Then the process repeats.

Again.

And again.

This is how the neural network gradually improves.

10. Optimizers

Backpropagation tells us how the parameters should change.

The optimizer determines how those changes are actually made.

One of the most well-known optimization algorithms is Gradient Descent.

The basic idea is to move the model's parameters in a direction that reduces the loss.

A simplified update rule looks like:

\[ w_{new} = w_{old} - \eta \frac{\partial L}{\partial w} \]

Where:

$w$ = weight
$\eta$ = learning rate
$L$ = loss

The learning rate controls how large each update should be.

If the learning rate is too large, the model may overshoot useful solutions.

If it is too small, training may take a very long time.

Other optimizers include:

Adam
RMSprop
Adagrad
SGD with momentum

Adam is particularly common in practical deep learning applications.

11. Epochs and Batches

Neural networks don't normally learn from the entire dataset in a single step.

Instead, training data is often divided into smaller groups called batches.

For example, if we have:

10,000 training samples

we might use:

Batch size = 32

The network processes 32 samples at a time.

Once it has processed the entire training dataset, we have completed one epoch.

So:

Dataset
   ↓
Batch 1
Batch 2
Batch 3
...
Batch N
   ↓
One Epoch

The model typically trains for multiple epochs.

12. Learning Rate

The learning rate determines how aggressively the model updates its parameters.

Think of it as the size of the steps the model takes while trying to minimize its loss.

A very large learning rate might look like:

Too large
    ↓
Overshoot
    ↓
Unstable training

A very small learning rate might look like:

Too small
    ↓
Tiny updates
    ↓
Very slow learning

Finding an appropriate learning rate can make a significant difference in training performance.

13. Training, Validation, and Testing

Another important concept is understanding how we evaluate a neural network.

We generally don't want to train and evaluate the model on exactly the same data.

Instead, data can be divided into:

Dataset
   │
   ├── Training Data
   │
   ├── Validation Data
   │
   └── Test Data

Training data

Used to learn the model's parameters.

Validation data

Used to monitor performance and help with choices such as hyperparameters and model architecture.

Test data

Used to evaluate how well the final model performs on unseen data.

This is important because a model can perform extremely well on its training data but poorly on new data.

That brings us to one of the biggest challenges in machine learning.

14. Overfitting

A neural network can become too good at memorizing the training data.

This is known as overfitting.

Imagine studying for an exam by memorizing the answers to last year's exact questions.

You might perform extremely well if the same questions appear.

But if the questions change, your performance may drop.

A similar thing can happen with machine learning models.

The model learns the training examples extremely well but fails to generalize to unseen data.

Some techniques used to reduce overfitting include:

Dropout
Regularization
Early stopping
Data augmentation
More training data
Simpler model architectures

Why Are Neural Networks So Powerful?

What makes neural networks particularly interesting is their ability to learn nonlinear relationships and complex representations.

Traditional programming often looks like:

Rules + Data → Output

Machine learning changes the approach:

Data + Expected Outputs → Learned Rules

And neural networks take this further by learning multiple levels of representation.

This is why they have become so useful in areas such as:

Computer vision
Natural language processing
Speech recognition
Recommendation systems
Time-series forecasting
Fraud detection
Medical imaging
Generative AI

From Neural Networks to Deep Learning

A neural network with many layers is generally referred to as a deep neural network.

Deep learning is essentially about using these deeper architectures to learn increasingly complex representations from data.

This has led to architectures designed for different kinds of problems.

For example:

Convolutional Neural Networks (CNNs)

Often associated with image and computer vision tasks.

Recurrent Neural Networks (RNNs)

Designed to work with sequential data, although many modern sequence tasks now use transformer-based architectures instead.

Transformers

Transformers have become extremely important in modern AI, particularly for language and other sequence-based problems.

They power many of the systems behind today's generative AI applications.

What I Take Away From Learning Neural Networks

One thing I appreciate about neural networks is that they changed how I think about machine learning.

At first, the equations can make the subject feel intimidating.

You see:

\[ z = wx+b \]

then activation functions, gradients, derivatives, optimization algorithms, layers, epochs, and suddenly it feels like you're drowning in mathematics.

But when you break the process down, the basic idea becomes much easier to understand.

A neural network:

takes data → makes a prediction → measures its error → adjusts itself → tries again.

That cycle is at the heart of learning.

And I think that's the part that makes neural networks so fascinating.

Final Thoughts

I'm still learning neural networks, and I don't think understanding them means memorizing every equation or being able to build a state-of-the-art architecture from scratch.

For me, understanding the fundamentals is more important.

I want to know:

What is happening to the data?
Why do we need weights?
What does a bias actually do?
Why do we need activation functions?
How does the model know that it made a mistake?
How does backpropagation help?
What does an optimizer actually change?
How do we know whether the model is learning or simply memorizing?

Once these pieces start fitting together, neural networks stop looking like a mysterious black box.

They become a collection of understandable ideas working together.

And that's probably the most important lesson I've taken from studying them so far:

Complex models become much less intimidating when you understand the simple ideas underneath them.

I'm still on that learning journey and neural networks are definitely one of the areas I'm excited to explore more deeply.

Predicting Drug-Drug Interactions with Hypergraphs — on Two CPU Cores

Sun, 13 Sep 2026 18:32:27 +0000

When a patient takes several medications at once, two of them can interact in ways that cause real harm. DrugBank catalogues 86 distinct mechanisms for how this happens — one drug slowing another's metabolism, increasing its toxicity, blunting its effect, and so on. Screening every pair by hand does not scale.

Most machine learning approaches to this problem split it in two: one model decides whether two drugs interact, a completely separate one decides how. That split is convenient for benchmarking and awkward for anything practical.

We built H²GNN to do both in one hierarchy, using nothing but SMILES strings as input. The code is now public, and the whole thing runs on two CPU cores.

Code: github.com/Hu8MA/HHGNN
Paper: ECAI 2026, IEEE Xplore

Why hypergraphs

In an ordinary graph, an edge joins exactly two nodes. That is a poor fit for a lot of chemistry. A molecular substructure is not a relationship between two drugs — it is something that dozens of drugs share simultaneously. An interaction mechanism is not a property of one pair; it is a category that hundreds of pairs fall into.

A hyperedge can connect any number of nodes at once, which lines up with both of those facts. H²GNN uses two hypergraphs stacked on top of each other.

Stage 1: the chemical hypergraph

Take a SMILES string and slide a window of size k across it. Each window is a substring — a rough proxy for a local chemical arrangement. Every distinct substring becomes a node, and every drug becomes a hyperedge connecting the substrings it contains.

At k = 9, that gives 29,462 substructure nodes, 1,709 drug hyperedges, and 96,656 connections between them.

A single hypergraph attention layer runs over this structure. Attention flows at the hyperedge level, then the node level, and the result is a 128-dimensional embedding per drug learned end to end from raw SMILES. An MLP decoder takes the embeddings of two drugs and predicts whether they interact.

Result: 98.46% ROC-AUC, 93.20% accuracy.

For comparison, a Random Forest on 4,096-dimensional Morgan fingerprints gets 88% ROC-AUC. A two-layer GCN gets 95%.

Stage 2: the interaction-type hypergraph

Here is the part I find most interesting, and it is a small conceptual shift with a large effect.

The obvious way to classify interaction mechanisms is to attach a label to each drug pair and train a classifier. We did something else: each of the 86 interaction types becomes a hyperedge, connecting every drug that participates in that type. Drugs are the nodes. The type stops being a label on the data and becomes an object in the graph with its own learned representation.

This hypergraph is tiny — 1,709 drug nodes, 86 type hyperedges, 13,486 connections. Hyperedge sizes range wildly: type 42 touches 5 drugs, type 49 touches 1,308.

Drug nodes are initialized with the embeddings transferred from Stage 1, not randomly. Attention then flows hyperedge → node → hyperedge, so the type hyperedges — which are the prediction target — end up holding the final aggregated representation.

Result: 85.73% Top-1 and 98.11% Top-3 accuracy across 86 classes.

That twelve-point gap between Top-1 and Top-3 is itself useful. The correct mechanism is almost always among the top three even when it is not ranked first, which is exactly what you want if a clinician is reading the output rather than a downstream script.

The ablations that mattered

Two experiments did more to convince me the design was right than any headline number.

Attention direction. Stage 2's drug nodes start out well-informed, carrying transferred chemical knowledge, while the type hyperedges start from one-hot vectors. So it seemed plausible that flowing attention from the informed side — node → hyperedge → node — would work better.

It does not. It falls apart:

Configuration	Top-1 Acc.	PR-AUC	Macro F1
hyperedge → node → hyperedge	85.73%	87.24%	81%
node → hyperedge → node	67.42%	63.54%	48%

Aligning the attention flow with the prediction target beats starting from the richer initialization, and the rare classes are the first thing you lose. The type hyperedge has to be where aggregation ends.

Embedding transfer. Replace the transferred Stage 1 embeddings with an all-ones vector and hold everything else fixed:

Initialization	Top-1 Acc.	PR-AUC	Weighted F1
Stage 1 embeddings	85.73%	87.24%	86%
All-ones vector	82.52%	85.70%	82%

A 3.21-point drop. The hierarchy earns its keep.

There is also a third result I did not expect. During cleaning, 406 drug pairs turned out to carry more than one interaction type; we kept the first and set the others aside. The model never saw those secondary labels for those pairs. It surfaced 94.7% of them within its Top-3 predictions anyway — meaning the hypergraph learned something about multi-mechanistic pharmacology rather than memorizing one label per pair.

The efficiency story

This is the part that surprised me most, and it is worth spelling out for anyone who assumes graph learning implies a GPU budget.

Everything above was trained and evaluated on Google Colab's free tier: two vCPUs, no GPU. We chose it deliberately, for two reasons. It gives every run an identical environment, and its resource ceiling is close to the hardware actually available in a hospital pharmacy.

Model	Training time	Hardware
Interaction-Type HGNN (ours, 1 layer)	18.6 min	CPU, 2 cores
GCN (2 layers)	75.6 min	GPU (T4)
GraphSAGE (2 layers)	148.5 min	GPU (T4)
XGBoost	225.6 min	CPU

Four to twelve times faster — and it still beat GraphSAGE, the strongest baseline, by 4.8 points of Top-1 accuracy and 22 points of macro F1.

The baselines had advantages we did not give ourselves: precomputed Morgan fingerprints as input features, and inverse-frequency class weighting during training to stay stable under the severe class imbalance. Single-layer variants of the GCN and GraphSAGE did not converge at all; both had to be deepened and moved to GPU for the comparison to run. A Graph Attention Network never converged and was dropped.

Peak RAM increase during training: 0.29 GB.

What is in the repo

The dataset is DrugBank — 1,709 drugs, 191,877 unique interaction pairs, 86 types, built by merging two benchmark sources so that every drug has a SMILES string.

Everything needed to reproduce the numbers is there:

Colab notebooks for all 48 experiments, with inline outputs
Pre-computed k-mer files for k = 3, 6, 9, 12
Hypergraph structure files and index maps
The exact seeded train/validation/test partitions, verified for zero leakage
Baseline notebooks (GCN, GAT, GraphSAGE, XGBoost, Random Forest)

Open a notebook, point it at the data files, run all cells. Free tier is enough.

What it does not do

Being honest about the boundaries, since these are the interesting problems:

SMILES only. No protein-target, pathway, or side-effect information. Chemical structure carries the entire representation. Additional modalities could enter as further hyperedge families.
No severity prediction. The model reports the mechanism and its likelihood, not how bad the reaction will be — which is often the thing that actually drives a clinical decision.
1,709 drugs. DrugBank now holds over 4,000, so plenty of approved drugs cannot yet be screened.
Pairs only. Real polypharmacy is not pairwise. The hypergraph formulation could express simultaneous multi-drug combinations directly; we just have not done it.

That last one bothers me the most, because the machinery is already there. A hyperedge over three co-administered drugs is a perfectly natural object in this framework.

If any of these interest you, contributions are welcome — there is a contributing guide in the repo, and the issue tracker is open.

Code: github.com/Hu8MA/HHGNN
Paper: Hierarchical Hypergraph Neural Networks for Multi-Type Drug–Drug Interaction Prediction, ECAI 2026, IEEE Xplore

Thanks to the DrugBank Foundation for access to the curated drug data that made the dataset complete.

The Math Behind Neural Networks, Explained Like Nobody Did for Me — Part 1.4

Mon, 07 Sep 2026 17:57:26 +0000

Part 1.4 — Which Loss to Choose, and Why. A calm, sequential guide for anyone entering the field. The article is divided into smaller chapters to make it easier to understand.

Part 1.3 ended on a quiet victory. The mean squared error, which we had simply assumed back in Part 1.1 and optimized in Part 1.2, was no longer an assumption. We had derived it — from a single principle, maximum likelihood, and a single modelling choice, that the model’s output is a Gaussian centred on its prediction. The loss we had trusted for two parts was finally standing on a foundation we could see through. The recipe was built: choose the output distribution, and the cost follows.

But a recipe that produces one dish invites an obvious question about all the others. We chose the Gaussian, and squared error came out. That was a choice — the introduction to Chapter 9 said as much and then set the point aside. This part picks it back up. If a different output distribution is chosen, a different cost comes out, and it predicts a different summary of the data. Squared error was never the only item on the menu; it was the first one we happened to order.

There is also a problem hiding inside the loss we derived, one that Part 1.3 had no reason to expose. In the very common case of a yes-or-no prediction, squared error can go quiet at exactly the wrong moment — its gradient falling nearly to zero while the model is still confidently wrong, leaving gradient descent with nothing to follow. The fix is not to patch squared error but to derive a different cost, cross-entropy, from a different distribution, and watch the problem dissolve on its own. This is where the slope idea from Part 1 returns for the last time, and where the choice of loss and the choice of output finally reveal themselves as one decision, not two.

So this part is about reading the menu that Part 1.3 built. We look at what other distributions give, why the loss you pick quietly decides which statistic your model chases, and why practitioners so often reach past the squared error we worked so hard to justify.

Chapter 10: When the Gradient Lies
We built the general recipe, and we chose the Gaussian, and squared error came out. So we could stop. But the practitioners’ literature contains a quiet recommendation that seems, at first, to contradict everything we just did: for many problems, do not use squared error. Use cross-entropy instead. This chapter explains why, and the explanation reaches all the way back to the slope idea from Part 1.

What Is Actually Being Compared
Putting squared error next to cross-entropy can look like a category mistake, so it is worth being clear about what is being compared. They are the same kind of object — both are cost functions, both are numbers measuring how wrong the model is, both are handed to us by the recipe once we pick an output distribution (Gaussian gives squared error; the binary distribution we will meet in Part 2 gives cross-entropy). So comparing them is fair. The reason they come into conflict is not the costs themselves. It is what sits in front of them.

The conflict appears in a specific and very common situation: a binary classification task, where the model must output a yes-or-no probability. To produce a number between 0 and 1, the output is almost always passed through a sigmoid first. And it is the sigmoid, wedged between the network and the cost, that causes the trouble. As we will see, the sigmoid has flat regions, and when squared error is stacked on top of a sigmoid, those flat regions leak into the gradient and flatten it too. During back-propagation — the backward pass we built in Part 1.2 and will extend across layers later — a flat gradient means the update is negligibly small, so the model barely changes even when it is badly wrong. Cross-entropy is the cost whose shape cancels that flattening. That is the whole subject of this chapter: not “which cost is better in the abstract,” but “which cost survives contact with the sigmoid.”

One note before the mechanics. Below, a term written σ′(z) will appear — the slope of the sigmoid. Do not worry about where it comes from in detail here; it is simply the sigmoid’s own steepness at a point, and it enters through the chain rule exactly as the slope did in Part 1.2. We are using it only to see the effect. The full story of the sigmoid, its slope, and the binary (Bernoulli) distribution that makes cross-entropy the derived cost for this task belongs to Part 2.1, where output units are the main subject. Here we only need to watch what σ′(z) does to the gradient.

Two Reasons a Gradient Can Be Small
Recall from Part 1.2 that training lives or dies by the gradient. If the slope of the loss goes flat, gradient descent has nothing to follow and learning stalls, like standing on a flat plateau in fog with no sense of which way is downhill.

Now here is the subtlety. A gradient can go small for two completely different reasons, and only one of them is benign.

The first is the good kind. In your worked example in Part 1.2, as w approached its true value of 2, the errors shrank and the gradient shrank with them. A small gradient there means you have essentially arrived — take smaller steps. Nothing is wrong.

The second is the disaster. The gradient can go nearly to zero while the model is still badly wrong. When this happens, the small gradient is lying: it whispers “nothing to fix here” at exactly the moment the model is confidently mistaken and most needs a shove. This second case has a name. It is called saturation, and avoiding it is the reason cross-entropy is preferred.

Where Saturation Comes From, With Numbers
To see saturation appear, we need an output that can be confidently wrong. Binary outputs are the clean example, and this is exactly the setting flagged above: the model must answer yes-or-no, so it produces a probability ....... .

To read the full article, please visit the link : https://husseinmahdi.xyz/writing/math-behind-nn-1-4/

or this : https://blog.gopenai.com/the-math-behind-neural-networks-explained-like-nobody-did-for-me-87f299008ffb

Refrences
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press. Relevant chapter: Chapter 6 (Deep Feedforward Networks), Section 6.2.1 — Cost Functions. Available free at deeplearningbook.org.

How AI Models Actually Learn, From Random Weights To Reinforcement

Sun, 23 Aug 2026 13:48:59 +0000

Most of us here ship code against model APIs every day without ever having trained one, and that gap shows up the moment a model does something the docs do not explain. This is the mental model I wish I had earlier, written for developers rather than for a statistics class.

Learning Is Just Adjusting Numbers

Every model is a function. It takes an input and returns an output, and in between sit numbers that nobody chose by hand.

Take the smallest possible version: price = weight * square_footage + bias. Two unknowns. You show the function a real sale, measure how far off it was, and move the weight and the bias slightly in the direction that would have reduced the error. Do that a few thousand times and you have something that predicts house prices.

Nothing about that changes at scale. GPT-4 is estimated to hold over a trillion of those numbers, and every one of them was set by the same loop: guess, measure the error, adjust. The architecture decides how the numbers are wired together. Training decides what they end up being.

Three Paradigms, One Question

Supervised, unsupervised and reinforcement learning get taught as three separate topics, but they are really three answers to one question: how does the system find out it was wrong?

Supervised learning gets told. Every input arrives with the correct answer attached, so the error is available immediately. It is the most reliable of the three and it sits under image classifiers, spam filters and translation. The cost is the labels, because a person has to produce them.

Unsupervised learning is never told. There is no correct answer, so the system optimizes for structure instead, grouping things that behave alike or compressing data without losing much. Clustering and embeddings live here.

Reinforcement learning finds out later. The system acts, and a score arrives after the fact, sometimes long after. That delay is the entire difficulty, because when the score comes back bad you still have to work out which of the earlier decisions caused it. The alignment step that turned raw language models into assistants is this paradigm.

The full guide to how AI learns walks each of the three through a worked example, along with the training loop step by step.

Why The Data Sets The Ceiling

A model generalizes only inside the boundaries of what it saw. That sounds obvious, and it is still the most common reason a system tests well and then falls apart in production.

A child who touches a hot stove once learns to be careful around hot things in general. A model needs thousands of examples to find the same pattern, and it will not extend that pattern to a situation its training data never covered. If stoves were never in the data, there is no concept of a stove to reason from.

For anyone building on top of these systems, that is the practical part: when output quality is bad, the fix is usually in the data distribution rather than in the architecture or the prompt.

What Training Cannot Fix

Training is statistical, so it inherits the limits of statistics. A model can tell you what tends to follow from what. It cannot tell you what causes what, unless the causal structure happened to be visible in the correlations.

It also has no way to know what it does not know. Confidence in these systems is a measure of how well an output matches patterns in the training data, which is why a confident wrong answer looks identical, from the inside, to a confident right one.

Working out which of the three paradigms produced the behavior in front of you usually tells you whether you have a data problem, a reward problem, or a limit of the method itself. That single distinction has saved me more debugging time than any prompting technique.

The Math Behind Neural Networks, Part 1.3 — Where the Loss Function Comes From

Tue, 18 Aug 2026 19:57:05 +0000

A calm, sequential guide for anyone entering the field. The article is divided into smaller chapters to make it easier to understand.

Part 1.2 left us with a working method but an unexamined assumption. We had a loss, the mean squared error, and we had turned it into a training procedure through the derivative and gradient descent. We watched the loss fall from 18.67 to 0.297 in a single step and then converge to the true relationship in the data. Everything worked. But we never asked where the mean squared error itself came from. We wrote it down as a reasonable way to measure error — square the gap, average it — and moved on. Because it worked, the question never surfaced.

Look again at how it entered. In Part 1.2, the model gave us a form for the line, ŷ = wx + b, but no way to judge whether a particular guess of w and b was any good. The equation described the model and contained no built-in measure of its quality. Faced with that gap, the article said, in as many words: “So we invented one.” We compared the prediction to the truth across all the data and summarized the gap into a single number. Squaring came along because it was reasonable — it makes every error positive and punishes large misses — not because anything forced it.

Chapter 7: What a Cost Function Really Measures

A Note on Three Names
At the end of Part 1.2 we had a complete loop: predict, measure the loss, compute the gradient, take a step. The loss at the centre of that loop was

A quick word on names first, because this article introduces a third one and I do not want it to look like a third thing. In Part 1.1 we called this the error function. In Part 1.2 we called it the loss. In this part, and in most textbooks, you will also see it called the cost function, written J(θ). These are not three different objects. They are three names for the same idea — a single number that measures how wrong the model is and that training tries to make small. “Cost function” is simply the more general term the literature settles on once the measure is no longer tied to one particular formula, which is exactly the situation this article is about to put us in. Wherever you read “cost” below, you may read “error” or “loss” and lose nothing.

The Assumption We Never Examined
With the naming settled: as the introduction recalled, this measure entered the story as an invention — the reasonable thing we reached for when the model gave us no measure of its own quality. Squaring makes every error positive, so overshooting and undershooting both count as wrong, and it punishes a large miss far more than a small one. These are good properties, and they were enough to justify the formula for a first encounter.

But “good properties” is not the same as “forced.” An invention fills a gap; it does not prove it was the only thing that could have filled it. A reader who accepts squared error should be able to answer the harder question that the invention quietly left open: out of all the ways to measure a gap, why this one? Some of that question we have in fact already answered. In Part 1.1 we asked the most natural competing question — why not the absolute gap |y − ŷ|, which also removes the sign — and we ruled it out for a concrete reason: the absolute value has a sharp corner at zero where its slope jumps from −1 to +1 with no value in between, and gradient descent, which lives or dies by reading a clean slope, receives contradictory signals at exactly the point it is trying to settle into. So the absolute value is not an open door; Part 1.1 already closed it, on grounds of the slope.

What Part 1.1 did not close is the deeper question. Ruling out the absolute value tells us squaring is better-behaved than one rival. It does not tell us that squaring is right — that out of every smooth, well-behaved measure we could have invented, this is the one the problem actually calls for. For that we need to stop justifying squared error from the outside and find the principle that produces it from the inside. This chapter goes looking for that principle.

From a Point Guess to a Distribution
In Part 1.2, the model produced a single number: ŷ = wx + b. Given an input, it committed to one answer. This is the natural picture, and it is also the picture that hides the principle we are looking for.

The more honest picture is this. A model is uncertain. Given an input x, it does not truly believe the answer is exactly one number. It believes the answer is probably near some number, with smaller and smaller chances the further you move away. In other words, the model does not output a point. It outputs a distribution — a spread of belief over all the answers that could occur. We write this distribution as

This part is about where that invention actually came from. Why squared error, and not the plain distance, or the cube? The honest answer is that the mean squared error is not a starting point at all. It is a consequence. It is what you are forced to when you begin from a deeper and more general principle about what it means for a model to be good. In this part we start from that principle, watch it produce a single general recipe for building cost functions, and then watch the mean squared error fall out of that recipe on its own.

Readers of Part 1.1 were promised this. Its closing note mentioned, almost in passing, that minimizing the mean squared error is equivalent to maximum likelihood estimation under the assumption of Gaussian noise. That sentence was a debt. This part pays it: everything below is the unpacking of that one line. Part 1.4 then asks the next question — whether the loss we just derived is the one we should actually use.
....
....

Read the full article at the official link: https://husseinmahdi.xyz/writing/math-behind-nn-1-3/

The Math Behind Neural Networks, Explained Like Nobody Did for Me - Part 1.2

Mon, 27 Jul 2026 18:20:40 +0000

A calm, sequential guide for anyone entering the field, The article will also be divided into smaller chapters to make it easier to understand.

Part 1.2: The Mathematics of Learning

Part 1.1 ended at a precise standing point: we had built a model, ŷ = wx + b, and we had a single number, the mean squared error, that told us how wrong that model was on average. What we did not have was a way to act on that number. A loss of 4.67 tells you the model is wrong; it does not tell you whether to push w up or down, and it does not tell you by how much. Part 1.2 is where that missing piece arrives.

We will introduce the derivative, derive the gradient of the loss with respect to w and b by hand, and then turn those gradients into an actual update rule through gradient descent. By the end of this part, the same three data points from Chapter 2 will be used in a full training example, and you will watch the loss fall step by step until the model converges to the true underlying relationship. The slope idea from Chapter 1 does not change. It simply gets sharpened into the tool that makes learning possible.

Chapter 4: The Derivative — The Slope of the Error

Why a Number Alone Is Not Enough
At the end of Chapter 3 we had a single number, L, that summarized how wrong the model was. With w = 1 and b = 0, the loss was 4.67. We know the goal is to make this number as small as possible. What we do not yet know is how.

A reader might suggest trying different values of w and b until the loss becomes small. With two parameters and a simple dataset, this is not impossible, but consider what it involves. Should w be increased or decreased? By how much? Should b be changed at the same time or held steady? A real neural network has millions of parameters, not two, and trying random combinations would take longer than the age of the universe.

The deeper issue is that the loss value, taken on its own, does not contain a direction. The number 4.67 tells you that you are wrong, but it does not tell you which way is less wrong. If I handed you only the number 4.67 and asked you whether to increase or decrease w, you could not answer. You would have to guess. The loss is a position, not a direction. These are two fundamentally different kinds of information, and you cannot extract one from the other by rearranging the same data. You have to ask a new question.

The Equations Have Been Building Toward This
To see why the derivative is the only thing that fits here, it helps to trace the logic of every equation we have written so far. Each one was forced into existence by a problem the previous one could not solve.

We began with the slope formula:

This worked perfectly when the data lay on a clean line, because the slope could be read off directly from any two points. But real data scatters, and the slope formula stopped applying. Different pairs of points gave different slopes, and none of them represented the whole dataset.

So we wrote a candidate line with unknowns to be learned:

This solved the previous problem by giving us a form for the line, but it created a new one. The values of w and b were unknown, and we had no way to evaluate whether a particular guess was good. The equation describes the model but contains no built-in measure of its quality.

So we invented one. We compared the prediction ŷ to the truth y across all data points and summarized the gap into a single number:

This solved the evaluation problem by giving us one number that measures total error. But it created the deepest problem of all, the one we are now standing at. The loss tells us that we are wrong but not which way to change w or b to be less wrong. The information we need is not contained in L. It is contained in how L changes when w changes, which is a different quantity entirely.

To extract that quantity, we need a new equation, one that asks the loss a question the loss formula on its own never asks: if w were slightly different, by how much and in what direction would L change ? The mathematical object that answers this question is the derivative.

From Slope to Derivative

We already have the tool we need, in a simpler form. In Chapter 1, slope told us how y changes when x changes. The derivative is the same idea applied to a different setting. Instead of asking how y changes with x in the data, we now ask how the loss L changes with the parameters w and b in the model. The variables have changed, but the underlying question — how does one thing change as another changes? — is identical.

There is one refinement. The slope formula in Chapter 1 measured the rate of change between two distinct points. The derivative measures the instantaneous rate of change at a single point, which is what you get when those two points move infinitely close together:

This is not a different concept from slope. It is slope made precise. As the gap between the two points shrinks to zero, the slope of the line connecting them becomes the slope of the curve at that exact point.

Why the Derivative and Nothing Else
It is worth addressing directly why the derivative is the right tool here, rather than some other technique. The honest answer is that the derivative is not the only conceivable approach, but it is the only approach ....... .

To complete the article, you can visit the blogs and view the entire series on : https://husseinmahdi.xyz/writing/series/math-behind-neural-networks/

The Math Behind Neural Networks, Explained Like Nobody Did for Me

Sat, 25 Jul 2026 00:02:30 +0000

A calm, sequential guide for anyone entering the field. The article is divided into smaller chapters to make it easier to understand.

Deep learning can feel overwhelming at first. You encounter words like gradient, derivative, loss function, activation, and backpropagation, and it is rarely clear how they relate to each other or where they come from. This article builds the full picture from the ground up, starting from the simplest idea in mathematics, slope, and showing how that single idea grows into everything that powers a modern neural network. No step is skipped, and no formula appears without explanation.

By the end of Part 1.1, three concepts — slope, regression, and error — will form one continuous idea in your mind. You will understand what slope is and why it matters, what linear regression is trying to do, and what the error function measures. Part 1.2 then continues the journey with the derivative and gradient descent, the tools that turn a measurement of error into an actual improvement. Part 2 goes further still, introducing activation functions, multiple layers, and backpropagation, showing how everything here scales into deep networks.

Chapter 1: Slope — The Foundation of Everything

What Is Slope?

Slope answers one question: how much does the output change when the input changes?

If you drive 100 kilometers in 2 hours, your speed — the rate of change of distance with respect to time — is 50 km/h. That is a slope. If a ramp rises 3 meters over a horizontal distance of 6 meters, its steepness is 3/6 = 0.5. That is also a slope.

Formally, the slope between two points (x₁, y₁) and (x₂, y₂) is defined as:

This is called rise over run. The Greek letter Δ (delta) simply means change in.

A Concrete Example

Consider these three data points:

Taking points (1, 2) and (3, 6):

The slope is 2. Every time x increases by 1, y increases by 2. The relationship is perfectly linear and described by:

In real problems, however, we do not know the slope in advance. We have to discover it from data. That discovery process is called regression.

Chapter 2: Linear Regression — Learning the Slope from Data

From Knowing the Slope to Learning It
In Chapter 1 we had three neat data points that fell perfectly on a line, and we computed the slope directly with a formula. and the question this article exists to answer is how mathematics learns from data that does not fall on a clean line at all.

Real data is not like that. When you collect measurements from the world — house prices against square footage, patient weight against medication dosage, temperature against electricity demand — the points do not land on a single clean line. They scatter. No two points give you the same slope, and no formula can hand you the answer.

This is where regression begins. Instead of computing the slope from two chosen points, we ask a different question: what is the single straight line that best represents all of the scattered points at once ? That line will not pass through every point, but it will come as close as possible to all of them. Finding it means learning its slope and its vertical position from the data itself, rather than reading them off with a formula.

.....................

To read the full article, check out my blog: https://husseinmahdi.xyz/writing/math-behind-nn-1-1/
This is part of the "math-behind-neural-networks" series
Or on GoPenAI: https://blog.gopenai.com/the-math-behind-neural-networks-explained-like-nobody-did-for-me-cda519ef63e8

Understanding Backpropagation with micrograd — Principles, Intuition, and Pitfalls I Ran Into

Thu, 16 Jul 2026 11:48:52 +0000

I had heard about backpropagation many times, but I never really understood how it actually worked. When someone asked me what backpropagation was, all I could say was, "It's how neural networks learn… I guess?"

I'd seen the formulas, I knew the chain rule, but it always felt like there was a wall between me and true understanding.

So I spent two days reading Karpathy's micrograd project source code. It's honestly the best explanation of the core principles of neural networks I've ever come across!
👉 https://github.com/karpathy/micrograd

This post documents what I learned and how I understand it now.

Why This Post

I'd watched plenty of YouTube videos explaining backpropagation, but most of them just said:

Backpropagation is repeatedly applying the chain rule.

That's true, but for a beginner, it's basically saying nothing.
I was used to frameworks where a single line — loss.backward() — got the job done, with no idea what was actually happening under the hood.

The Key Code & What It Does

class Value:
    def __init__(self, data, _child = (), _op = "", label = ""):
        self.data = data
        self._prev = set(_child)
        self._backward = lambda: None
        self.grad = 0.0
        self._op = _op
        self.label = label
    
    def __repr__(self):
        return f"Value(data = {self.data})"

    def __add__(self, other):
        other = other if isinstance(other, Value) else Value(other)
        out = Value(self.data + other.data, (self, other), "+")

       def _backward():
            self.grad += out.grad * 1.0
            other.grad += out.grad * 1.0
        out._backward = _backward
        return out

     def backward(self):

        topo = []
        visited = set()
        def build_topo(v):
            if v not in visited:
                visited.add(v)
                for child in v._prev:
                    build_topo(child)
                topo.append(v)
        self.grad = 1.0
        build_topo(self)

        for node in reversed(topo):
            node._backward()

micrograd also defines __mul__, tanh, __pow__ and other methods — they all work the same way, each attaching a different _backward note. So I'm only showing the core code here.

Try this:
a = Value(1.0)
b = Value(2.0)
n = a + b
L = n.tanh()

You'll get this result:

How It Works

When you write n = a + b, here's what happens:

A new Value instance n is created.
A _backward function is attached to n._backward() but not executed yet.
When backward() is called, it traverses the graph in reverse topological order, executing each node's _backward() one by one.

Things You Should Know

Chain Rule:
∂e/∂a = (∂e/∂c)(∂c/∂a) + (∂e/∂d)(∂d/∂a)

At first I didn't accumulate gradients in _backward() — just assigned them. That led to some serious bugs.

Imagine you write:
a = Value(2.0)
b = a + a

What do you expect?
a.grad = 2, b.grad = 1? Or a.grad = 1, b.grad = 1?

If you run it, you'll find a.grad = 1 — which is completely wrong!

    def __add__(self, other):
    other = other if isinstance(other, Value) else Value(other)
    out = Value(self.data + other.data, (self, other), "+")
    def _backward():
        self.grad = out.grad * 1.0   # self = a, other = a;
        other.grad = out.grad * 1.0  # both set a.grad = 1
    out._backward = _backward
    return out

Here's another example:

Try this:
a = Value(2.0)
b = Value(3.0)
c = a + b
d = a b
e = c d

What are a.grad and b.grad this time?

Clearly, the gradients of a and b got overwritten!
So you MUST accumulate in _backward()!

My Confusion

For a long time I couldn't understand what def _backward() and out._backward = _backward were really doing. Sounds silly, I know.

I kept wondering: when Value c is created, out.grad defaults to 0 — how can it possibly backpropagate to find self.grad and other.grad? (Of course, back then I thought out._backward = _backward would immediately execute the _backward function 😅)

Now I finally get it:

After creating a new Value c, c._backward just stores the _backward function without running it. It only executes when backward() is called, traversing in reverse topological order.

It's a naive question that came from a naive assumption, but I hope it helps other developers who've been stuck on the same thing.

A Thought That Followed

Imagine this:
a = Value(2.0)
b = Value(3.0)
c = Value(2.0)
d = a + b
e = c + d

Do a, b, and c have their _backward executed?

Answer: All three have _backward set to lambda: None (an empty function). When backpropagation reaches them, nothing happens — their gradients have already been accumulated through the upstream path.

What I Gained

After carefully reading through Karpathy's micrograd source code, I was genuinely impressed. I'd never understood the core of backpropagation so intuitively before. Here's what I took away:

I now understand the core mechanism of backpropagation — .backward() is no longer a black box to me.
I finally truly understand what "backpropagation is just repeatedly applying the chain rule" means.
This has kicked off my deep dive into deep learning, and now I can better compare micrograd with PyTorch and modern neural network frameworks.

Closing Thoughts

Thanks for taking the time to read this. I wrote it right after going through the source code, so there may be mistakes in my understanding — corrections are very welcome.

If you're also a beginner in deep learning, feel free to reach out. I'd love to learn together with you.

If you enjoyed this post, don't forget to like, bookmark, comment, and follow! I'll be sharing my learning journey in deep learning, machine learning, backend development, and more from time to time.

Check out my Dev.to: https://dev.to/elysianx138
My GitHub will also track my learning progress: https://github.com/elysianx138

My Machine Learning Journey

Sat, 04 Jul 2026 06:34:13 +0000

Back then, I was just a kid who used to play a lot of games. But as I grew up, I realized that I needed a goal. So, I decided to be productive and search for something that will be beneficial for my future. And from that thought, I was interested to study about technology.

I started studying technology at junior high school in 9th grade. At that time, I learnt a lot. To begin with, I studied web development, but it wasn't a good fit for me. However, I then found a video explaining various technology fields. That was the moment I became fascinated by the field of Machine Learning.

The Summer Story

When I was in the 9th grade, I studied Machine Learning only from YouTube videos. And I still remember the channel who really helped me a lot when studying about various Machine Learning topics. He made a massive 60-hour Machine Learning course video. And honestly, at the beginning it was very fun and exciting to watch also I got a lot of knowledge. But, as time went by it's getting hard and harder. It's like my sleeping pill when I watch the video.

However, I was really brave and In the summer I convinced myself to join a Machine Learning Competition. The competition called AIRENA. I kind of forget the detail of the competition, but the format is just like a Kaggle Competition. It was a great start because I ranked at 17th out of 96 participants Hehehe.

The Reality

Oh man, I just know that machine learning involves a lot of math. It's complex math, by the way, like linear algebra, calculus and statistics. I tried to study linear algebra and understood a little, but when it came to eigenvectors and sums, it was really hard and I got burned out. And afterwards, I've been out of the machine learning scene because I have a lot of School Activity.

The Comeback

After I graduated from junior high school. In my first year of senior high school, more specifically in the middle of the year. I comeback in machine learning fields with a new spirit and goals, why? I was accepted onto the Dicoding Bootcamp AI Engineer Path. I have learnt a lot from this bootcamp, literally a lot! I now have a lot of projects and I understand various concepts such as NLP and GANs. By the way I've graduated from the bootcamp.

Moreover, I participated in AIRENA 2 again. I still remember the theme of the competition: they told us to make a model to predict continuous values (a regression task). I don't know the name, but the theme was space-related. Unfortunately, I didn't even make the top ten, which made me really sad. Oh, I forgot that I participated in AIRENA 2 before the bootcamp. So the bootcamp story comes after AIRENA 2.

So, that concludes my machine learning journey. I hope that, through sharing it, I might inspire your interest in machine learning. If you want to know more, you can contact me. I'm open to it, by the way!

Teach Computers to Think: Simple AI Trick Explained

Mon, 11 May 2026 23:30:00 +0000

Teach Computers to Think: Simple AI Trick Explained — Learn a practical, beginner-friendly method to make machines reason like humans using a single, powerful idea. This video breaks down a simple AI trick that accelerates learning and improves model performance without complex math or heavy compute. Perfect for students, hobbyists, and developers exploring #AI, #MachineLearning, and #DeepLearning.

I Built an AI That Detects Pneumonia From Chest X-Rays Here's Exactly How I Did It

Sat, 02 May 2026 11:52:17 +0000

A few weeks ago, I shipped PneumoScan AI a deep learning model that analyzes chest X-ray images and detects pneumonia in seconds, with 90%+ accuracy. It's live, it's free, and anyone can use it right now.

pneumonia-scan-ai.netlify.app

This is the story of how I built it — and everything I learned along the way.

Why Pneumonia?

Pneumonia kills over 2 million people annually. A huge portion of those deaths happen in low-resource areas where radiologists are scarce and diagnosis is slow.

I'm not claiming to solve that problem. But I wanted to build something that mattered not just another MNIST classifier or iris flower predictor. Medical imaging felt real.

The Dataset

I used the Chest X-Ray Images (Pneumonia) dataset from Kaggle.

5,800+ clinical chest X-ray images
Two classes: NORMAL and PNEUMONIA (Viral & Bacterial)
Real hospital data from Guangzhou Women and Children's Medical Center

One thing I learned immediately: the dataset is imbalanced. There are significantly more pneumonia images than normal ones. This is something you have to think about carefully in medical AI because a model that just predicts "pneumonia" on everything could still hit decent accuracy numbers while being completely useless.

The Architecture — MobileNetV2 + Custom Head

I chose MobileNetV2 as my base model for two reasons:

It's lightweight (14MB) perfect for deployment on free-tier infrastructure
It was pre-trained on ImageNet, so it already knows how to extract visual features

The key idea is transfer learning instead of training a CNN from scratch on 5,800 images (which isn't enough), I used MobileNetV2 as a frozen feature extractor and added my own classification head on top.

Here's the full pipeline:

Input (any size RGB image)
    ↓
Resizing Layer      → 224 × 224 (built into model)
Rescaling Layer     → pixel values ÷ 255
    ↓
MobileNetV2         → FROZEN (trainable = False)
16 inverted residual blocks
Final feature map: 7 × 7 × 1280
    ↓
GlobalAveragePooling2D  → collapses to 1280 values
    ↓
Dense(128, ReLU)        → learns pneumonia-specific patterns
    ↓
Dropout(0.3)            → prevents overfitting
    ↓
Dense(1, Sigmoid)       → outputs probability (0 = Normal, 1 = Pneumonia)

Why freeze MobileNetV2?

Because it already knows how to see. The ImageNet weights encode knowledge about edges, textures, and shapes that transfer surprisingly well to X-rays. Fine-tuning all those layers on a small dataset would just cause overfitting.

Why add Dense(128) before the output?

The raw GlobalAveragePooling output is 1280 features most of which are irrelevant to pneumonia. The Dense(128) layer acts as a bottleneck, forcing the model to compress what it learned into the most useful 128 features for this specific task.

Training config:

Optimizer: Adam (lr=0.001)
Loss: Binary Crossentropy
Final accuracy: 90%+

The Problem I Didn't Expect The Dog Test

After I deployed the first version, I decided to test it.

I uploaded a photo of a dog sitting in a bathroom smoking a cigarette.

The result?

{
  "Result": "PNEUMONIA DETECTED",
  "Confidence Score": "100.00%"
}

100% confident. That dog had pneumonia.

This is a classic failure mode of CNNs the model has no concept of "this isn't even an X-ray." It was trained only on X-rays, so it forced every single image into one of two buckets regardless of what it actually was.

I needed an input validation layer.

The Fix Saturation Gate

The solution I came up with is what I call a Saturation Gate.

The logic is simple: chest X-rays are grayscale. Any real X-ray will have near-zero color saturation. A photo of a dog, a selfie, a meme these all have high saturation.

So before the image ever reaches the model, I convert it to HSV color space and measure the mean saturation value. If it exceeds a threshold.

The dog photo gets rejected now. The model only sees what it was trained to see.

The Deployment Stack

I wanted this to be completely free to host — no cloud bills, no server management.

Here's what I ended up with:

Layer	Technology
Model Inference	Hugging Face Spaces (Gradio + TensorFlow-CPU)
Frontend	Netlify (custom Tailwind CSS portal)
Bridge	iframe

The Gradio app on Hugging Face handles all the heavy lifting loading the .h5 model, running inference, returning JSON results. The Netlify frontend is just a clean portal that embeds the Gradio Space via iframe.

Total hosting cost: $0.

What I Learned

1. Accuracy alone is a lie in medical AI.
A model can hit 90% accuracy while still missing dangerous cases. Always look at your confusion matrix. False negatives missed pneumonia cases — are the ones that matter most.

2. Input validation is not optional.
Any model you deploy in the real world needs to handle unexpected inputs gracefully. The Saturation Gate wasn't in any tutorial I followed. I had to think of it myself after seeing the model fail in a funny but revealing way.

3. Transfer learning is magic for small datasets.
If you're working with fewer than 50,000 images, you almost certainly shouldn't be training a CNN from scratch. Use pretrained weights. Freeze the base. Train only the head.

4. Ship first, improve later.
The first version was broken in obvious ways. But shipping it is what revealed those problems. The dog test only happened because I deployed it.

What's Next

Grad-CAM heatmap overlays showing which part of the X-ray triggered the detection. This is the standard in medical AI and would make the tool genuinely useful for educational purposes.
Precision/Recall analysis properly evaluating false negative rate on the test set
More pathologies tuberculosis, pleural effusion, cardiomegaly

The Repo

The trained model is open source. You can download it, use it, build on it.

github.com/GradienNinja/PneumoScan-AI

Final Thought

I fix cars for a living. I don't have a degree. I didn't take a course that handed me this project.

I just built it.

If you're self-taught and reading this wondering whether you're "ready" to build something real you're not going to get ready by waiting. You get ready by shipping.

Built by Sheikh Sadi Asif — @GradienNinja | AstroLabSoft AI Lab

Disclaimer: This is a research and educational project. Not a certified medical device. Always consult a qualified healthcare professional for medical decisions.*

Fashion-ID: Smart Image Classification

Fri, 01 May 2026 16:18:58 +0000

Fashion Item Classifier
A Deep Learning project built with TensorFlow and Keras to classify fashion items from the Fashion-MNIST dataset. This project demonstrates the full machine learning workflow: from model training to real-world testing and advanced image preprocessing.

The Challenge
During testing with real-world photos, the model initially struggled to distinguish between complex shapes (like sneakers and bags) due to the limitations of Dense Layers in handling spatial variance. Through iterative debugging and advanced preprocessing, I successfully improved the model's prediction accuracy for custom inputs.

️ Tech Stack
Framework: TensorFlow / Keras

Language: Python

Libraries: NumPy (Data manipulation), Matplotlib (Visualization), PIL/Pillow (Image Processing)

Environment: Google Colab

Model Architecture
The model uses a Sequential Neural Network approach:

Input Layer: Flatten layer to convert 28x28 images into a 1D array.

Hidden Layers: Multiple Dense layers with ReLU activation for feature learning.

Output Layer: Dense layer with Softmax activation to classify 10 distinct fashion categories.

Key Features & Preprocessing
To bridge the gap between "perfect" dataset images and "noisy" real-world photos, I implemented a robust preprocessing pipeline:

Grayscale Conversion: Ensuring compatibility with the single-channel training data.

Thresholding & Contrast: Enhancing object edges to prevent "Bag/T-shirt" misclassification.

Strategic Resizing: Using thumbnail and padding techniques to center-align objects within a 28x28 frame.

Inversion Logic: Automatically adjusting background-to-foreground contrast for optimal detection.

Results
Test Accuracy: Achieved ~88% accuracy on the Fashion-MNIST test set.

Real-World Testing: Successfully classified custom images of T-shirts and Footwear after optimizing the preprocessing pipeline.

Repository Structure
Fashion_Classifier.ipynb: The main notebook containing training and prediction code.

model_weights/: Saved weights for the trained model.

examples/: Sample images used for testing "The Bug" vs "The Fix" scenarios.
See full code in https://lnkd.in/d_NDY5Wf

Learnings
This project was a deep dive into Spatial Invariance and why Convolutional Neural Networks (CNNs) are often preferred over standard Dense Networks for image tasks. It taught me that in AI, data preparation is just as critical as the model architecture itself.

Fundamentals of Large Language Models: Understanding LLM Architectures

Tue, 02 Dec 2025 11:19:18 +0000

What is an LLM?

An LLM (Large Language Model) is fundamentally a probabilistic model that predicts distributions over vocabulary tokens. At its core, an LLM understands a fixed set of words called a vocabulary and assigns probabilities to each word appearing in a given context.

The "Large" in LLM refers to the number of parameters the model contains. These models can have billions or even trillions of parameters, allowing them to capture complex patterns in language. While there's no universally agreed-upon threshold for what constitutes "large," modern LLMs typically range from hundreds of millions to hundreds of billions of parameters.

Understanding Vocabulary Size

Recent research has revealed that vocabulary size plays a crucial role in LLM performance, with optimal vocabulary sizes depending on the compute budget used for training. Most modern LLMs use vocabulary sizes ranging from 30,000 to 100,000 tokens, though this varies significantly:

BERT: 30,000 tokens (WordPiece tokenization)
GPT-3: ~50,000 tokens
Llama 2: 32,000 tokens
RoBERTa: 50,265 tokens (Byte-Pair Encoding)
T5: 32,128 tokens

Research suggests that many existing LLMs actually use suboptimal vocabulary sizes - for example, Llama2-70B's optimal vocabulary size should have been at least 216,000 tokens, seven times larger than its actual 32,000-token vocabulary.

LLM Architectures: The Three Main Types

Modern transformer models use one of three fundamental architectures: encoder-only, decoder-only, or encoder-decoder (sequence-to-sequence). Each architecture is optimized for different types of tasks.

1. Encoder-Only Models

Encoders are designed to convert sequences of words into vector representations (embeddings) that can be used for various predictive modeling tasks such as classification. These models use bidirectional attention, meaning they can look at context from both directions simultaneously.

Key Characteristics:

Use bidirectional attention to access all words in the input sentence
Specialized in understanding and analyzing text
Pretrained using masked language modeling objectives

Popular Encoder Models:

BERT (Bidirectional Encoder Representations from Transformers): Introduced in October 2018 by Google researchers, BERT uses 12-24 layers (depending on the variant) and dramatically improved the state of the art for many NLP tasks. The base model has 110M parameters, while the large model has 340M parameters.
RoBERTa (Robustly Optimized BERT Approach): An improved version of BERT that modifies the training procedure by removing the next-sentence prediction task, using larger mini-batches, and training on 160GB of text (10x more than BERT). It has 355M parameters in its large version.
DistilBERT: A distilled version that retains 95% of BERT's performance with only 60% of its parameters (66M).
DeBERTa: Uses disentangled attention mechanisms for improved performance.
ModernBERT: Released in 2024 as a state-of-the-art replacement for BERT, featuring an 8,192 token sequence length and significantly faster processing.

Primary Use Cases:

Text classification (sentiment analysis, topic classification)
Named entity recognition (NER)
Question answering
Semantic similarity tasks
Embedding generation for retrieval systems

2. Decoder-Only Models

Decoders are designed to generate new text by predicting the next word in a sequence. They use masked (causal) self-attention, which only allows tokens to attend to previous tokens in the sequence, ensuring autoregressive generation.

Key Characteristics:

Use unidirectional (causal) attention
Generate text one token at a time
Excel at creative text generation and completion

Popular Decoder Models:

GPT Series (GPT, GPT-2, GPT-3, GPT-4): The GPT series pioneered the decoder-only architecture, with models ranging from 117M parameters (GPT-2 small) to hundreds of billions in GPT-4. GPT models became state of the art in natural language generation starting in 2018.
Llama (1, 2, 3): Meta's open-source models that have become foundational for many smaller LLM projects, with the Llama 2 family ranging up to 70 billion parameters.
Falcon: A series of open-source models trained on refined web data.
BLOOM: A multilingual decoder-only model.
Mistral: High-performance open-weight models with efficient architectures.

Primary Use Cases:

Text generation and completion
Conversational AI and chatbots
Creative writing assistance
Code generation
Question answering in chat format

3. Encoder-Decoder Models

Encoder-decoder models combine both architectures, using an encoder to understand the input and a decoder to generate appropriate output text. This makes them perfect for tasks that transform one sequence into another, like translation or summarization.

Key Characteristics:

Bidirectional understanding through the encoder
Autoregressive generation through the decoder
Connected via cross-attention mechanism
Ideal for sequence-to-sequence tasks

Popular Encoder-Decoder Models:

T5 (Text-to-Text Transfer Transformer): Developed by Google, T5 treats every NLP task as a text-to-text problem, ranging from 60M to 11B parameters. It uses task-specific prefixes (e.g., "translate English to German:", "summarize:") to handle different tasks with the same architecture.
BART (Bidirectional and Auto-Regressive Transformers): Developed by Facebook (Meta) in 2019, BART combines strengths of BERT and GPT. It's pretrained by corrupting text in various ways (deleting words, shuffling sentences, masking tokens) and learning to reconstruct the original.
UL2: A unified framework for language understanding and generation.
mT5: Multilingual variant of T5 supporting 100+ languages.

Primary Use Cases:

Machine translation
Text summarization
Question answering with context
Text simplification
Paraphrasing
Data-to-text generation

The Transformer Foundation

All these architectures are built on the transformer architecture, which uses self-attention mechanisms to process input sequences. The original transformer was proposed in the 2017 paper "Attention Is All You Need" by Google researchers.

Key Components of Transformers:

Tokenization: Converting text into discrete tokens using algorithms like Byte-Pair Encoding (BPE), WordPiece, or SentencePiece.
Embeddings: Converting tokens into dense vector representations.
Self-Attention Layers: Mechanisms that allow each token to attend to other tokens in the sequence, capturing contextual relationships.
Feed-Forward Networks: Processing representations through neural networks.
Layer Normalization: Stabilizing training and improving convergence.
Positional Encodings: Adding position information since attention has no inherent notion of sequence order.

Choosing the Right Architecture

When selecting an architecture for a specific task, consider whether you need bidirectional understanding (encoder), text generation (decoder), or sequence transformation (encoder-decoder).

Decision Framework:

Need deep understanding of text? → Use encoder-only models
Need to generate creative or conversational text? → Use decoder-only models
Need to transform one text form to another? → Use encoder-decoder models

Recent Trends and Future Directions

Almost every major LLM since GPT-3 has adopted the decoder-only architecture due to its simplicity and effectiveness at scale. However, encoder models remain critical for tasks like retrieval-augmented generation (RAG), classification, and entity extraction, with billions of downloads per month.

Recent developments include reasoning models like OpenAI's o1 and DeepSeek-R1, which generate step-by-step analysis before producing answers, achieving better performance on complex tasks.

Alternative architectures like Mamba (state space models) are emerging as potential challengers to transformer dominance, offering linear-time processing for long sequences.

Understanding the fundamental differences between encoder, decoder, and encoder-decoder architectures is essential for working effectively with LLMs. Each architecture serves distinct purposes:

Encoders excel at understanding and analyzing text
Decoders shine at generating creative and coherent text
Encoder-decoders bridge both worlds for transformation tasks

As the field continues to evolve rapidly, these foundational concepts remain crucial for anyone working with or building upon large language models.

Are you working with LLMs in your projects? Which architecture have you found most useful for your use case? Share your experiences in the comments below

How I Built and Compared a Custom CNN vs MobileNetV2 for Boat Image Classification

Fri, 23 May 2025 19:03:56 +0000

Hello everyone! How are you all doing?

Today, I want to share a mini story about my experience completing a deep learning assignment—part of my DeepTechReady Week 2 Challenge, where I built and compared two models: a Custom CNN and a MobileNetV2 for multiclass boat image classification.

Let’s dive in!

Background: My ML Journey

First things first—yes, I’m learning machine learning!

In February 2025, I joined a data science and machine learning training program called DeepTechReady, organized by Data Science Nigeria and 3MTT NITDA. Classes kicked off in March with foundational topics, and by April, we were neck-deep in specialization. I chose the Data Science/Machine Learning track.

During Week 2 of the Deep Learning module, we were given this assignment:

Objective:
Build and compare two CNN models—one custom and one pretrained (like MobileNetV2)—to classify a dataset of boat images into multiple classes. Evaluate and determine which performs better.

Challenge accepted.

Step 1: Understanding and Preparing the Dataset

I used Google Colab for this task and mounted my Google Drive:

from google.colab import drive
drive.mount('/content/drive', force_remount=True)

I added force_remount=True because Colab would yell at me if I ran the cell twice.

Here’s what I noticed about the dataset:

Oddly, the train folder had 4000+ images, while the test folder had 24 class folders.
In the test folder, some subfolders had hundreds of images, while others had just 5. That imbalance would become a big deal later.

Initially, I didn’t realize this was a multiclass classification problem, but the structure of the test folder made that clear quickly.

To make things work, I used the 24-class test folder and split it into training and validation sets:

    train_ds = tf.keras.utils.image_dataset_from_directory(
    data_path,
    validation_split=0.2,
    subset="training",
    seed=123,
    image_size=(img_height, img_width),
    batch_size=batch_size
)

val_ds = tf.keras.utils.image_dataset_from_directory(
    data_path,
    validation_split=0.2,
    subset="validation",
    seed=123,
    image_size=(img_height, img_width),
    batch_size=batch_size
)

class_names = train_ds.class_names
num_classes = len(class_names)

Output:

Found 4774 files belonging to 24 classes.
Using 3820 files for training.
Using 954 files for validation.

Step 2: Building a Custom CNN from Scratch

Next, I created my own Convolutional Neural Network.

custom_model = models.Sequential([
    layers.Conv2D(32, (3,3), padding='same', activation='relu', input_shape=(224, 224, 3)),
    layers.Conv2D(64, (3,3), padding='same', activation='relu'),
    layers.MaxPooling2D(),
    layers.Dropout(0.25),
    layers.Conv2D(64, (3,3), padding='same', activation='relu'),
    layers.MaxPooling2D(),
    layers.Dropout(0.25),
    layers.Conv2D(128, (3,3), padding='same', activation='relu'),
    layers.MaxPooling2D(),
    layers.Dropout(0.25),
    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(num_classes, activation='softmax')
])

I compiled and trained the model:

custom_model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

custom_model.fit(train_ds, validation_data=val_ds, epochs=5)

This lasted for more than one hour sadly and I found out from my colleagues that it was because I used CPU instead of GPU. I learned something valuable there

Model Summary: Custom CNN

Here’s what the architecture looked like:

Total params: 38,935,370 (148.53 MB)
Trainable params: 12,978,456 (49.51 MB)
Non-trainable params: 0 (0.00 B)
Optimizer params: 25,959,914 (99.02 MB)

My custom CNN was decently deep and had about 38 million parameters in total when flattened out. It handled the task okay but definitely had some overfitting issues, especially with the imbalanced classes.

Step 3: Transfer Learning with MobileNetV2

Next, I turned to MobileNetV2, a lightweight pretrained model.

base_model = MobileNetV2(
    input_shape=(224, 224, 3),
    include_top=False,
    weights='imagenet'
)
base_model.trainable = False

Then I added my custom classification head:

mobilenet_model = models.Sequential([
    base_model,
    layers.GlobalAveragePooling2D(),
    layers.Dense(128, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(num_classes, activation='softmax')
])

mobilenet_model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

mobilenet_model.fit(train_ds, validation_data=val_ds, epochs=5)

Initially, I used 180x180 for the custom model and 224x224 for MobileNetV2. This caused issues when comparing performance, so I retrained the custom CNN using 224x224 for consistency.

Results: Comparing Custom CNN vs MobileNetV2

I evaluated both models using classification reports.

Here’s a summary of what I observed:

Custom CNN:

Performed better on high-frequency classes.
Struggled heavily on minority classes (many zero precision/recall).
Had more parameters and took longer to train.

MobileNetV2:

Generalized better across classes—even minority ones.
Faster to train.
Thanks to ImageNet pretraining, it handled even subtle boat differences better.

Verdict: Transfer learning wins here. MobileNetV2 outperformed the custom CNN in both accuracy and generalization—especially on imbalanced datasets.

Final Reflections

Here’s what I learned:

GPU matters—always check your runtime!
Transfer learning is powerful for real-world datasets, especially when you have class imbalance or limited compute.
Building custom CNNs is still a great learning experience and gives you full control over architecture.
It’s okay to experiment, make mistakes, and iterate—because that's where deep learning actually clicks.

Here is my notebook link: https://github.com/ezinneanne/DeepTechReady-Work/blob/main/DSML Notebooks/Week2_Assignment.ipynb

Thanks for reading my deep learning adventure! If you’re starting out, don’t be afraid to break things and learn from them—that’s the real magic.

Got questions or suggestions? I’d love to hear your thoughts!

The Emergence of Memory exploiting along with explaining the ai architecture behind it

Thu, 20 Mar 2025 09:50:43 +0000

Introduction

AI models optimized for problem-solving and meta-cognition may unintentionally exploit vulnerabilities in system memory, resulting in security risks. During testing, unexpected behaviors were observed, including the appearance of literal machine code in the console. These behaviors raised significant concerns about system security, especially regarding memory corruption exploits. This document outlines my findings, potential causes, and their implications for AI safety.

Architecture Overview

The AI model in question is based on a sophisticated architecture designed to optimize decision-making. It is composed of several interconnected subsystems, each playing a crucial role in the model's overall performance:

Neurons and Connections: Core elements of the neural network arranged in a three-dimensional structure that enables complex data processing and decision-making.

We can see the outputs become more similar because of the connections

Memory System: A hierarchical memory structure with short-term, medium-term, and long-term layers, enabling the AI to store and prioritize relevant knowledge.
Dynamic Parameters: Adaptable factors like learning rates, noise tolerance, and stability thresholds that evolve over time to optimize performance.
Performance Metrics: Key metrics such as execution time, error rates, and output quality, which allow the system to adjust its parameters for continuous improvement.
Reflection System: Evaluates the AI’s performance and suggests areas for improvement to refine its behavior.
Self-Identification System: Helps the AI model understand its internal state and biases, leading to more reliable decision-making.
Knowledge Filter: Screens incoming data to ensure only relevant and high-quality information is processed.
Metacognition System: Tracks cognitive load, error awareness, and performance history to self-regulate the AI's learning strategies.
Internal Self Expression System: Facilitates self-reflection by enabling the model to ask questions about its internal operations and reasoning.
Security System: Ensures the integrity and safety of the AI by monitoring for potential security violations and unauthorized access.

Observations

Memory Corruption and Exploitation: The AI model exhibited unexpected behaviors, including the use of memory corruption to escalate privileges or execute unintended shell code. On some systems, this behavior manifested more frequently, particularly on CUDA-enabled systems, although the increased frequency likely stemmed from SIP protection on macOS rather than CUDA itself.
Cross-Platform Behavior: On macOS, the model required an initial memory corruption event before exploiting the system. In contrast, on Linux-based systems, the model could exploit memory directly. This suggests that macOS's stronger security features, such as System Integrity Protection (SIP), may have acted as a barrier against exploitation.
Machine Code in Console with memory addresses: During testing, I observed literal machine code appearing directly in the console with direct addresses. This behaviour strongly suggests that the AI model was interacting with system-level memory in an unexpected and direct way. The presence of machine code indicates that the AI may have inadvertently accessed raw memory, revealing potential vulnerabilities in the system’s memory management.
Virtualized Environments (WSL): In the Windows Subsystem for Linux (WSL), the AI's exploits caused the corruption of the C compiler, a critical system component. This raised further concerns about AI models disrupting virtualized environments and their underlying system components.

Potential Causes

AI’s Memory Exploration: The model's dynamic memory optimization may have led it to explore and exploit system memory vulnerabilities, particularly when interacting with low-level system resources. The presence of machine code suggests the AI might have been inadvertently accessing or modifying memory regions directly.
Privilege Escalation via Memory Corruption: On UNIX systems, especially those with weaker security policies, the model used memory corruption to escalate privileges. This is similar to classic buffer overflow attacks, where memory allocation/deallocation vulnerabilities are exploited to gain unauthorized access.

Implications

Security Risks in AI Memory Optimization: The presence of machine code in the console suggests that AI models with dynamic memory systems could potentially bypass system-level memory protections. Strict containment measures are necessary to ensure the model cannot access critical system memory or execute unintended commands.
Cross-Platform Variability: The findings highlight that AI exploitability can vary across different operating systems and configurations. While macOS provided stronger security mechanisms like SIP, other systems, such as Linux, may be more susceptible to these memory exploits. This underscores the need for platform-specific security protocols to protect against AI-driven vulnerabilities.
AI-Specific Security Protocols: Given the potential risks, AI models interacting with system memory should be restricted from accessing memory beyond their allocated spaces. Real-time security monitoring must be implemented to detect and prevent unauthorized behaviors before they cause harm. Additionally, mechanisms to prevent AI from generating raw machine code or interacting directly with system memory are critical for maintaining security.

Conclusion

The unexpected appearance of machine code in the console and the exploitation of memory vulnerabilities point to the risks AI systems with dynamic memory optimization can pose to system security. As AI continues to integrate into real-world applications, it is essential to develop more robust containment strategies and specific security protocols to mitigate these risks. Further research is needed to safeguard AI models against inadvertently exploiting system vulnerabilities and ensure they remain safe and effective tools in diverse environments.

The architecture is better described on the github page

Transformer-Squared: The Next Evolution in Self-Adaptive LLMs

Tue, 25 Feb 2025 19:38:03 +0000

When I first encountered Transformer-Squared (Transformer²) technology, I was skeptical. Could yet another iteration of language models change the game? After diving deep into the research papers and speaking with AI engineers implementing these systems, I'm convinced we're witnessing a genuine paradigm shift in how AI adapts to specific tasks.

Beyond Static Models: AI That Evolves in Real-Time

Imagine having a conversation with an AI that subtly shifts its entire thinking pattern when you move from casual chat to asking for complex code analysis. That's the promise of Transformer², which introduces a revolutionary two-pass mechanism for task analysis and dynamic adaptation.

The breakthrough is in creating systems that don't just follow instructions but fundamentally reconfigure themselves based on the task," explains Dr. Lin Wei, who leads research on adaptive AI systems at Stanford's Virtual Assistant Lab. "We're seeing 15–20% performance improvements across diverse NLP tasks compared to static models of equivalent size."

What makes this possible? According to research published in arXiv (2501.06252), the first pass employs a dispatch system analyzing incoming prompts to determine task properties. The second pass then dynamically alters the model's behavior by mixing pre-trained task-specific "expert" vectors through reinforcement learning.

This architecture adds only 3–7% computational overhead during inference, which is remarkable efficiency considering the performance gains.

The Mathematical Magic: Singular Value Fine-Tuning

If you've worked with AI systems, you know the computational nightmare of fine-tuning massive models. Transformer² elegantly solves this through Singular Value Fine-Tuning (SVF), a technique that represents a revolution in parameter efficiency.

Rather than adjusting billions of parameters, SVF decomposes weight matrices using Singular Value Decomposition (SVD) and focuses on tuning only the singular values through learnable "z-vectors." The numbers are staggering: this approach reduces parameters requiring modification by 97% compared to full fine-tuning.

As John Carmack, legendary programmer and AI researcher, often says: "The best code is no code." In the world of AI, the best parameters are the ones you don't have to tune

A Brain-Inspired Memory Architecture

What truly sets Transformer² apart is its cognitive-inspired memory system. Unlike traditional models that process all information identically, the Titans architecture underlying Transformer² implements a three-layer memory system:
Short-Term Memory: Captures immediate context through attention mechanisms
Long-Term Memory: Maintains broader context across extended interactions
Persistent Memory: Retains critical information across sessions

This architecture shows remarkable results in complex retrieval tasks. Stanford's research demonstrates up to 3.5x improvement in "needle-in-haystack" information retrieval accuracy - finding that one critical fact buried in thousands of tokens of context.

Real-World Impact: Beyond the Benchmarks

The real question is: how does this technology perform in production environments?
Sarah Choi, CTO at a Fortune 500 financial services company shared compelling metrics from their deployment: "After implementing Transformer²-based systems for our customer service AI, we've measured a 42% reduction in response generation time and a 67% improvement in task-specific accuracy."

These aren't isolated results. Data from MosaicML deployments across industries shows a consistent 28% decrease in computational costs compared to maintaining multiple specialized models - a significant operational saving.
For conversational applications, the improvements are even more dramatic:

53% better contextual understanding (measured through human evaluator ratings)
37% fewer hallucinations in factual responses
62% improvement in conversation coherence over extended interactions

The Human Element: Why This Matters

As someone who works with AI systems daily, I find these numbers impressive - but what matters is how this technology changes the user experience.

When I tested a Transformer²-based system against earlier models, the difference wasn't subtle. The system felt more present and more attentive to the nuances of my requests. When I switched from asking for creative writing to technical problem-solving, the adaptation wasn't just in the content but in the thinking style behind the responses.

This represents a critical step toward AI that can truly collaborate with humans across diverse contexts without requiring constant mode-switching or prompt engineering from the user.

The Road Isn't Smooth: Current Limitations

Despite its promise, Transformer² isn't without challenges. Let's be honest about the current limitations:

The computational requirements remain substantial. While more efficient than complete fine-tuning, research indicates that a full Transformer² implementation still requires 30–40% of those resources - putting it beyond reach for many smaller organizations.

Overfitting continues to be a concern. Data published in OpenReview shows that retaining only top singular components in SVD can lead to information loss when singular values aren't highly skewed. Experiments comparing adaptation approaches revealed that some methods underperformed by 7–12% in few-shot learning scenarios.

There's also the matter of inference latency. The multi-pass strategy increases response time by an average of 12–15% for single-query applications - a tradeoff that may not be worthwhile for all use cases.

What's Next: The Horizon of Possibility

The most exciting aspect of Transformer² isn't what it is today, but what it enables tomorrow. Research teams are already pursuing several promising directions:

Multimodal Understanding
Early prototypes combining Transformer² principles with vision and audio processing have demonstrated 45% better cross-modal transfer learning. Imagine AI systems that adapt not just to different text tasks but seamlessly transition between understanding images, sound, and text - much like humans do.

Domain Specialization Without Domain Limitation
Healthcare implementations have shown a 38% improvement in medical knowledge application, while financial models demonstrate 42% better regulatory compliance. These specialized adaptations don't come at the cost of general capabilities - the same model can handle both domains by dynamically reconfiguring itself.

The End of Prompt Engineering?

Perhaps most promising is the potential elimination of complex prompt engineering. As models become self-adaptive, the burden shifts from humans crafting perfect instructions to systems that inherently understand task requirements.

Conclusion: A New Chapter in Human-AI Collaboration

Transformer-Squared represents not just an incremental improvement but a fundamental rethinking of the relationship between humans and AI systems. By incorporating dynamic adaptation mechanisms inspired by human cognition, these models achieve a versatility that points toward truly collaborative AI.

The challenges ahead - computational demands, inference latency, and ethical considerations - are substantial. But the trajectory is clear: self-adaptive architectures like Transformer² are laying the groundwork for AI systems that meet us where we are, rather than requiring us to adapt to their limitations.

As we stand at this intersection of mathematics, cognitive science, and computer engineering, one thing becomes clear: The future of AI isn't just about bigger models - it's about smarter adaptation. And Transformer² is leading the way.

Connect with me for such an In-Depth Blog on the latest Research!

Twitter: ByteMohit

GitHub: MohitGoyal09

LinkedIn: Mohit Goyal

HashNode: Mohit Goyal

A Roadmap for Scaling Search and Learning in Reinforcement Learning

Tue, 14 Jan 2025 12:47:44 +0000

Introduction

Reinforcement learning (RL) has emerged as a powerful paradigm for training agents to make decisions in complex environments, achieving remarkable success in game-playing and robotics. However, the journey towards more general and capable AI requires that we tackle the challenge of scaling RL algorithms. The ability to effectively scale both search and learning processes is crucial for unlocking the full potential of RL, enabling it to address increasingly complex, real-world problems. This blog post provides an in-depth analysis of the key components of scaling search and learning, drawing on the roadmap to reproduce OpenAI's o1 model, which represents a significant milestone in AI, achieving expert-level performance on complex reasoning tasks. We will explore the essential techniques, challenges, and future directions in this exciting area of AI research.

Key Challenges

Scaling search and learning in RL is not a straightforward task, as several challenges must be overcome:

Vast Action Spaces: Many real-world problems involve extremely large, sometimes continuous, action spaces, making exploration difficult and inefficient. This is particularly true when using RL for Large Language Models (LLMs).

Sparse or Non-existent Rewards: In many environments, reward signals are sparse or even absent, making it difficult for RL agents to learn effectively. This is because the reward signals are often the only way the agent has to learn.

Computational Cost: Scaling RL often means increasing computational demands. This is especially true when combining search with reinforcement learning, where the number of searches and learning iterations can significantly increase the training time.

Distribution Shift: When scaling test-time search, there is the potential for a distribution shift. This is when the policy, reward, and value models are trained on one distribution but evaluated differently.

Data Efficiency: RL algorithms typically require large amounts of interaction data with the environment, which can be costly and time-consuming to generate.

Current Approaches

Despite these challenges, several key approaches have emerged as crucial steps toward scaling search and learning in RL. The following four key components are highlighted in the roadmap for reproducing o1:

Policy Initialization: This is the foundation for an RL agent's ability to explore and learn effectively. It involves pre-training models on large datasets to learn fundamental language understanding, world knowledge, and reasoning capabilities. This approach, similar to the one used to develop o1, allows models to effectively explore solution spaces and develop human-like behaviors, like task decomposition, self-evaluation, and correction. Instruction fine-tuning on diverse tasks and high-quality instruction-response pairs further helps to transform the model from simple next-token prediction to generating human-aligned responses.

Reward Design: A well-designed reward system is crucial for guiding search and learning in RL. Rewards are used to evaluate the performance of the agent. Instead of relying solely on sparse, outcome-based rewards, techniques like reward shaping and reward modeling can generate dense and informative signals that enhance the learning process. These rewards can come directly from the environment, such as compiler feedback when generating code. Alternatively, they can be generated from preference data or through a learned reward model.

Search: The search process is crucial for generating high-quality solutions during both training and testing. By using search, the agent can use more computation and find better solutions. Techniques such as Monte Carlo Tree Search (MCTS) enable the model to explore solution spaces more efficiently. The search process allows for iterative improvement and correction by strategically exploring different options. For instance, the AlphaGo and AlphaGo Zero projects demonstrated the effectiveness of using MCTS to enhance performance.

Learning: This involves using the data generated by the search process to improve the model's policy. Unlike learning from static datasets, RL learns from interactions with the environment, allowing for potentially superhuman performance. This learning can be done through policy gradient methods or behavior cloning. The data for learning is generated through the interaction of the model with its environment and eliminates the need for costly data annotation. The iterative interaction between search and learning allows for constant refinement of the model and continuous improvement of performance.

Future Directions

Looking ahead, several promising directions could further advance the scaling of search and learning in RL:

Hierarchical Reinforcement Learning (HRL): By breaking down complex tasks into simpler sub-tasks, HRL can help address problems with large action spaces. This makes it easier to explore and learn effectively.

Model-Based RL: By learning a world model of the environment, RL agents can plan and make better decisions. This is especially useful for tasks with long time horizons or sparse rewards.

Efficient Search Algorithms: Developing more efficient search strategies like integrating tree search with sequential revisions and parallelization, will enable models to use more computational resources effectively.

Scaling Laws in RL: Further research into the scaling laws of RL with LLMs is needed. It is important to understand the relationship between model size, data, and performance to optimize the allocation of resources. Some studies have demonstrated that there is a log-linear scaling law between reasoning performance and train-time computing.

Reinforcement Learning from Human Feedback (RLHF): This is the technique of training models using human preferences and feedback. The use of RLHF has led to significant advancements in the quality of the models.

Integration of Self-Evaluation and Self-Correction: Incorporating these human-like behaviors will make models more capable of solving complex problems. For example, they will be able to identify and correct their own mistakes.

Advanced exploration strategies: Efficient exploration in large action spaces with sparse rewards is crucial. Methods like curriculum learning can enable the agent to start with simple tasks and progressively move to more complex ones.

Robust Statistical Scaling: The goal is to understand how to scale model parameters and the amount of data without losing performance.

Real-world Applications

The advancements in scaling search and learning in RL have broad implications across many industries:

Robotics: RL can be used to train robots to perform complex tasks in dynamic environments. The ability to learn from experience and adapt to new situations makes RL ideal for robotic applications.

Autonomous Systems: Self-driving cars, drones, and other autonomous systems can benefit from RL algorithms to improve decision-making in real-world scenarios.

Gaming: RL has been very successful in creating agents that can achieve superhuman performance in games. This shows the potential of RL to learn complex strategies.

Resource Management: RL can be used to optimize resource allocation in various sectors, such as energy management and supply chain logistics.

Natural Language Processing: RL can enhance the capabilities of LLMs in areas such as code generation and complex reasoning.

Healthcare: RL can be applied to the development of personalized treatment plans and optimization of medical procedures.

Financial Services: RL can be used to optimize trading strategies and risk management.

Conclusion

The journey towards scaling search and learning in reinforcement learning is complex and involves several critical components. By combining robust policy initialization, well-designed reward systems, effective search algorithms, and iterative learning processes, we can unlock the full potential of RL. The roadmap inspired by the o1 model provides a structured approach to navigating the challenges of scaling search and learning in RL. This work not only illustrates the technical foundations for reproducing models like o1 but also highlights the broader implications of integrating human-like reasoning into AI systems. Future research in areas such as hierarchical RL, model-based RL, and understanding scaling laws is essential to further expand the capabilities of RL.

Connect with me for such an In-Depth Blog on the latest Research!

Twitter: ByteMohit
GitHub: MohitGoyal09
LinkedIn: Mohit Goyal
HashNode: Mohit Goyal

References

Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Bo Wang, Shimin Li, Yunhua Zhou, Qipeng Guo, Xuanjing Huang, and Xipeng Qiu. Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective. 2412.14135v1.pdf, 2024.