Coder Legion - Recent posts tagged scikit-learn

Building Decision Trees with Pandas and Scikit-learn

Fri, 14 Aug 2026 14:10:38 +0000

The Theory

So what's a Decision Tree Regressor and how does it work?
It's a regression model so it returns continuous numerical values. It gets a prediction by asking a series of yes/no questions about the chosen features, building a binary tree structure where each question splits the data further until it reaches a prediction at the leaf node.

It learns by choosing which questions to ask

So for a given feature, a "split" is a threshold value that divides the data into two groups, those above and those below.

So say we wanted to make predictions on house price and we have features like the number of rooms, the size of the land, etc. Then at each node the algorithm tries every possible split across every feature and picks the one that minimises the mean squared error (MSE) of the resulting groups i.e. the split that produces two groups where house prices within each group are as similar to each other as possible ( it sums the MSE from each set of two groups and picks the set with the lowest combined MSE).

Where,

\[\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (x_i - \bar{x})^2\]

example:

What value does the leaf node actually hold?

This is the key bit. During training, multiple houses end up at each leaf node. The predicted value stored at that leaf is simply the mean house price of all training houses that landed there.

How do we know if our model is any good?

A reasonable way to validate the model is to compare, on average, how close a prediction is to the actual value. We will use Mean Absolute Error (MAE).

\[MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i|\]

At this point you may be wondering why are we using MSE for training and MAE for validation. Can't we just pick one metric and use it for splitting the tree and validating it?

We use MSE for training because it's faster than MAE. During training, the tree has to calculate errors for millions of potential splits. Because MSE uses squared numbers, it unlocks a clever mathematical shortcut (variance reduction) that lets the computer calculate splits using a simple running tally.

\[\sum (y_i - \bar{y})^2 = \sum y_i^2 - \frac{(\sum y_i)^2}{n}\]

Look closely at the right side of the identity. There are no subtractions between individual data points and the mean. Instead, the entire formula is built out of only three components, The sum of all squared target values, the sum of all target values (which is then squared) and the count n.

The computer can keep a running tally of the sums, the squared sums, and the counts as it slides through split points. It never has to look back at individual data points to calculate the error for a group.

We validate with MAE because humans understand it: MSE leaves you with an error in "squared units" (like squared euros), which would probably raise a few eyebrows at your local bank.

Limitations to consider.

Decision Tree tends to overfit because it memorises the training data rather than learning general patterns. This may result in highly specific "rules" that only apply to historical data, causing it to struggle to generalise.

When you train a decision tree, it uses a greedy algorithm. This means at every single split, it only cares about making the absolute perfect decision for the data right in front of it at that exact moment. It doesn't look ahead, and it doesn't plan for the future.

If you don't stop it (by setting limits like max_depth), the tree will keep asking questions until it has separated every single unique case in your training set into its own tiny leaf node.

This is exactly why Ensemble Methods like XGBoost exist - it builds hundreds of shallow trees and combines them, which is far more robust. Think of a Decision Tree as the foundation concept that XGBoost is built on top of.

The Code

Learning path:

Explore the data and identify prediction target
Transform - Clean the data
Select features and Model type
Train the model - find patterns from the data
Predict and Evaluate the model
Optimise the model

Step 1: Load and Explore the data

Using Pandas to read from a csv file and return a pandas DataFrame.

import pandas as pd

housing_df = pd.read_csv('./datasets/housing_data.csv')

print(f"Statistical Summary \n {housing_df.describe()}")

print(f"Dataset Features \n {housing_df.columns}")

Output

Statistical Summary

Rooms Price ... Longtitude

Propertycount

count 13580.000000 1.358000e+04 ... 13580.000000 13580.000000

mean 2.937997 1.075684e+06 ... 144.995216 7454.417378

std 0.955748 6.393107e+05 ... 0.103916 4378.581772

min 1.000000 8.500000e+04 ... 144.431810 249.000000

25% 2.000000 6.500000e+05 ... 144.929600 4380.000000

50% 3.000000 9.030000e+05 ... 145.000100 6555.000000

75% 3.000000 1.330000e+06 ... 145.058305 10331.000000

max 10.000000 9.000000e+06 ... 145.526350 21650.000000

Output

Dataset Features

Index(['Suburb', 'Address', 'Rooms', 'Type', 'Price', 'Method', 'SellerG', 'Date', 'Distance', 'Postcode', 'Bedroom2', 'Bathroom', 'Car', 'Landsize', 'BuildingArea', 'YearBuilt', 'CouncilArea', 'Lattitude', 'Longtitude', 'Regionname', 'Propertycount'], dtype='object')

Step 2: Transform the data

(i) Remove any rows that have NaN (missing) value ( axis: 0 -> rows, 1 -> columns )

clean_housing_df = housing_df.dropna(axis=0)

(ii) Select the prediction target 'y' and the features 'X'.

y = clean_housing_df.Price
features = ['Rooms', 'Bathroom', 'Landsize', 'Longtitude', 'Lattitude']
X = clean_housing_df[features]

(iii) Split the data into a training set and validation set

from sklearn.model_selection import train_test_split

train_X, val_X, train_y, val_y = train_test_split(X, y, random_state=1)

Step 3: Select Model type and train.

(i) Scikit-learn Decision Tree Regressor ( random state = 1. If the split results in a tie then breaking is deterministic )

from sklearn.tree import DecisionTreeRegressor

housing_regressor_model = DecisionTreeRegressor(random_state=1)

(ii) Fit the model

housing_regressor_model.fit(train_X, train_y)

Step 4: Predict

predicted_home_prices = housing_regressor_model.predict(val_X)

print(predicted_home_prices)

Output

[ 503000. 1857000. 760000. ... 4200000. 800000. 1785000.]

Step 5: Validate

Using the Mean Absolute Error metric as discussed above.

from sklearn.metrics import mean_absolute_error

mae = mean_absolute_error(val_y, predicted_home_prices)
print(mae)

Output

251698.44673983214

Step 6: Optimise

How can we minimise the MAE?

By using a process known as Hyperparameter Optimisation .
We can tune our decision tree using the max_leaf_nodes parameter, which is passed to the model before training. This setting allows us to specify the upper limit of leaf nodes the model is allowed to create, helping us control the balance between underfitting (too few leaves) and overfitting (too many leaves).

Let see if we can find a value for max_leaf_nodes that minimise the MAE.

mae_scores = []
candidate_levels = [10, 100, 200, 400, 500, 1000, 5000]

for max_leafs in candidate_levels:
    # Initialize and fit the model with the current candidate size
    model = DecisionTreeRegressor(max_leaf_nodes=max_leafs, random_state=0)
    model.fit(train_X, train_y)
    
    # Predict and evaluate
    preds_val = model.predict(val_X)
    mae = mean_absolute_error(val_y, preds_val)
    
    # Store the result as a sublist: [error, node_count]
    mae_scores.append([mae, max_leafs])
    print(f"Max leaf nodes: {max_leafs} | MAE: {mae:.2f}")

# Find the sublist with the lowest MAE score
smallest_mae = min(mae_scores, key=lambda x: x[0])
print(f"\nThe optimal choice for max_leaf_nodes is: {smallest_mae[1]} (MAE: {smallest_mae[0]:.2f})")

Output

Max leaf nodes: 10 | MAE: 320570.21

Max leaf nodes: 100 | MAE: 256533.25

Max leaf nodes: 200 | MAE: 244453.42

Max leaf nodes: 400 | MAE: 243238.90

Max leaf nodes: 500 | MAE: 241920.26

Max leaf nodes: 1000 | MAE: 243230.84

Max leaf nodes: 5000 | MAE: 256050.02

The optimal choice for max_leaf_nodes is: 500 (MAE: 241920.26)

Weather Service Project (Part 3): Predicting the Future with AI and OpenWeatherMap

Thu, 22 Jan 2026 06:49:58 +0000

In the first part of this series, we set up the backbone of our global weather service, collecting raw data using Python and GitHub Actions. Then, in Part 2, we transformed that raw data into a beautiful, interactive dashboard, leveraging GitHub Pages/Netlify, JavaScript, PapaParse.js, and Chart.js.

Now, it's time for the grand finale: adding predictive power to our Weather Service. We'll explore how to augment our historical data visualization with actual forecasts. This installment focuses on a dual approach: integrating an official, reliable forecast from a professional service (OpenWeatherMap) and, more excitingly, building and training our very own simple AI model (Linear Regression) to predict tomorrow's weather based on the historical data we've meticulously collected. Finally, we'll visualize both forecasts on our dashboard, allowing for a direct comparison and a real-world test of our AI's accuracy.

Let's turn our data into a crystal ball!

The Predictive Core: OpenWeatherMap and Our Custom AI

The goal for this predictive functionality was twofold:

Official Forecast: Obtain a reliable, multi-day forecast from a professional weather service (OpenWeatherMap - OWM).
Custom AI Prediction: Create our own simple AI model (Linear Regression) trained on the historical data we've collected, to predict the next day's weather.
Visualization & Comparison: Display and compare both forecasts to gauge the accuracy and performance of our custom AI model.

1. ⚙️ Backend Logic: `read_weather.py` Gets Smarter

Our read_weather.py script, previously responsible for data collection, now expands its role to gather data from both OWM and our historical archives, consolidating everything into a single predicciones.json file.

Step 1: Fetching OpenWeatherMap's 5-Day Forecast

We decided that in addition to our 1-day AI prediction, a 5-day forecast from OWM would provide valuable context.

API Endpoint: We opted for the free data/2.5/forecast API (since OneCall 3.0 required a payment method).
Data Processing: This API returns data in 3-hour blocks. We had to add Python logic to:
- Iterate over the list of ~40 forecasts.
- Group them by day (ignoring the current day).
- For each of the next 5 days, calculate the maximum, minimum, and average temperature from all 3-hour blocks within that day.
Result: A list of 5 objects (one per day) containing OWM's max, min, and average temperature predictions.

Step 2: Implementing Our AI Model (1-Day Prediction)

This is the core of our "homemade AI." For each city:

Data Loading: We used pandas to read the city's historical CSV file (e.g., datos/Madrid.csv).
Feature Engineering: Since we had multiple readings per day, the most crucial step was transforming this data:
- Resampling: We used df.resample('D') from pandas to group data by day, calculating the actual daily aggregates (e.g., temp_max, temp_min, avg_temp, avg_humidity).
- Feature Creation (X): We created new "shifted" columns (.shift(1)) so that each row (representing a day) contained the previous day's data (e.g., temp_max_lag1, avg_humidity_lag1). We also added day_of_year to capture seasonality.
- Target Creation (y): We defined what we wanted to predict (e.g., the actual temp_max of the current day).

Training 3 Models: Instead of one, we trained three independent Linear Regression models (scikit-learn):
- model_max: Trained with y = df_clean['temp_max'].
- model_min: Trained with y = df_clean['temp_min'].
- model_avg: Trained with y = df_clean['avg_temp'].
Prediction:
- We took the last row of aggregated data (representing "today's" data).
- Fed this data to the 3 models to predict "tomorrow's" values.
- We included a safeguard (MIN_RECORDS_FOR_IA = 10) so the model only attempts to predict if it has sufficient historical data (e.g., 10 clean days).

Step 3: Consolidate and Save

The script combines the results from Steps 1 and 2 into a JSON structure and saves it to predicciones.json:

{
  "Madrid": {
    "pred_owm_5day": [ 
      { "date": "...", "max": 15.0, "min": 10.0, "avg": 12.5 }, 
      ... (5 days) ...
    ],
    "pred_ia": {
      "max": 14.8,
      "min": 7.5,
      "avg": 11.2,
      "records": 120
    }
  },
  "A Coruña": {
      ...
      "pred_ia": { "max": null, "min": null, "avg": null, "records": 9 } // Example of insufficient data
  }
}

2. Frontend Logic: `index.html` Visualizes the Future

The frontend is responsible for loading this predicciones.json file and presenting it in a visually appealing and informative way.

Step 1: Data Loading

loadPredictions(): We created a new async function that runs once during initialization (before updateDashboard).
allPredictionsCache: This function loads predicciones.json and saves it into this new global variable so that all visualization functions have access to it.

Step 2: Visualization in the "Super-Cards" (KPIs)

We wanted a direct and clear comparison.

OWM 5-Day Forecast:
- We created a helper function buildForecastHTML().
- This function takes the pred_owm_5day list and generates an HTML block with a list of the 5 days and their max/min temperatures (e.g., "Sat, Nov 9: 15.1°C / 10.0°C").
AI 1-Day Forecast (Comparison):
- We created a second helper function buildIAForecastHTML().
- This function takes the pred_ia object and the first day of the OWM forecast (pred_owm_5day[0]).
- Comparison Logic: For max, min, and average temperatures, it displays the AI's value and then, next to it, the difference from OWM.
- Visual Impact: The difference is colored red (if our AI predicts warmer) or blue (if it predicts cooler), giving us an immediate visual cue of our model's deviation.
- It also handles the "Insufficient Data" case (${ia_preds.records}/${MIN_RECORDS_FOR_IA}).
updateKPIs(): The card template was modified to call these two new functions, displaying both forecast blocks.

Step 3: Visualization in the Charts

We wanted the forecasts to be integrated directly into the existing charts.

Evolution Chart (Dotted Line):
- In updateChart(), we added a new dataset for each city.
- This dataset uses the average of the OWM prediction (pred_owm_5day).
- We applied the style borderDash: [5, 5] to draw it as a dotted line.
- We "stitched" the beginning of this line to the last real data point to make it appear as a seamless continuation.
Variation Chart (Striped Bars):
- In updateVariationChart(), we added another dataset for each city.
- The y data for this set is day.max - day.min (the variation) from the OWM forecast.
- For styling, we created a helper function createStripedPattern() that draws a striped pattern on a canvas.
- We used this pattern as the backgroundColor for the forecast bars, differentiating them from the solid bars of real data.

Conclusion (Part 3)

With this final installment, our Weather Service project is complete! We've successfully integrated both professional 5-day forecasts from OpenWeatherMap and a custom 1-day AI prediction model, all powered by our collected historical data. The frontend now provides a rich, interactive experience that not only visualizes past weather but also offers a glimpse into the future, complete with a comparative analysis of our AI's performance.

This journey has covered everything from backend data collection, automation with GitHub Actions, static site hosting with Netlify, to dynamic frontend development with vanilla JavaScript, advanced data parsing with PapaParse.js, interactive charting with Chart.js, and finally, dipping our toes into Machine Learning for predictive analytics.

We've built a robust, serverless, and insightful application entirely on free services. The possibilities for expansion (e.g., more complex ML models, different data sources, user accounts) are endless, but for now, we have a fully functional weather oracle!

References and Links of Interest:

Complete Web Service: You can see the final project in action here: https://datalaria.com/apps/weather/
Project GitHub Repository: Explore the source code and project structure in my repository: https://github.com/Dalaez/app_weather
OpenWeatherMap API: https://openweathermap.org/api
Pandas: Python Data Analysis Library: https://pandas.pydata.org/
Scikit-learn: Machine Learning in Python: https://scikit-learn.org/
PapaParse.js: Fast in-browser CSV parser for JavaScript: https://www.papaparse.com/

Coder Legion - Recent posts tagged scikit-learn

Building Decision Trees with Pandas and Scikit-learn

The Theory

The Code

Step 1: Load and Explore the data

Step 2: Transform the data

Step 3: Select Model type and train.

Step 4: Predict

Step 5: Validate

Step 6: Optimise

Weather Service Project (Part 3): Predicting the Future with AI and OpenWeatherMap

The Predictive Core: OpenWeatherMap and Our Custom AI

1. ⚙️ Backend Logic: read_weather.py Gets Smarter

Step 1: Fetching OpenWeatherMap's 5-Day Forecast

Step 2: Implementing Our AI Model (1-Day Prediction)

Step 3: Consolidate and Save

2. Frontend Logic: index.html Visualizes the Future

Step 1: Data Loading

Step 2: Visualization in the "Super-Cards" (KPIs)

Step 3: Visualization in the Charts

Conclusion (Part 3)

References and Links of Interest:

1. ⚙️ Backend Logic: `read_weather.py` Gets Smarter

2. Frontend Logic: `index.html` Visualizes the Future