Coder Legion - Recent posts tagged memoryoptimization

Part 1: What Is slots? The Python Memory Hack Nobody Taught You

Mon, 06 Jul 2026 10:35:56 +0000

There's a hidden switch in Python that can cut your object's memory by 50% — and break your code in five different ways. Here's the full story.

The Moment of Discovery

You're profiling a Python service. Memory usage is climbing. You check the heap — and see millions of small objects, each carrying a 56-byte dict overhead. For a class with just two attributes, that's absurd.

You Google. You find a word you've never seen: slots.

You try it. Memory drops. You're a genius.

Three days later, inheritance breaks. pickle fails in production. A framework you depend on throws AttributeError: 'User' object has no attribute 'dict'.

Welcome to slots.

The Problem: Python's Default Object Is a Hash Table

Every Python object, by default, stores its attributes in a dictionary:

class User:
    def __init__(self, name, age):
        self.name = name
        self.age = age

u = User("Abhishek", 40)
print(u.__dict__)  # {'name': 'Abhishek', 'age': 40}

dict is a hash table. Hash tables are flexible — you can add, remove, or rename attributes anytime. But flexibility costs memory.

For a simple object with two fields, the dict alone can use 50–60 bytes. The actual data? Maybe 16 bytes. You're paying 4x overhead for a feature you rarely use.

The Hack: Replace the Dictionary With an Array

slots tells Python: "This object will only ever have these attributes. Don't give it a dictionary. Give it a fixed array."

class User:
    __slots__ = ('name', 'age')
    
    def __init__(self, name, age):
        self.name = name
        self.age = age

u = User("Abhishek", 40)
print(hasattr(u, '__dict__'))  # False — no dictionary at all!

Instead of a hash table, Python stores name and age at fixed offsets in a C struct. No hashing. No dynamic keys. Just direct array access.

Result: Faster attribute access. Dramatically lower memory. But you lose the superpower that makes Python... Python.

For a data pipeline creating 10 million objects? That's the difference between 600 MB and 100 MB.

What You Lose (A Preview)

Here's the catch — and it's bigger than most tutorials admit:

No dynamic attributes — you can't add fields later
Multiple inheritance breaks — layout conflicts
pickle gets fragile — serialization edge cases
Frameworks may reject you — Django, SQLAlchemy, Pydantic

I cover each trap in detail in Part 2 — the architecture-level breakdown of when slots becomes a design prison.

The Modern Way: dataclasses With slots=True

Python 3.10+ gives you slots benefits without the footguns:

from dataclasses import dataclass

@dataclass(slots=True)
class User:
    name: str
    age: int

u = User("Abhishek", 40)
print(u.__dict__)  # AttributeError — still no dict, but clean syntax

You get the memory savings, the type hints, and repr/eq for free. But the constraints remain: no dynamic attributes, inheritance limits still apply.

The Real Question: Do You Actually Need It?

slots is not a default optimization. It's a specialized tool for specialized problems.

The Analogy

Think of dict as a backpack. You can put anything in it, anytime. Heavy, but flexible.

slots is a suit with tailored pockets. Exact fit, no extra weight. But try to carry something new — a water bottle, a laptop — and you're stuck.

Most days, you want the backpack. Race day? You want the suit. Know which day it is before you dress.

Bottom Line

slots is Python's hidden performance switch. It can save you gigabytes of RAM and shave milliseconds off hot loops. But flip it without understanding the cost, and you'll spend weeks debugging AttributeError in code that should work.

It's not a hack. It's a contract. Read the fine print.

Continue to Part 2: The slots Trap: When Memory Optimization Becomes a Design Bug

Part 2: The slots Trap — When Memory Optimization Becomes a Design Bug

Mon, 06 Jul 2026 10:35:33 +0000

In Part 1, we learned what slots is and why it saves memory. Now let's talk about why it can destroy your architecture — and the five traps that catch every developer who uses it without reading the fine print.

Trap 1: No Dynamic Attributes — Ever

class User:
    __slots__ = ('name', 'age')

u = User("Abhishek", 40)
u.email = "*Emails are not allowed*"  # AttributeError!

Without dict, there's nowhere to store email. This isn't a bug — it's the design. But it breaks workflows you rely on:

Debugging: Can't attach temporary attributes for inspection
Monkey-patching: Can't override behavior in tests
Frameworks: Django ORM, SQLAlchemy (some modes), Pydantic v1 expect dict

The escape hatch (that isn't one):

class User:
    __slots__ = ('name', 'age', '__dict__')  # You're doing it wrong

Now you pay for both slots and dict. The worst of both worlds.

Trap 2: Multiple Inheritance Breaks Silently

class A:
    __slots__ = ('a',)

class B:
    __slots__ = ('b',)

class C(A, B):  # TypeError: multiple bases have instance lay-out conflict
    pass

Python can't merge two fixed C arrays into one. You'd need:

class C(A, B):
    __slots__ = ('a', 'b', 'c')  # Manual layout management

One wrong field order, and you're corrupting memory offsets. This is C-level debugging in Python.

Rule: Never use slots in base classes meant for public inheritance.

Trap 3: pickle and copy Lie to You

import pickle

class Config:
    __slots__ = ('host', 'port')

c = Config()
c.host = 'localhost'
c.port = 8080

data = pickle.dumps(c)
c2 = pickle.loads(data)  # Works... until it doesn't

pickle handles slots — mostly. But add weakref for compatibility, and the serialized format changes silently. Upgrade Python versions, and old pickles may fail with cryptic errors.
copy.deepcopy has similar edge cases. The object looks the same. The behavior isn't.

Trap 4: Frameworks Reject You

You don't discover this until integration testing. By then, slots is baked into your core models.

Trap 5: Memory Savings Are Context-Dependent

slots wins at scale. For most applications, the savings don't justify the constraints. Measure first. sys.getsizeof() and tracemalloc will tell you if you actually have a problem.

When slots Is the Right Call

Use it when all of these are true:

You're creating millions of instances
The attribute set is fixed and known upfront
You control the full inheritance chain
You don't need dynamic attributes, weak references, or arbitrary
framework integration
You've profiled and confirmed dict is your actual bottleneck

That's a narrow window. Most Python code doesn't live there.

The Unified Lesson

slots isn't an optimization. It's a contract change. It trades flexibility for memory efficiency, and that trade is permanent for the class hierarchy.

The trap isn't technical — it's psychological. Developers see "memory optimization" and apply it everywhere. But Python's strength is flexibility. slots buys you C-struct memory layout at the cost of Python's dynamic nature.

Know the price before you pay it.