Coder Legion - Recent posts tagged serialization

I Built a Serializer That Refuses to Own Your Data

Mon, 28 Sep 2026 23:36:42 +0000

I Built a Serializer That Refuses to Own Your Data

How The Singularity Workshop turned a recurring binary-IO problem into a tiny .NET boundary—and why the smallest part of the design may be the most important part.

There is a particular kind of software problem that appears harmless at first.

You have some state.

You need to turn it into bytes.

So you write a serializer.

Then another part of the application needs bytes, so you add another serializer.

Then a cache needs them.

Then a manifest needs them.

Then a transport layer needs them.

Then somebody decides JSON is better here, binary is better there, and suddenly the architecture has six different opinions about what a piece of data is.

That was the problem I was running into while building The Singularity Workshop.

The interesting part is that I didn't actually need a giant serialization framework.

I needed a boundary.

So I built one.

It is called:

TheSingularityWorkshop.FSM_Serialization

And it is deliberately small.

That is the point.

The Problem Wasn't Turning Objects Into Bytes

Serialization is usually described as a mechanical operation:

object → serializer → bytes

That description hides something important.

An object has meaning.

Bytes do not.

Bytes are a representation of meaning.

That distinction becomes increasingly important as an architecture grows.

The Workshop ecosystem is being built around several deliberately separated responsibilities:

FSM_API
    │
    │ behavior and state
    ▼
FSM_COS
    │
    │ composition and runtime assembly
    ▼
FSM_Serialization
    │
    │ representation
    ▼
storage / transport / host

I didn't want FSM_COS to become a serializer.

I didn't want a MicroBundle to have to know how a filesystem works.

I didn't want a database abstraction hiding inside a representation layer.

And I definitely didn't want every higher-level system inventing its own binary stream interface.

So I extracted the smallest reusable boundary I could identify.

That boundary became FSM_Serialization.

The First Design Decision

The most important rule in the package is also the simplest:

The type that owns the meaning should own the representation rules.

FSM_Serialization provides the machinery for crossing the byte boundary.

It does not decide what your data means.

That means the package doesn't contain some enormous reflection-based object graph engine that looks at your class and decides what should be serialized.

Instead, the type explicitly says:

public interface IBinaryPackable
{
    void Pack(IBinaryStream stream);
}

And the reverse direction is equally explicit:

public interface IBinaryUnpackable
{
    void Unpack(IBinaryStream stream);
}

If a type needs both:

public interface IBinarySerializable :
    IBinaryPackable,
    IBinaryUnpackable
{
}

That's the entire conceptual model.

No magic.

No global serializer registry.

No reflection-based guesswork.

No framework deciding that a private field suddenly became part of your persistent format.

The domain owns that decision.

Why I Didn't Just Use `System.IO.Stream`

This is probably the first question a .NET developer will ask.

And honestly, it is a good question.

.NET already has System.IO.Stream.

Why introduce another interface?

Because I wasn't trying to replace Stream.

I was trying to establish a package-owned contract.

The current abstraction is:

public interface IBinaryStream : IDisposable
{
    bool CanRead { get; }
    bool CanWrite { get; }

    long Position { get; set; }
    long Length { get; }

    int Read(Span<byte> buffer);
    void Write(ReadOnlySpan<byte> buffer);
    void Flush();
}

Notice what isn't there.

There is no:

filename;
filesystem;
database;
socket;
HTTP request;
serializer registry;
schema;
domain metadata;
application configuration.

The interface only knows about the byte boundary.

That makes it possible to write serialization code against one very small abstraction while letting the host decide what is actually underneath it.

The Two Directions Matter

There is another small decision here that I think becomes important in larger systems.

Packing and unpacking are separate capabilities.

A type that produces a representation does not necessarily need to consume one.

Likewise, a type may consume a representation without being responsible for producing it.

So the package doesn't force everything into one giant:

Serialize()
Deserialize()

method.

Instead:

                 DOMAIN TYPE
                     │
          ┌──────────┴──────────┐
          │                     │
        Pack                  Unpack
          │                     │
          ▼                     ▲
     representation          representation
          │                     │
          └──────────┬──────────┘
                     ▼
                byte stream

That separation costs almost nothing.

But architecturally, it keeps the ownership visible.

The Smallest Possible Example

Let's make this concrete.

Imagine a domain object containing one integer.

That's intentionally boring.

Boring is useful here because it lets us see exactly where the responsibility lives.

using System.Buffers.Binary;
using TheSingularityWorkshop.FSM_Serialization;

public sealed class ExampleState : IBinarySerializable
{
    public int Value { get; private set; }

    public ExampleState()
    {
    }

    public ExampleState(int value)
    {
        Value = value;
    }

    public void Pack(IBinaryStream stream)
    {
        Span<byte> buffer = stackalloc byte[sizeof(int)];

        BinaryPrimitives.WriteInt32LittleEndian(
            buffer,
            Value);

        stream.Write(buffer);
    }

    public void Unpack(IBinaryStream stream)
    {
        Span<byte> buffer = stackalloc byte[sizeof(int)];

        if (stream.Read(buffer) != buffer.Length)
            throw new EndOfStreamException();

        Value = BinaryPrimitives.ReadInt32LittleEndian(buffer);
    }
}

There is no serializer object.

There is no configuration object.

There is no registration step.

The object itself defines its representation.

That is intentional.

Now Let's Actually Serialize Something

The package includes MemoryBinaryStream, which is exactly what it sounds like: an in-memory implementation of IBinaryStream.

That makes testing and temporary representations extremely simple.

using var stream = new MemoryBinaryStream();

var original = new ExampleState(42);

original.Pack(stream);

stream.Position = 0;

var restored = new ExampleState();

restored.Unpack(stream);

Console.WriteLine(restored.Value);

Output:

That's it.

The complete flow is:

ExampleState
     │
     │ Pack
     ▼
MemoryBinaryStream
     │
     │ bytes
     ▼
Position = 0
     │
     │ Unpack
     ▼
ExampleState

The package supplied the boundary.

The domain supplied the meaning.

Installation

If you want to try it yourself, the current published alpha is:

dotnet add package TheSingularityWorkshop.FSM_Serialization --version 0.1.0-alpha.2

Or:

<PackageReference
    Include="TheSingularityWorkshop.FSM_Serialization"
    Version="0.1.0-alpha.2" />

The current package targets .NET 8.

This is intentionally an alpha release.

That doesn't mean the contracts are casual.

Quite the opposite.

A binary representation is one of those things where small assumptions can become enormous compatibility problems later.

So I'm treating the package as alpha while treating its architectural boundaries seriously.

What About Real Files?

This is where StreamBinaryStream enters.

The package includes an adapter around an ordinary .NET Stream:

using var file = File.OpenRead("state.bin");
using var stream = new StreamBinaryStream(file);

Now the same domain serialization code can operate against the package abstraction without knowing that a file exists.

The relationship is:

ExampleState
     │
     ▼
IBinaryStream
     │
     ▼
StreamBinaryStream
     │
     ▼
System.IO.Stream
     │
     ▼
file / memory / whatever the host provides

That is a very different architecture from:

ExampleState
     │
     ▼
ExampleStateSerializer
     │
     ▼
FileSystemSerializerManager
     │
     ▼
ApplicationStorageService

The latter might be useful somewhere.

It just doesn't belong in this package.

The Stream Adapter Is Supposed to Be Boring

This is one of those cases where boring code is a compliment.

StreamBinaryStream essentially delegates:

public int Read(Span<byte> buffer)
    => _stream.Read(buffer);

public void Write(ReadOnlySpan<byte> buffer)
    => _stream.Write(buffer);

public void Flush()
    => _stream.Flush();

That's not a weakness.

It's the point.

The adapter is not supposed to invent behavior.

It establishes a stable contract between two layers.

Then I Ran Into a Very Important Detail

A byte stream is not the same thing as an array.

This matters:

int count = stream.Read(buffer);

You cannot blindly assume:

count == buffer.Length

So a fixed-width decoder in FSM_Serialization explicitly checks:

if (stream.Read(buffer) != buffer.Length)
    throw new EndOfStreamException();

That may look like a tiny implementation detail.

It isn't.

It establishes an architectural rule:

Never interpret an incomplete representation as a complete value.

The serializer shouldn't quietly manufacture missing data.

If the representation is incomplete, the domain implementation gets to decide what that means.

In the current examples, that means rejecting it.

Serialization Is Not Storage

This distinction became increasingly important as the Workshop architecture grew.

Consider these questions:

Serialization

How are these values represented as bytes?

Storage

Where do those bytes live?

Transport

How do those bytes move between boundaries?

Those are three different questions.

So:

                 SEMANTIC DATA
                      │
                      ▼
                representation
                      │
                      ▼
              FSM_Serialization
                      │
              ┌───────┴───────┐
              ▼               ▼
           storage         transport

A database is not a serializer.

A filesystem is not a serializer.

An HTTP endpoint is not a serializer.

A cache is not a serializer.

They may all consume serialized data.

That doesn't make them the same architectural layer.

This is exactly the kind of distinction that seems unnecessary when an application is small and becomes incredibly valuable when an ecosystem starts growing.

Why Binary?

I am not claiming that binary is universally superior to JSON.

It isn't.

JSON is fantastic for many things.

Text is fantastic for many things.

Human-readable configuration is valuable.

Debugging matters.

Authoring matters.

Interoperability with external systems matters.

The reason I created this package is narrower.

There are situations where I want an explicit, compact, byte-oriented representation.

For example:

runtime state;
manifests;
dependency metadata;
compact configuration;
cache entries;
identity records;
version records;
lineage metadata;
persistence;
transport payloads.

The important thing is that the semantic model shouldn't have to become identical to its wire representation.

Deterministic Does Not Mean Magical

The package's NuGet description talks about deterministic binary serialization.

There is an important qualification.

Implementing:

IBinaryPackable

does not magically make your representation deterministic.

You have to define deterministic rules.

For example, suppose your object contains a collection that is semantically unordered.

If you serialize it in whatever order happens to be present at runtime, two semantically identical objects could produce different byte sequences.

If deterministic bytes matter, the domain needs something like:

semantic collection
        │
        ▼
canonical ordering
        │
        ▼
explicit representation
        │
        ▼
deterministic bytes

That distinction matters if bytes are later used for:

caching;
content identity;
synchronization;
comparison;
reproducible builds;
lineage;
integrity checks.

Again, the package supplies the boundary.

It does not pretend to know your domain's canonical ordering rules.

Versioning Gets Interesting

This is another reason I didn't want a magical object serializer.

Once bytes leave your process, they can outlive the code that created them.

Imagine:

Application v1
      │
      ▼
    bytes
      │
      │ time passes
      ▼
Application v2

What happens?

That is a representation contract question.

A durable binary format may need to distinguish between:

semantic object version
format version
dependency version

Those are not necessarily the same thing.

Your semantic object can change while the representation remains compatible.

Your representation can change while the semantic concept remains the same.

A dependency can change independently of both.

FSM_Serialization intentionally doesn't impose one universal versioning strategy.

Different domains need different compatibility policies.

What it does is make the boundary explicit enough that you can actually have that conversation.

The Package Came From a Real Application

This is probably the most important part of the story.

FSM_Serialization wasn't created because I thought:

"The world needs another serialization library."

There are already plenty of serialization libraries.

The package emerged from building the Workshop itself.

The development pattern was:

working application
       │
       │ repeated infrastructure appears
       ▼
explicit contract
       │
       │ extract
       ▼
reusable package
       │
       │ consume again
       ▼
working application

That distinction matters.

I wasn't trying to abstract everything before I had a reason to abstract it.

The application became the proving ground.

Once the boundary became clear, I extracted it.

That is a much healthier way to create infrastructure.

And This Is Where FSM_COS Enters the Story

The larger ecosystem is becoming increasingly compositional.

FSM_COS is concerned with questions like:

What runtime composition was requested?

Which MicroBundles are required?

What dependencies are reachable?

How do those capabilities reconcile?

When is the composition stable enough to hand to a host?

FSM_Serialization answers a different question:

How do we represent the values that need to cross a byte boundary?

That means a future conceptual flow can look something like:

Runtime Manifest
      │
      ▼
    FSM_COS
      │
      │ composition
      ▼
Runtime Assembly
      │
      │ representation
      ▼
FSM_Serialization
      │
      ▼
     bytes

And then those bytes might eventually go somewhere else:

                 FSM_Serialization
                        │
             ┌──────────┼──────────┐
             ▼          ▼          ▼
          storage    transport    cache
             │          │          │
             └──────────┼──────────┘
                        ▼
                      host

The serializer doesn't become FSM_COS.

FSM_COS doesn't become a serializer.

That separation is exactly what I'm trying to preserve.

The Tests Tell the Same Story

The package currently has a deliberately small test surface.

It verifies things such as:

an empty MemoryBinaryStream is readable and writable;
bytes survive a write/read round trip;
existing byte arrays can initialize a memory stream;
null input is rejected;
end-of-input returns zero;
positioning and overwriting work;
StreamBinaryStream delegates correctly;
null wrapped streams are rejected;
writes reach the underlying stream;
IBinaryPackable and IBinaryUnpackable can round-trip;
IBinarySerializable combines both contracts;
incomplete binary input is rejected.

One of the tests is essentially the entire architectural idea in executable form:

var original = new TestValue(123456789);

using var stream = new MemoryBinaryStream();

original.Pack(stream);

stream.Position = 0;

var restored = new TestValue();

restored.Unpack(stream);

Assert.Equal(original.Value, restored.Value);

Tests aren't merely protecting the implementation here.

They are defining the boundary.

I Also Benchmarked It

Because eventually somebody is going to ask:

"Okay, but what does that abstraction cost?"

Good.

Let's measure it.

The companion benchmark project uses BenchmarkDotNet against .NET 8 and exercises payloads from 16 bytes through 100,000 bytes.

The interesting result isn't that FSM_Serialization magically beats MemoryStream.

It doesn't.

That's not what I was trying to prove.

The useful result is that the abstraction can remain extremely close to the underlying primitive for substantial workloads.

At 100,000 bytes, the benchmark measured approximately:

MemoryStream write:
39,894.85 ns

MemoryBinaryStream write:
40,005.59 ns

That's effectively the same scale.

Likewise, the adapter path was approximately:

MemoryStream read:
2,272.47 ns

StreamBinaryStream read:
2,298.40 ns

Again, very close.

And the end-to-end IBinarySerializable benchmark measured:

16 bytes       64.08 ns
1,024 bytes   207.47 ns
10,000 bytes  1,481.61 ns
100,000 bytes 60,152.74 ns

But there is an important qualification.

The benchmark measures the whole operation, including things like creating the test value, creating the stream, packing, rewinding, allocating the destination payload, and unpacking.

So I'm not going to pretend those numbers represent some universal "serialization speed."

They represent a measured workload.

That's much more useful.

The Benchmark Question I Actually Care About

The interesting engineering question isn't:

"Is my interface faster than MemoryStream?"

The interface isn't supposed to be the star of the show.

The question is:

Can I establish a useful architectural boundary without making the underlying operation prohibitively expensive?

The current measurements suggest that, for these workloads, the answer is yes.

And that's exactly where I want infrastructure to live.

The abstraction should be visible architecturally and nearly invisible computationally.

What I Deliberately Did Not Build

This may be the most important section of the entire package.

FSM_Serialization should not become:

an FSM engine;
a game framework;
a GUI framework;
a browser framework;
a Unity integration layer;
a database abstraction;
a filesystem framework;
an HTTP client;
an application configuration system;
a MicroBundle implementation;
an Experience runtime;
a reflection-driven object graph serializer;
a global serializer registry.

Those things may all be useful.

They just aren't this thing.

That restraint is part of the architecture.

A small package becomes powerful when other systems can depend on it without having to inherit the systems it was never supposed to understand.

So What Would You Actually Use It For?

If you're building a .NET application and you have a domain type whose state needs to cross a byte-oriented boundary, the starting point is incredibly small.

Install the package:

dotnet add package TheSingularityWorkshop.FSM_Serialization --version 0.1.0-alpha.2

Define your representation:

public sealed class MyState : IBinarySerializable
{
    public int Id { get; set; }

    public void Pack(IBinaryStream stream)
    {
        Span<byte> buffer = stackalloc byte[sizeof(int)];

        BinaryPrimitives.WriteInt32LittleEndian(
            buffer,
            Id);

        stream.Write(buffer);
    }

    public void Unpack(IBinaryStream stream)
    {
        Span<byte> buffer = stackalloc byte[sizeof(int)];

        if (stream.Read(buffer) != buffer.Length)
            throw new EndOfStreamException();

        Id = BinaryPrimitives.ReadInt32LittleEndian(buffer);
    }
}

Use memory when you need a temporary representation:

using var stream = new MemoryBinaryStream();

state.Pack(stream);

byte[] bytes = stream.ToArray();

Use a normal .NET stream when the host provides one:

using var file = File.Create("state.bin");
using var stream = new StreamBinaryStream(file);

state.Pack(stream);

And that's basically the entire learning curve.

The difficult part isn't learning the API.

The difficult part is deciding what your domain actually means by its representation.

I think that's exactly where the difficult part belongs.

A Serializer Shouldn't Become Your Architecture

This is the idea that kept resurfacing while I built the package.

It's very easy for infrastructure to become the thing it was supposed to merely support.

A serializer starts storing metadata.

Then it starts deciding schemas.

Then it starts managing versions.

Then it starts talking to databases.

Then it starts managing transports.

Then it starts discovering domain types.

Eventually your "serialization library" is quietly running your architecture.

I wanted the opposite.

I wanted something I could put underneath the architecture and forget about until I actually needed to cross the boundary.

So the package can be reduced to one equation:

semantic state
      +
explicit representation rules
      +
byte boundary
      =
portable representation

Not:

object
  +
magic serializer
  =
architecture

That's the difference.

The Boundary I Want to Preserve

The entire package can ultimately be reduced to this:

                 DOMAIN
                   │
          owns meaning and rules
                   │
                   ▼
          Pack / Unpack contract
                   │
                   ▼
          FSM_Serialization
                   │
              byte boundary
                   │
        ┌──────────┴──────────┐
        ▼                     ▼
     storage               transport
        │                     │
        └──────────┬──────────┘
                   ▼
                  HOST
                   │
             manifestation

The serializer owns the boundary.

The domain owns the meaning.

The host owns the destination.

That is the architecture.

Why I Think Small Infrastructure Is Worth Talking About

There is a tendency in software writing to focus on the enormous systems.

The massive framework.

The clever algorithm.

The huge benchmark.

The complicated application.

But some of the most consequential architectural decisions are tiny.

An interface.

A boundary.

A decision about ownership.

A decision about what a component doesn't know.

FSM_Serialization is one of those experiments.

It isn't trying to replace every serialization technology in .NET.

It isn't trying to declare that binary is the future.

It is much more modest:

I needed a clean byte boundary for a growing compositional architecture, so I built the smallest one I could.

Then I tested it.

Then I benchmarked it.

Then I packaged it.

And now I'm putting it out there so somebody else can kick it.

You Don't Have to Take My Word for It

That's ultimately why I publish these projects.

Install it.

Implement one type.

Serialize something you actually care about.

Put it under your own workload.

Decide whether the ownership model makes sense to you.

Measure it.

Break it.

Tell me where the boundary doesn't hold.

That's much more interesting to me than somebody simply agreeing with the architecture.

The package is available now as an alpha.

And if you find a place where this tiny boundary lets you remove a surprisingly large amount of architectural mess, I'd genuinely like to hear about it.

Support Our Work:

Patreon Page:
https://www.patreon.com/c/TheSingularityWorkshop

Support Us (PayPal Donation):
https://www.paypal.com/donate/?hosted_button_id=3Z7263LCQMV9J

We'd love to hear your thoughts! Please Like this post, Love the code, and Share your feedback in the comments.

phpser: a fast, secure binary serializer for PHP cache workloads

Tue, 02 Jun 2026 19:01:31 +0000

I've reached for igbinary on nearly every PHP project I've shipped in the last decade. It's smaller and faster than PHP's native serialize(), it's stable, and it has been the obvious default for so long that reaching for it stopped being a decision.

So phpser started as curiosity, not a complaint. igbinary is good. Could a serializer built specifically for cache workloads do better?

I wanted two things from it. It should be fast on the shapes a cache actually holds, where a value is decoded far more often than it's encoded. And it should be safe to decode bytes from a store an attacker might reach, because unserialize() on untrusted input is one of PHP's oldest exploit primitives. igbinary gives you the speed; the safety you bolt on yourself. phpser builds in both.

On the shapes that matter for caches it encodes 10 to 70% faster than igbinary and decodes 12 to 75% faster, with packed numeric data also 65% smaller on the wire. Its signed mode refuses to decode any payload that wasn't produced with your key, so a poisoned cache entry never reaches the code that builds objects. The rest of this post is how it gets both.

Why a serializer built for caches?

Because igbinary optimizes for the general case, and a cache is not the general case, on two axes.

The first is the read/write asymmetry. A PHP cache pays decode cost on every single read. Encode happens once, when you write the value; decode happens every time anything reads it back. For a read-heavy cache that ratio is easily 100 to 1. igbinary, like most general serializers, balances the two sides. A cache serializer shouldn't.

The second is trust. The thing reading those bytes back is often reading from redis, memcached, a file, or a cookie, any of which an attacker may be able to write to. A general serializer treats decode as a pure data operation. A cache serializer has to treat it as a trust boundary.

igbinary is still the right default for general use. I went looking for the specific shapes where a cache-focused design could pull ahead, and there are three that show up everywhere in real PHP backends:

Packed numeric arrays. range(0, 999), ID lists, analytics buckets, sensor readings.
Deep-nested structures. Trees, recursive config, nested document structures.
Same-class object batches. Laravel queue payloads, cached Eloquent models, any array of a few hundred identical-shape DTOs.

Designing the format for the reader

The performance half of phpser borrows an instinct from Rust's rkyv. rkyv's pitch is that deserialization should be nearly free, because the writer already laid the bytes out the way the reader needs them. You don't parse an rkyv archive so much as point at it.

phpser isn't zero-copy, and I want to be precise about that before the comparison runs away. PHP values are refcounted zvals with owned hashtables; you can't hand PHP a pointer into a cache buffer and call it an array. phpser does a real decode pass and builds real zvals. It's not rkyv.

What transferred is the instinct, not the mechanism. rkyv made me stop thinking about the wire format as a neutral container and start thinking about it as a set of instructions to the reader. If the writer knows something that saves the reader work, the writer should record it, even when that makes encoding a little more complex. Once you adopt that lens, a set of concrete decisions falls out of it.

A string dictionary, and an intern that survives decode

The honest starting point: a front-loaded string dictionary isn't novel. igbinary already does this, it calls them compact_strings. Both serializers emit each distinct string once and reference it afterward, so the property name "created_at" repeated across a thousand cached rows costs one copy, not a thousand.

The dictionary isn't where the win is. The win is on the decode side, and it's the most direct application of the design-for-the-reader rule.

When phpser decodes a dictionary string the first time, it allocates a zend_string. Every later reference to that same dictionary index doesn't allocate; it bumps the refcount on the one already built. A thousand rows that all carry the key "user_id" produce exactly one string allocation and 999 refcount increments. PHP's own machinery is built for exactly this, interned strings are shared by refcount throughout the engine, so phpser isn't fighting the runtime, it's leaning on it.

Fast to encode, too

Designing for the reader could have meant a slow writer. It doesn't, because of two encoder choices, and the result is that phpser encodes faster than igbinary on every shape I test.

The first is the intern cache. phpser keeps an open-addressed zend_string*-to-slot hash, grown without eviction. Before hashing a string's bytes, it checks pointer identity: PHP interns string literals, so the "id" in row 1 and the "id" in row 900 are usually the same pointer and resolve with no byte work at all. Just as important, a unique value string, a name, an email, a SKU, takes a single-probe miss instead of a linear scan. The per-value dedup lookup stays off the critical path even on payloads full of strings that never repeat.

The second is objects. Encoding a PHP object the obvious way calls get_properties, which materializes a properties hashtable even for a plain object whose layout is fixed and known. For a batch of a few hundred DTOs that's hundreds of throwaway hashtables. phpser serializes a plain object straight from its declared property slots and skips the hashtable, the way native serialize() does. PHP 8.4 lazy objects fall back to get_properties, because their initializer has to run first.

Tagged scalar runs, and building the array in place

Two more decisions, on the decode side, are what make the packed-numeric numbers as large as they are.

The first is tagged scalar runs. igbinary encodes [1, 2, 3, ...] as a sequence of tagged values: a type tag and a varint, a thousand times over. phpser detects a uniform run and emits one PACKED_LONGS header plus the thousand integers as raw zigzag varints, no per-element tag. Decode becomes one tight loop with zero tag dispatch.

The second is building the hashtable in place. When the wire format says PACKED_LONGS of length N, the decoder knows the final size before it reads a single element. So it allocates the array once with zend_new_array(N) and writes the values directly into PHP 8's packed arPacked storage with ZVAL_* macros. That skips N calls to zend_hash_next_index_insert, and with them N hash computations, N capacity checks, and the incremental table growth that a naive decoder pays as it discovers the array's size one element at a time. The writer recorded the size so the reader could allocate once and fill, which is the rkyv instinct applied as far as a non-zero-copy format can take it.

The benchmarks

Here is the full shape-by-shape comparison, run on my machine, against igbinary. The bench harness (bench.php in the repo) round-trips every shape for correctness first, then times encode and decode separately, because decode is the number that matters for a cache.

Methodology: phpser 0.1.2, PHP 8.4.22-dev NTS, release build (not a debug or ASan build, which would inflate everything 2 to 5x), igbinary 3.2.17RC1, Intel Core i9-13950HX. 1,000 iterations per shape, median of 9 runs after a discarded warm-up.

Shape	Size: igbinary → phpser	Encode: igbinary → phpser	Decode: igbinary → phpser
packed_1k	5,495 → 1,941 B (-65%)	4.6 → 1.4 µs (-70%)	7.3 → 1.8 µs (-75%)
packed_10k	59,495 → 21,749 B (-63%)	46.4 → 13.7 µs (-70%)	74.0 → 18.9 µs (-74%)
deep_50	419 → 424 B (+1%)	1.3 → 0.62 µs (-54%)	1.8 → 1.6 µs (-15%)
dto_100	7,083 → 6,362 B (-10%)	15.5 → 13.9 µs (-10%)	26.9 → 23.5 µs (-13%)
dto_1000	73,372 → 64,863 B (-12%)	194 → 165 µs (-15%)	275 → 227 µs (-18%)
rowset_100	4,570 → 4,771 B (+4%)	10.0 → 7.3 µs (-27%)	10.7 → 10.8 µs (+1%)
rowset_1000	47,459 → 47,972 B (+1%)	157 → 71 µs (-55%)	104 → 107 µs (+4%)
dto_mixed	21,644 → 17,927 B (-17%)	58.8 → 39.8 µs (-32%)	112 → 81 µs (-28%)

The packed rows are the ones that jump out: roughly two-thirds smaller and three-quarters faster to decode, on a real shape, not a synthetic micro-case. packed_1k is range(0, 999), which is what an ID list or an analytics bucket looks like.

The DTO rows are the relatable ones. dto_1000 is a thousand small typed objects of one class, the shape a Laravel queue batch or a page of cached models actually has. 12% smaller, 18% faster to decode, 15% faster to encode, from the dictionary dedup on property names and a class-entry lookup cache that amortizes zend_lookup_class_ex across the batch. Encode is faster than igbinary on every row; the largest margins are on the object-heavy dto_mixed (32% faster, 17% smaller) and the mixed rowset_1000 (55% faster).

Where it gives a little back

The one row where phpser loses is the mixed associative rowset. rowset_1000 decodes about 4% slower than igbinary, and the rowset payloads run 1 to 4% larger. That's the front-loaded dictionary showing its one downside: the decoder walks the dictionary header before it touches values, and on a heterogeneous rowset with few repeated strings that header walk doesn't buy back its cost. It's a small tax, and it's on the exact axis I chose to de-prioritize, but it's real and measured, so there it is.

The structural limit is the same decision seen from another angle: phpser isn't streamable. The dictionary lives at the head of the payload and values reference it by index, so you can't decode the stream incrementally as it arrives. The front-loaded dictionary is what makes the other decodes fast and what makes streaming impossible. You don't get to keep both. If you need a streaming parser, this is the wrong format.

I also cross-checked the whole suite on arm64 to make sure none of this was an x86 quirk. Same direction on every shape, with narrower encode margins on the object cases. The decode wins and the single rowset_1000 decode tax both reproduce.

Signed payloads: safe to decode from an untrusted cache

The performance half is only one reason a cache serializer is its own problem. The other is that decoding attacker-controlled bytes is dangerous. Native unserialize() on untrusted input lets a crafted payload instantiate any allowed class and drive its __wakeup, __destruct, or other magic methods into a state the code never anticipated. That's the mechanism behind object-injection and gadget-chain attacks, and a cache is the soft spot: a redis instance, a memcached pool, a file cache, or a cookie is exactly the kind of store an attacker reaches in a real incident, and whatever sits there gets decoded on the next read.

phpser's answer is a signed mode built on HMAC-SHA256. You serialize with a secret key, and you refuse to decode anything that wasn't signed with that same key. Verification is constant-time and runs before any decoding work, so a tampered or foreign-keyed payload never reaches the part of the decoder that builds values or constructs objects.

$key = random_bytes(32);            // generate once, keep it in app config or a secrets manager

// on write
$blob = phpser_serialize_signed($cacheValue, $key);
$redis->set('user:42', $blob);

// on read
$blob  = $redis->get('user:42');
$value = phpser_unserialize_signed($blob, $key);

if ($value === null) {
    // tampered, truncated, or signed with a different key.
    // nothing was decoded; treat it as a cache miss and rebuild.
    $value = rebuild_user(42);
}

The contract is deliberately blunt. phpser_unserialize_signed() returns null on any signature failure rather than throwing, so a poisoned cache entry degrades to a miss instead of an exception in a hot read path. The decode only proceeds once the MAC matches. This is authentication, not encryption: the bytes are still readable, but they can't be forged without the key, and that's the property that keeps a crafted object graph out of your decoder.

Signed mode also refuses an empty key. An empty key would reduce HMAC-SHA256 to a fixed, keyless tag anyone can recompute, so a caller writing phpser_serialize_signed($v, getenv('SECRET') ?: '') with the variable unset would be shipping forgeable payloads without knowing it. Both signed entry points throw on an empty key before doing any work, so that misconfiguration fails loudly instead of silently defeating the signature.

If you genuinely can't sign, because you're decoding bytes from a source you don't control and can't key, the second line of defense is allowed_classes, with the same shape as PHP's native unserialize():

// reject every class: unknown objects decode as __PHP_Incomplete_Class, never instantiated
$value = phpser_unserialize($blob, ['allowed_classes' => false]);

// or allowlist only the classes you actually expect to read back
$value = phpser_unserialize($blob, ['allowed_classes' => [UserDto::class, OrderDto::class]]);

The same option works on phpser_unserialize_signed() too, so you can combine a valid signature with a class allowlist for defense in depth. Underneath both paths, the decoder is hardened on its own: a recursion depth cap of 512 bounds stack use against deliberately deep payloads (decode returns null, encode throws), and crafted payloads naming a missing enum case or a non-serializable class like Closure are rejected rather than crashing, matching what PHP's own unserialize() refuses.

What I took away

igbinary is still the serializer I'd reach for on a general-purpose workload, and I'll keep using it. It's mature, it's everywhere, and on a mixed rowset it's still a hair ahead on decode.

For a read-heavy cache, phpser gives me two things at once. It's faster on the shapes caches actually hold, encode and decode both, because the wire format is designed around the reader rather than balanced between reader and writer. And signed mode means I can decode from redis without treating every read as a potential injection. Speed and trust are the two things a cache serializer has to get right, and they're the two things a general serializer leaves half-finished. Building both in was the point.

pie install iliaal/phpser

Source, wire-format spec, and the bench harness: github.com/iliaal/phpser

Coder Legion - Recent posts tagged serialization

I Built a Serializer That Refuses to Own Your Data

I Built a Serializer That Refuses to Own Your Data

The Problem Wasn't Turning Objects Into Bytes

The First Design Decision

Why I Didn't Just Use System.IO.Stream

The Two Directions Matter

The Smallest Possible Example

Now Let's Actually Serialize Something

Installation

What About Real Files?

The Stream Adapter Is Supposed to Be Boring

Then I Ran Into a Very Important Detail

Serialization Is Not Storage

Serialization

Storage

Transport

Why Binary?

Deterministic Does Not Mean Magical

Versioning Gets Interesting

The Package Came From a Real Application

And This Is Where FSM_COS Enters the Story

The Tests Tell the Same Story

I Also Benchmarked It

The Benchmark Question I Actually Care About

What I Deliberately Did Not Build

So What Would You Actually Use It For?

A Serializer Shouldn't Become Your Architecture

The Boundary I Want to Preserve

Why I Think Small Infrastructure Is Worth Talking About

You Don't Have to Take My Word for It

Resources & Code:

NuGet Packages:

GitHub Repositories:

phpser: a fast, secure binary serializer for PHP cache workloads

Why a serializer built for caches?

Designing the format for the reader

A string dictionary, and an intern that survives decode

Fast to encode, too

Tagged scalar runs, and building the array in place

The benchmarks

Where it gives a little back

Signed payloads: safe to decode from an untrusted cache

What I took away

Why I Didn't Just Use `System.IO.Stream`