Coder Legion - Recent posts tagged x86

Notes on x86 PVH boot

Fri, 25 Sep 2026 14:47:35 +0000

Motivation

My Rust KVM project now being able to boot Linux on ARM64, I decided to try and
boot Linux on x86. I have been trying in the past to set up the boot procedure
via boot_params struct. At the time it was a lot more complex and now I wanted
to have some result as soon as possible. So I turned to the PVH
(Para-Virtualized Hardware) Linux boot which simplifies this process a lot. As
a Xen Project protocol, PVH provides a relatively simple interface between a
VMM and the Linux kernel, without requiring the VMM to reproduce the
traditional firmware/bootloader path.

Entering protected mode via BIOS

The traditional way of booting an operating system on an x86 machine is to start
in real mode and then proceed to protected mode. From there, if CPU is 64-bit,
we can go to long mode. In the PVH Linux entry path, Linux performs the
transition from the required 32-bit protected-mode entry state into 64-bit mode
itself.

Setting up real mode

x86 starts at reset vector (physical address 0xFFFFFFF0)
CPU is in 16-bit real mode and firmware like SeaBIOS takes over
SeaBIOS looks at first sector (512 bytes) of a bootable device
if the sector contains 0xAA55 at 510-byte offset, it's loaded at 0x7C00
load the rest of data manually from our boot device (e.g. on 0x8000)
jump to the code section we just loaded (e.g. to address 0x8000)
if necessary, set up Stack Pointer and Stack Segment

Getting to protected mode

make sure interrupts are disabled
set up and load a minimal GDT (Global Descriptor Table)
set PE (Protected Mode Enable) bit in CR0 (Control Register)
perform far jump selecting a proper GDT entry

GDT example

A minimal GDT code set up looks like the following:

gdt_start:
dq 0
gdt_code:; index 1
dw 0xFFFF; segment limit (low)
dw 0x0000; segment base (low)
db 0x00; base address (middle)
db 10011010b; access bits (read/write, executable, code or data, present)
db 11001111b; flags (32-bit protected, 4k blocks) - segment limit (high)
db 0x00; base address (high)

gdt_data:; index 2
dw 0xFFFF; segment limit (low)
dw 0x0000; segment base (low)
db 0x00; base address (middle)
db 10010010b; access bits (read/write, code or data, present)
db 11001111b; flags (32-bit protected, 4k blocks) - segment limit (high)
db 0x00; base address (high)
gdt_end:

gdt_descriptor:
dw gdt_end - gdt_start - 1
dd gdt_start

Then we enter protected mode by executing:

cli
lgdt [gdt_descriptor]

mov eax, cr0
or eax, 1
mov cr0, eax

jmp 0x08:protected_mode_start ; 0x08 is 1 << 3 (select index 1 in GDT)

The far jump reloads CS using the descriptor selected by 0x08. The other
segment registers must be loaded with appropriate protected-mode selectors as
well. Each segment register has a visible selector and hidden descriptor-cache
state containing information such as the base, limit and access rights.

How PVH helps

If we are in a virtual machine, we don't really need to go through all of these
steps. We don't need to have firmware or BIOS to set up CPU state properly, we
can simply do this from the VM.

Here, we can manipulate the vCPU to our will, and therefore we can set segment
registers ourselves to valid values, skipping GDT setup. We do need to do some
administration to help Linux understand what's going on and this is exactly
where PVH comes in.

ELF PVH note

To determine whether an x86 Linux ELF image supports the PVH boot ABI, we can
inspect its Xen ELF notes and look for XEN_ELFNOTE_PHYS32_ENTRY (note type
18). Its descriptor gives the physical 32-bit entry point at which the guest
should be started.

Newer kernels may also contain XEN_ELFNOTE_PHYS32_RELOC (type 19), which
describes alignment and physical-address constraints for relocatable PVH
kernels.

PVH boot memory setup

Boot procedure from PVH is more straightforward, but requires more memory
management from VM:

get RIP from ELF PVH note entry
load Linux kernel ELF segments into memory
copy kernel command line to memory
construct an E820-like memory map using HvmMemEntry entries
construct and fill out HvmStartInfo struct
load memory entries and HvmStartInfo to memory

If using initramfs you also need to load it into memory, along with its
separate initramfs command line. We also need to fill out HvmModuleEntry
struct to point to initramfs and its command line address. Note that
here we are not talking about kernel modules, but rather Xen modules, and
for Linux that only means initramfs.

A good reference for the structures above is actually in the Linux kernel
headers and can be found on:

https://elixir.bootlin.com/linux/v7.1.3/source/include/xen/interface/hvm/start_info.h

Setting up registers

RIP points to entrypoint from ELF note 18 entry
RBX points to HvmStartInfo physical address (this is PVH specific)
set up segment registers to valid Protected Mode state
enable PE bit in CR0 (as in ordinary boot)
initialize CR4 (Control Register 4) to zero
clear VM (bit 17), IF (bit 9) and TF (bit 8) in RFLAGS

Linux kernel-side PVH register setup is described here:

https://elixir.bootlin.com/linux/v7.1.3/source/arch/x86/platform/pvh/head.S#L30

Segment registers

I think the best source for how to set up segment registers is actually my own
code from my Rust KVM project:

let code = kvm_segment {
base: 0,
limit: 0xffff_ffff,
selector: 1 << 3,
type_: 0b1010, // 3rd bit: code/data = 1
   // 2nd bit: conforming = 0
   // 1st bit: readable = 1
   // 0th bit: accessed = 0
present: 1,
dpl: 0, // descriptor privilege level (0: kernel .. 3: userspace)
db: 1,  // default operand / address size (0: 16bit, 1: 32bit)
s: 1,   // system (1: code/data segment, 0: system segment)
l: 0,   // long mode (0: not 64bit segment, 1: 64bit segment)
g: 1,   // granularity (0: limit in bytes, 1: limit in 4Kb blocks)
avl: 0, // available for software (reserved for custom OS purposes)
unusable: 0,
padding: 0,
};

let data = kvm_segment {
selector: 2 << 3,
type_: 0b0010, // 3rd bit: code/data = 0
   // 2nd bit: conforming = 0
   // 1st bit: writable = 1
   // 0th bit: accessed = 0
..code };

sregs2.tr = kvm_segment {
base: 0,
limit: 0x67, // as per Xen project PVH protocol
g: 0,
..code };

With BIOS boot, these segment descriptors would live in a GDT in guest memory
and the CPU would obtain their hidden descriptor state by loading the
corresponding selectors. With KVM, we can bypass the GDT entirely and directly
initialize the vCPU's segment state through KVM_SET_SREGS.

Conclusion

PVH turned out to be a convenient way of getting my x86 KVM implementation to
the point where it could boot a real Linux kernel without first implementing
the traditional x86 boot protocol.

There is still plenty left to implement around the x86 side of the VMM, but
being able to reach the Linux kernel through PVH gives me a much more useful
starting point than reproducing the complete BIOS boot sequence first.

The implementation is available in the x86 initialization code in my kvm-rust
repository:

https://github.com/StjepanPoljak/kvm-rust/blob/main/src/arch/x86_64/init.rs

I plan to continue extending functionality of the x86 side of my VMM.

Automating Stack Corruption Analysis in GDB with Python

Thu, 11 Jun 2026 15:55:57 +0000

A bug in my operating system

During a recent visit to my wife's family in Sarajevo, I decided to revisit my
hobby operating system in QEMU. I discovered that the boot process consistently
froze while printing the BIOS memory map. What initially looked like a
protected-mode issue eventually turned into a useful exercise in automating
debugging using GDB's Python scripting.

Manual debugging failure

After reinspecting some common pitfalls in my protected mode setup I was more
confident that the issue was stack corruption in my print_memory_map function.
If I had an unmatched push or pop, it could have corrupted return addresses and
eventually redirected execution flow into invalid memory. Single-stepping
through the routine manually quickly became impractical. The function mixed BIOS
interrupt handling, memory map parsing and multiple helper calls, making it
difficult to reason about stack state over time.

Setting up GDB scripting in Python

I needed to automate this and GDB's integration with Python was the most
promising route I could take. The idea was to do exactly what I started
manually: break at a specific (suspicious) function and then start single
stepping while inspecting how the stack pointer behaved.

First things first, we import GDB module in Python, connect to the remote
target and set up a breakpoint (this is done by inheriting from
gdb.Breakpoint):

import gdb

class StackTraceBreakpoint(gdb.Breakpoint):

    def __init__(self, func_name):
        super(StackTraceBreakpoint, self).__init__(func_name)
        self.active = False
    
    def stop(self):
        if self.active:
            raise Exception("Recursion detected - stopping.")
        self.active = True

        return True

if __name__ == "__main__":
    gdb.execute("target remote :5555")
    gdb.execute("symbol-file ../build/arch/x86/bios-legacy/boot-stage1-5.elf")

    tracer = StackTraceBreakpoint("print_memory_map")
    gdb.execute("continue")

StackTraceBreakpoint

So this is kind of bare-bones of what I wanted to do. This code will simply add
a breakpoint with custom logic after connecting to QEMU and loading symbols.
As soon as we do gdb.execute("continue") GDB will run and, if and when it hits
our breakpoint, it will execute whatever we wrote in the stop() method. For
now I only added a kind of assertion that we cannot analyze recursions (I didn't
use any in my code anyway and logic would be a bit more complex).

Single-stepping

Now what we need to do is start single-stepping after we hit our breakpoint. So
we add a while loop with gdb.execute("stepi") after gdb.execute("continue").
Note that the stepi instruction steps over machine instructions, not over
source code statements. Also note that we cannot start single-stepping in
stop() method because GDB won't be in a state which can accept these kinds of
debugging requests.

Detecting stack imbalance

Furthermore I have wrapped this single-stepping logic in trace_step() method
in our breakpoint class. This method is not part of GDB breakpoint API, but
rather just a convenience for tracking the number of pushes and pops in a
consistent manner. To run this script we need to call:

gdb -ex 'source debug-stack.py'

Another thing we need to track is whether we have entered another function (in
which case we won't be counting pushes and pops) and if we have returned from
it. If I ever wanted to inspect routines being called, I would just run the same
script for them (my intention here is not creating a custom emulator on top of
GDB). So I added a counter func_count which will increase on call and
decrease on ret instruction. Here is a rough idea:

def trace_step(self):
insn_full = gdb.execute("x/i $pc", to_string=True).strip()
insn = re.search(r'^=>.*:\s*([^\s].*)$', insn_full).group(1).split(" ")[0]

match insn:
case "push":
if self.func_count == 0:
self.pushes += 1
case "pop":
if self.func_count == 0:
self.pops += 1
case "call":
self.func_count += 1
case "ret":
self.func_count -= 1

Here you can see how I'm extracting the instruction from GDB. And what is left
is just improving logic and also fetching registers like PC, SP and CS for
debugging. I log everything into a file as GDB can get really noisy with
standard output (I didn't find a way to turn off all logging in GDB completely
when single stepping). The script is available in my GitHub repository:

https://github.com/StjepanPoljak/raspios/tree/master/scripts/debug-stack.py

Finding the root cause

Finally, you can see an example output detecting my very issue:

[START] print_memory_map SP=fffd
[0000:8183] push %ax (SP=0xfffd)
[0000:8185] push %bx (SP=0xfff9)
[0000:8187] push %cx (SP=0xfff5)
[0000:8189] push %dx (SP=0xfff1)
[0000:81a5] call 0x66e980a5 (SP=0xffed)
[0000:80a3] push %ax (SP=0xffeb)
[0000:80a7] call 0xab18056 (SP=0xffe7)
(...)
[0000:806e] ret  (SP=0xffef)
[0000:8098] call 0xf6ec8056 (SP=0xfff1)
[0000:8054] push %ax (SP=0xffef)
[0000:8056] push %bx (SP=0xffeb)
[0000:806a] pop %bx (SP=0xffe7)
[0000:806c] pop %ax (SP=0xffeb)
[0000:806e] ret  (SP=0xffef)
[0000:8098] call 0xf6ec8056 (SP=0xfff1)
[0000:8054] push %ax (SP=0xffef)
[0000:8056] push %bx (SP=0xffeb)
[0000:806a] pop %bx (SP=0xffe7)
[0000:806c] pop %ax (SP=0xffeb)
[0000:806e] ret  (SP=0xffef)
[0000:809d] pop %esi (SP=0xfff1)
[0000:809e] pop %bx (SP=0xfff3)
[0000:80a0] pop %ax (SP=0xfff7)
[0000:80a2] ret  (SP=0xfffb)
[FAIL] Extra pop detected at [0000:81fa].

So the real culprit was an extra pop eax in my print_memory_map routine:

print_memory_map:
push eax
push ebx
push ecx
push edx

; --- ommited print loop logic ---

.noprint_newline:
pop eax

cmp ecx, [memory_map_size]
jne .print_memory_map_loop

pop edx
pop ecx
pop ebx
pop eax
ret

Removing this line will cause my debugging script to successfully pass.

Try it out yourself

You can try it out yourself, just check out my operating system, raspios, on
GitHub:

https://github.com/StjepanPoljak/raspios

Build and run it with:

mkdir build
cd build
ARCH=x86 cmake ..
make
make qemu_debug

Then, in the scripts folder run gdb -ex 'source debug-stack.py'.

Conclusion

This was a good reminder that low-level debugging often benefits from
lightweight tooling tailored to the problem at hand. In this case, a small
amount of Python automation around GDB made stack corruption analysis
significantly more manageable than manual instruction tracing.

Gloria JIT v4.4.0 — Bare-Metal Control, Memory Primitives, and Structured Flow

Mon, 01 Jun 2026 22:29:15 +0000

What is Gloria JIT?

Gloria JIT is a low-level programming language and compiler that is part of the ForgeZero ecosystem. It is written in Go and compiles source code directly to x86-64 machine code — no LLVM, no GCC, no Clang in the middle.

The goal is straightforward: give the programmer direct, unmediated control over the machine. No intermediate representation handed off to a third-party backend. No optimizer making decisions you didn't ask for. The compiler emits raw bytes, and those bytes run.

This makes Gloria JIT an interesting project for anyone curious about how compilers actually work, or for developers who want to explore bare-metal programming without the abstraction layers that most toolchains introduce.

v4.4.0 expands the language significantly, adding structured control flow, direct memory and I/O access, and a bare-metal output path for VGA framebuffer writing.

`while` Loops — Structured Control Flow

Before this release, repeated execution required manual branching. v4.4.0 adds proper while loops.

A loop runs as long as its condition is non-zero. Inside the loop body you can use standard assignment and mutation operators (=, +=, -=), and built-in calls are permitted as well. This is a meaningful step toward the kind of control flow you'd expect from a general-purpose language.

Memory Primitives — `peek` and `poke`

Two new built-in functions expose direct memory access:

peek(address) — reads a 16-bit value from the given address
poke(address, value) — writes a 16-bit value to the given address

Both accept either immediate values or runtime variables as arguments.

This is the kind of primitive that exists in very few high-level languages, but is essential for bare-metal work — writing to hardware registers, inspecting memory-mapped I/O, or building your own allocator from scratch. Handle with care.

x86-64 Port I/O

For environments that use port-mapped I/O (common in older PC hardware and embedded x86 systems), two new built-ins are available:

in8(port) — reads a single byte from the given I/O port (zero-extended)
out8(port, value) — writes a byte to the given I/O port

This enables direct hardware interaction at a level that most programming languages simply do not expose.

VGA Framebuffer Output

print(string) now supports a bare-metal execution path that writes directly to the VGA text buffer at memory address 0xB8000.

For context: on x86 PCs, this address is where text-mode video memory lives. Writing bytes there places characters directly on screen — no operating system, no drivers, no system calls involved. This is how early PC software (and modern bootloaders) produce output.

Details of the implementation:

Default text color is green (0x0A)
Register R15 is reserved as a cursor offset and is preserved across calls
Escape sequences \n and \t are resolved at compile time into the appropriate control characters

To support testing without real hardware, two utilities are included: patchVGA() swaps the real framebuffer address for a heap-allocated buffer, and dumpVGA() renders that buffer to stdout via a direct syscall, with zero heap allocations.

Register Constants

To reduce ambiguity in the backend IR and make generated code easier to reason about, named constants have been introduced for all general-purpose x86-64 registers:

regRAX(0), regRCX(1), regRDX(2), regRBX(3),
regRSP(4), regRBP(5), regRSI(6), regRDI(7),
regR8(8) ... regR15(15)

Internal Changes

A few backend improvements shipped alongside the language features:

emitLowLevelPrint now accepts a kernelMode flag to switch between syscall-based output and direct VGA writes
emitPushReg and emitPopReg now cover the full register set including R8–R15
New operations: emitMovMemToReg64, emitMovRegToMem64
parseStringLiteral handles escape sequences at compile time rather than at runtime
emitBareMetalPrint introduced for the VGA output path

Where This Is Heading

With v4.4.0, Gloria JIT operates in two modes simultaneously: as a userspace compiler with kernel-aware syscall output, and as a bare-metal code generator capable of running without an operating system underneath it.

The next focus areas are optimization passes and IR stability. If you're interested in how compilers work from the ground up — or in low-level x86 programming without giving up a proper language — this is worth following.

Gloria JIT is part of the ForgeZero project. Follow along on [AlexVoste](https://github.com/forgezero-cli/forgezero

Coder Legion - Recent posts tagged x86

Notes on x86 PVH boot

Motivation

Entering protected mode via BIOS

Setting up real mode

Getting to protected mode

GDT example

How PVH helps

ELF PVH note

PVH boot memory setup

Setting up registers

Segment registers

Conclusion

Automating Stack Corruption Analysis in GDB with Python

A bug in my operating system

Manual debugging failure

Setting up GDB scripting in Python

StackTraceBreakpoint

Single-stepping

Detecting stack imbalance

Finding the root cause

Try it out yourself

Conclusion

Gloria JIT v4.4.0 — Bare-Metal Control, Memory Primitives, and Structured Flow

What is Gloria JIT?

while Loops — Structured Control Flow

Memory Primitives — peek and poke

x86-64 Port I/O

VGA Framebuffer Output

Register Constants

Internal Changes

Where This Is Heading

`while` Loops — Structured Control Flow

Memory Primitives — `peek` and `poke`