Repository navigation
termio: pipeline pty reads to overlap parsing with draining (25 - 55% IO throughput increase) - #13209
Merged
Merged
Conversation
This replaces the single-threaded pty read loop on posix systems with a two-stage pipeline: a new `io-gather` thread drains the pty into a small ring of large buffers while the existing `io-reader` thread parses the previous batch concurrently. The motivating discovery was actually found by Fable, but the resulting code was hand-written and hand-verified (in addition to model-verified as an extra check): on macOS the kernel tty output queue caps every read on the pty master at about 1 KiB regardless of the read buffer size. Instrumenting a pty with a 64 KB buffer while streaming a 6.49 MB file produced 6,337 reads where every read was exactly 1024 bytes. This immediately made me realize two things about the old loop that we've had since like 2023 which called processOutput once per read: all per-call overhead (terminal lock, render wakeup, cursor timestamp) was paid per kilobyte of bulk output, and the child could never run more than 1 KiB ahead of us, so while we parsed the child (e.g. `cat`) sat blocked on a full kernel queue instead of producing. In 2023, I justified this architecture by saying "reads are generally small" but I didn't understand then that reads are generally small because the kernel makes them small even if there is a lot of data. To preserve latency for the more typical small-reads-that-are-actually-small, sub-1 KB payloads deliver on the first EAGAIN with no added latency. End-to-end throughput was measured by timing `cat file > /dev/ttysN` against a fresh app instance (M4 Max, macOS 26, ReleaseFast, medians of repeated interleaved A/B runs): | stream | before | after | change | |-------------------------|--------------|---------------|---------| | ascii.txt (6.5 MB) | 91-92 MB/s | 114-123 MB/s | +25-30% | | unicode.txt (8 MB) | 116-117 MB/s | 180-183 MB/s | +55% | | DOOM-Fire-Zig | 530 fps | 770 fps | +45% | The pipeline now delivers the parse stage's full measured capacity (the parse thread is pegged while gather spends ~33% of a core, so any IO throughput improvements are now fully parser-bound). **Linux note:** This needs to be verified on Linux. I think broadly architecture is better and should never be worse. But its possible some of the magic constants need to be tuned differently. Would love more testing there.
Contributor
Author
vtebench - Ghostty Main vs this PR
vtebench - Ghostty Main vs. this PR vs. Alacritty vs. Kitty (typical fast)
|
Contributor
|
@mitchellh Equally impressive gains on Linux for me (NixOS, Hyprland, Ryzen 5 9600X with iGPU) with just some quick, dirty tests: ASCII throughput w/ hyperfineBefore:
After:
vtebenchBefore: After: DOOM-fire-zigBefore: After: |
mitchellh
added a commit
that referenced
this pull request
Jul 6, 2026
…t increase) (#13220) This is a series of five commits that optimizes VT processing throughput in various ways. Each commit is isolated, individually benchmarked, and carries a detailed commit message so please read each for details about each change. After #13209 made IO fully parser-bound, these gains should translate directly into end-to-end IO throughput (until some other stage becomes the new bottleneck). Plain ASCII processing went from ~128 MB/s to ~725 MB/s. `time cat ascii_150MB.txt` went from 1.5s before 13209 to 1.2s on main to 566ms on this branch. ## The changes 1. **batch printed codepoint runs into direct row fills**. Profiling showed ~85% of plain-text time inside `Terminal.print`, re-answering the same questions (margins, modes, width, charset, style) for every single character. A new `print_slice` stream action delivers runs of decoded codepoints to `Terminal.printSlice`, which hoists the invariants and fills rows with a masked-compare + branch-free store loop, falling back to `print()` for anything complex. **Result: 2.2x–5.7x on ascii plus unicode text.** 2. **dispatch CSI finals directly from stream fast paths**. Every byte through `Parser.next` copies a ~240 byte `[3]?Action` and a typical CSI copied it twice. New `csi_entry/final` fast paths dispatch directly without the action array. **Result: +17-18% on CSI streams.** 3. **bulk-parse CSI parameter bytes at the slice level**. Parameter digits/separators are consumed in a tight slice loop with parser state in locals instead of re-entering the per-byte path. **Result: +29-41% on escape-heavy streams.** 4. **skip style map update when SGR leaves style unchanged**. Skip the release/hash/probe/use churn when an SGR attribute is a no-op. **Result: +4-7% on TUI-refresh patterns, -2-3% on adversarial random-color streams** (tradeoff detailed in the commit message). This one is more questionable, but willing to measure on real workloads. ## Benchmarks Measured with `ghostty-bench +terminal-stream` (full terminal handler, 100 MB deterministic synthetic corpora, 120x80 terminal, M4 Max, macOS 26, ReleaseFast, hyperfine means of 10 runs, ~15 ms process startup included in all numbers). These are parser-stage numbers, not end-to-end app numbers. | stream | before | after | throughput | change | |----------------------------|--------|--------|------------------|--------| | ascii (no newlines) | 784 ms | 138 ms | 128 → 725 MB/s | 5.7x | | ascii lines | 833 ms | 198 ms | 120 → 505 MB/s | 4.2x | | unicode mixed-script | 779 ms | 320 ms | 128 → 313 MB/s | 2.4x | | CJK (all wide) | 424 ms | 126 ms | 236 → 794 MB/s | 3.4x | | unicode, mode 2027 on | 807 ms | 367 ms | 124 → 273 MB/s | 2.2x | | CJK, mode 2027 on | 495 ms | 198 ms | 202 → 505 MB/s | 2.5x | | csi mix (SGR/CUP/EL/modes) | 648 ms | 414 ms | 154 → 242 MB/s | 1.6x | | sgr fire (doom-fire-like) | 495 ms | 303 ms | 202 → 330 MB/s | 1.6x | | TUI redraw (repeat styles) | 642 ms | 291 ms | 156 → 344 MB/s | 2.2x | | osc | 8.26 s | 8.20 s | (untouched path) | ~1.0x | **End-to-end note:** #13209 measured the parse thread pegged while the gather thread used ~33% of a core, so parser gains of this size may make gather (or the renderer lock) the new bottleneck for plain text before the full 5.7x shows up end to end. I'll take a look at that soon... ## LLM Notes These findings were almost all found by Fable 5. I went through each change and simplified quite a lot, read every single line, re-ran verifications by hand. Fable in particular isn't good at writing elegant Zig code, so there's a lot of style stuff. Ultimately though, I understand all of this and feel comfortable with the changes.
jcollie
pushed a commit
to jcollie/ghostty
that referenced
this pull request
Jul 6, 2026
ghostty-org#13209 After ghostty-org#13209 the IO pipeline delivers the parse thread's full measured capacity, so IO throughput is now bound by VT processing. Profiling `terminal-stream` on plain text showed ~85% of wall time inside Terminal.print: every printable codepoint paid the full per-character cost (right margin computation, grapheme clustering checks, width lookup, wrap/insert mode checks, charset mapping, per-cell style bookkeeping, dirty marking, cursor advance) even though for typical bulk output every one of those answers is the same for thousands of consecutive characters. This adds a new print_slice stream action carrying a run of printable codepoints, emitted whenever the SIMD ground-state path decodes multiple codepoints at once, plus Terminal.printSlice which processes such runs in batch. Since action dispatch is comptime, delivering a slice through the existing vt handler interface has the same codegen as a dedicated entry point; handlers that don't care about batching can simply loop and treat each codepoint as a print action. printSlice hoists all run-invariant checks (status display, insert and wraparound modes, charset state, hyperlink state) out of the loop and then fills cells row by row. A single masked u64 compare classifies each destination cell as "simple" (plain codepoint cell, narrow, no hyperlink, style already matching the cursor); runs of simple cells are written with a branch-free store loop, style-only mismatches are handled inline with the same ref-counting printCell does, and anything needing real cleanup (wide spacers, grapheme data, hyperlinks) exits the fast path with the cursor positioned on the offending cell so print() handles that one codepoint with full generality. Dirty marking, previous_char, and cursor advancement happen once per row instead of once per character. The fast path handles both narrow and wide codepoints (CJK/emoji are written as wide+spacer_tail pair fills, including spacer-head handling at the right edge) and stays exact under grapheme clustering (mode 2027): a codepoint only joins a run if it is width 1 or 2 and is a grapheme break from the previously written codepoint, so print() would never have attached it to the previous cell. The first codepoint of a batch defers to print() whenever the previous cell could carry cluster state we can't cheaply reason about (including a pending wrap, where print attaches to the pending cell instead of wrapping). Correctness is verified by a new differential fuzz test that runs the same operations through per-codepoint print and randomly chunked printSlice, comparing full screen dumps, cursor state, and page integrity (style refcounts, grapheme maps) after every operation, across wraps, margins, mode toggles, hyperlinks, charsets, and wide/combining/ZWJ/RI/jamo codepoints. Throughput measured with ghostty-bench terminal-stream (full terminal handler, 100 MB deterministic corpora, 120x80, M4 Max, ReleaseFast, hyperfine means of 10 runs; ~15ms process startup included in all numbers): | stream | before | after | change | |---------------------------|--------|--------|--------| | ascii (no newlines) | 784 ms | 138 ms | 5.7x | | ascii lines | 833 ms | 198 ms | 4.2x | | unicode mixed-script | 779 ms | 320 ms | 2.4x | | CJK (all wide) | 424 ms | 126 ms | 3.4x | | unicode, mode 2027 on | 807 ms | 367 ms | 2.2x | | CJK, mode 2027 on | 495 ms | 198 ms | 2.5x |
Contributor
Author
|
Awesome, thank you for confirming! |
deblasis
added a commit
to deblasis/wintty
that referenced
this pull request
Jul 7, 2026
The Windows read loop still used a 1 KiB buffer, a leftover from the posix loop shape where the kernel caps pty reads at ~1 KiB anyway. ConPTY has no such cap: measured on Server 2025 against both the in-box conhost and our bundled OpenConsole pair, a 64 KiB read returns 46-65 KiB per call under bulk output, so the small buffer just meant 45-63x more ReadFile calls, terminal lock acquisitions, and render wakeups for the same stream. Throughput doesn't move (ConPTY's own VT rendering caps it at 10-13 MB/s either way), so this is purely per-byte overhead reduction. The new windows_read_capacity const is deliberately aliased to buffer_capacity so one batch bounds terminal lock hold time the same on every platform, with a note to revisit if the posix side retunes. The same measurements showed upstream's io-gather thread (ghostty-org#13209 shape) buys nothing while ConPTY is the transport; that part is deliberately not ported and should be revisited with any future transport change. Harness: probe/conpty-read branch.
mitchellh
added a commit
that referenced
this pull request
Jul 7, 2026
Fixes a frame time regression reported with fortio's `fps -fire` benchmark (fortio.org/terminal/fps): frame times nearly doubled at typical grid sizes after #13209 (the pipelined pty reader), and were more than 5x worse at small grids. ## The problem The `fps -fire` program follows a request/response pattern: write a frame, end it with a cursor position query (CSI 6n), and block until the reply arrives before starting the next frame. The gather stage treats any burst of 1 KiB or more as a saturated stream. When its spin retries come up dry, it sleeps in a 1ms poll expecting the writer's next refill so it can publish fewer, larger batches. But a frame-synced writer will never refill here: it is blocked waiting for a reply to a query that is sitting inside the very batch the gather stage is holding back. The poll always sleeps its full timeout, adding ~1.2ms to every round trip. Ouch! ## The fix Sleeping for a refill gap is only free while the parse stage is busy, since the wait hides behind parse time. Once the parser is idle, every additional microsecond spent bridging is added straight to output latency. So: 1. When the spin retries exhaust and the parse stage is idle, deliver the batch immediately instead of polling. 2. When the parse stage is busy, use a `pipe2` pipe to allow the parser to notify the gather thread it is idle. In the middle of the `poll` loop and sleep, we can get interrupted immediately and deliver the batch. The pipe is only written while the gather stage is actively polling, so an interactive terminal never pays the syscalls, and a saturated stream never idles the parser, preserving full batching and throughput for bulk output. Win, win, win! ## Benchmarks | workload | pre-#13209 | before | after | |---|---|---|---| | fps -fire 80x24 | 0.262 ms / 3674 fps | 1.435 ms / 694 fps | 0.234 ms / 4106 fps | | fps -fire 160x45 | 1.012 ms / 975 fps | 1.823 ms / 545 fps | 0.701 ms / 1405 fps | | fps -fire 284x68 | 2.443 ms / 407 fps | 2.391 ms / 414 fps | 1.351 ms / 732 fps | | cat 19.3 MB | 0.204 s (95 MB/s) | 0.086 s (224 MB/s) | 0.082 s (236 MB/s) | ## LLM Notes Bisect script written by hand and found the offending commit. Fable 5 found the likely cause. I manually came up with the proposed solution and wrote it out. Fable helped benchmark for me to verify my assumptions conceptually and in the real world.
mitchellh
added a commit
that referenced
this pull request
Jul 7, 2026
Fixes a frame time regression reported with fortio's `fps -fire` benchmark (fortio.org/terminal/fps): frame times nearly doubled at typical grid sizes after #13209 (the pipelined pty reader), and were more than 5x worse at small grids. ## The problem The `fps -fire` program follows a request/response pattern: write a frame, end it with a cursor position query (CSI 6n), and block until the reply arrives before starting the next frame. The gather stage treats any burst of 1 KiB or more as a saturated stream. When its spin retries come up dry, it sleeps in a 1ms poll expecting the writer's next refill so it can publish fewer, larger batches. But a frame-synced writer will never refill here: it is blocked waiting for a reply to a query that is sitting inside the very batch the gather stage is holding back. The poll always sleeps its full timeout, adding ~1.2ms to every round trip. Ouch! ## The fix Sleeping for a refill gap is only free while the parse stage is busy, since the wait hides behind parse time. Once the parser is idle, every additional microsecond spent bridging is added straight to output latency. So: 1. When the spin retries exhaust and the parse stage is idle, deliver the batch immediately instead of polling. 2. When the parse stage is busy, use a `pipe2` pipe to allow the parser to notify the gather thread it is idle. In the middle of the `poll` loop and sleep, we can get interrupted immediately and deliver the batch. The pipe is only written while the gather stage is actively polling, so an interactive terminal never pays the syscalls, and a saturated stream never idles the parser, preserving full batching and throughput for bulk output. Win, win, win! ## Benchmarks | workload | pre-#13209 | main | this PR | |---|---|---|---| | fps -fire 80x24 | 0.262 ms / 3674 fps | 1.435 ms / 694 fps | 0.234 ms / 4106 fps | | fps -fire 160x45 | 1.012 ms / 975 fps | 1.823 ms / 545 fps | 0.701 ms / 1405 fps | | fps -fire 284x68 | 2.443 ms / 407 fps | 2.391 ms / 414 fps | 1.351 ms / 732 fps | | cat 19.3 MB | 0.204 s (95 MB/s) | 0.086 s (224 MB/s) | 0.082 s (236 MB/s) | ## LLM Notes Bisect script written by hand and found the offending commit. Fable 5 found the likely cause. I manually came up with the proposed solution and wrote it out. Fable helped benchmark for me to verify my assumptions conceptually and in the real world. Commit message written by hand except for the benchmark results.
deblasis
added a commit
to deblasis/wintty
that referenced
this pull request
Jul 18, 2026
The Windows read loop still used a 1 KiB buffer, a leftover from the posix loop shape where the kernel caps pty reads at ~1 KiB anyway. ConPTY has no such cap: measured on Server 2025 against both the in-box conhost and our bundled OpenConsole pair, a 64 KiB read returns 46-65 KiB per call under bulk output, so the small buffer just meant 45-63x more ReadFile calls, terminal lock acquisitions, and render wakeups for the same stream. Throughput doesn't move (ConPTY's own VT rendering caps it at 10-13 MB/s either way), so this is purely per-byte overhead reduction. The new windows_read_capacity const is deliberately aliased to buffer_capacity so one batch bounds terminal lock hold time the same on every platform, with a note to revisit if the posix side retunes. The same measurements showed upstream's io-gather thread (ghostty-org#13209 shape) buys nothing while ConPTY is the transport; that part is deliberately not ported and should be revisited with any future transport change. Harness: probe/conpty-read branch.
deblasis
added a commit
to deblasis/wintty
that referenced
this pull request
Jul 29, 2026
The Windows read loop still used a 1 KiB buffer, a leftover from the posix loop shape where the kernel caps pty reads at ~1 KiB anyway. ConPTY has no such cap: measured on Server 2025 against both the in-box conhost and our bundled OpenConsole pair, a 64 KiB read returns 46-65 KiB per call under bulk output, so the small buffer just meant 45-63x more ReadFile calls, terminal lock acquisitions, and render wakeups for the same stream. Throughput doesn't move (ConPTY's own VT rendering caps it at 10-13 MB/s either way), so this is purely per-byte overhead reduction. The new windows_read_capacity const is deliberately aliased to buffer_capacity so one batch bounds terminal lock hold time the same on every platform, with a note to revisit if the posix side retunes. The same measurements showed upstream's io-gather thread (ghostty-org#13209 shape) buys nothing while ConPTY is the transport; that part is deliberately not ported and should be revisited with any future transport change. Harness: probe/conpty-read branch.
deblasis
added a commit
to deblasis/wintty
that referenced
this pull request
Aug 10, 2026
The Windows read loop still used a 1 KiB buffer, a leftover from the posix loop shape where the kernel caps pty reads at ~1 KiB anyway. ConPTY has no such cap: measured on Server 2025 against both the in-box conhost and our bundled OpenConsole pair, a 64 KiB read returns 46-65 KiB per call under bulk output, so the small buffer just meant 45-63x more ReadFile calls, terminal lock acquisitions, and render wakeups for the same stream. Throughput doesn't move (ConPTY's own VT rendering caps it at 10-13 MB/s either way), so this is purely per-byte overhead reduction. The new windows_read_capacity const is deliberately aliased to buffer_capacity so one batch bounds terminal lock hold time the same on every platform, with a note to revisit if the posix side retunes. The same measurements showed upstream's io-gather thread (ghostty-org#13209 shape) buys nothing while ConPTY is the transport; that part is deliberately not ported and should be revisited with any future transport change. Harness: probe/conpty-read branch.
deblasis
added a commit
to deblasis/wintty
that referenced
this pull request
Aug 21, 2026
The Windows read loop still used a 1 KiB buffer, a leftover from the posix loop shape where the kernel caps pty reads at ~1 KiB anyway. ConPTY has no such cap: measured on Server 2025 against both the in-box conhost and our bundled OpenConsole pair, a 64 KiB read returns 46-65 KiB per call under bulk output, so the small buffer just meant 45-63x more ReadFile calls, terminal lock acquisitions, and render wakeups for the same stream. Throughput doesn't move (ConPTY's own VT rendering caps it at 10-13 MB/s either way), so this is purely per-byte overhead reduction. The new windows_read_capacity const is deliberately aliased to buffer_capacity so one batch bounds terminal lock hold time the same on every platform, with a note to revisit if the posix side retunes. The same measurements showed upstream's io-gather thread (ghostty-org#13209 shape) buys nothing while ConPTY is the transport; that part is deliberately not ported and should be revisited with any future transport change. Harness: probe/conpty-read branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This replaces the single-threaded pty read loop on posix systems with a two-stage pipeline: a new
io-gatherthread drains the pty into a small ring of large buffers while the existingio-readerthread parses the previous batch concurrently.After this, our IO throughput is now within noise (1 to 3%) of our VT parsing and processing throughput. This means that any VT performance improvements should raise IO throughput. We're completely bound by it now.
Important
Against all odds, we've found a massive (25% to 55%) performance improvement in Ghostty IO. This is a subsystem we were happy to get 2% improvement gains in recent years. And the change is relatively simple and understandable, too.
The motivating discovery was actually found by Fable, but the resulting code was hand-written and hand-verified (in addition to model-verified as an extra check): on macOS the kernel tty output queue caps every read on the pty master at about 1 KiB regardless of the read buffer size. Instrumenting a pty with a 64 KB buffer while streaming a 6.49 MB file produced 6,337 reads where every read was exactly 1024 bytes.
This immediately made me realize two things about the old loop that we've had since like 2023 which called processOutput once per read: all per-call overhead (terminal lock, render wakeup, cursor timestamp) was paid per kilobyte of bulk output, and the child could never run more than 1 KiB ahead of us, so while we parsed the child (e.g.
cat) sat blocked on a full kernel queue instead of producing.In 2023, I justified this architecture by saying "reads are generally small" but I didn't understand then that reads are generally small because the kernel makes them small even if there is a lot of data.
To preserve latency for the more typical
small-reads-that-are-actually-small, sub-1 KB payloads deliver on the first EAGAIN with no added latency.
Benchmarks
End-to-end throughput was measured by timing
cat file > /dev/ttysNagainst a fresh app instance (M4 Max, macOS 26, ReleaseFast, medians of repeated interleaved A/B runs):The pipeline now delivers the parse stage's full measured capacity (the parse thread is pegged while gather spends ~33% of a core, so any IO throughput improvements are now fully parser-bound).
Linux note: This needs to be verified on Linux. I think broadly architecture is better and should never be worse. But its possible some of the magic constants need to be tuned differently. Would love more testing there.
Some Background on LLM Usage
As noted above, the original promising path was found by Fable 5. I decided to throw some money at analyzing our IO throughput, and went into it expecting minor improvements at the cost of unmaintainable special case optimizations (typical LLM results when given such tasks on an already-optimized path).
I spun off a Fable 5 session (API pricing) prior to some holiday weekend (American, July 4th) festivities. When I came back late that night, it pointed this path out with some pretty questionable code/results.
The frustrating thing is that I swear I tried this years ago and it didn't deliver results. Unfortunately, it was long enough ago (probably 2023) that I can't remember nor do I have any evidence. But, my brain had already clocked this possibility and blocked it out as "already tried and failed." The code in question that this PR ultimately touched has been basically unchanged for 2+ years.
As a second point, this is a great example of what I love about LLMs. I was not in a performance-hunting mood and I have other tasks I need to get to while at the computer, so improving Ghostty performance wasn't my current mode and I would not have worked on it anytime soon while at the computer. But, I try to keep an agent running all the time, and before my family came over for holiday afternoon, I decided today I would try budgeting $100 to Fable to focus on Ghostty's IO performance. Why not?
I came back, saw some questionable but interesting results, and decided it was worth some human hours to validate, understand, and work on this myself. It was worth it. 😄
Anyways, the total API cost of this if you're curious: $88.28.
Remember, I hand-wrote the code so thats basically what it cost me to discover this path. Fable did produce its own solution (in about 3x the LoC change of this diff with non-idiomatic Zig styles, overly defensive guards, and simultaneously poor error handling). So, I guess I did pay for a solution. A bad one. Haha. But the problem it found was real and good.