embedded-terminal: RIS (ESC c) keeps the OSC 4 palette changes that xterm resets #1695

A program that runs in an EmbeddedTerminalRenderable can change palette colors
with OSC 4 and then reset the terminal with RIS (ESC c, what the reset
command sends). xterm restores the palette on RIS. The embedded terminal does not:
palette color 1 stays what the program set, and text drawn after the reset uses it.
The same holds for DECSTR (CSI ! p), which the embedded terminal ignores
completely.

The dynamic colors (OSC 10, 11, 12) also survive RIS in the embedded terminal.
That matches xterm, which does not reset them on RIS or DECSTR.

Reproduction

From packages/core, save as repro.ts and run bun repro.ts. Each row sets palette
color 1 and the OSC 10, 11, and 12 colors in a new terminal, sends the reset in the
first column, and then asks for the four colors. Replies are shortened to two hex
digits per channel:

import { createTestRenderer } from "./src/testing"
import { EmbeddedTerminalRenderable } from "./src/renderables/EmbeddedTerminal"

const { renderer } = await createTestRenderer({ width: 10, height: 2 })

// Set palette color 1 and the OSC 10, 11, and 12 colors, send a reset, then ask for the four colors.
async function colorsAfter(reset: string) {
  const replies: string[] = []
  const term = new EmbeddedTerminalRenderable(renderer, {
    cols: 10,
    rows: 2,
    onData: (data) => replies.push(new TextDecoder().decode(data)),
  })
  renderer.root.add(term)
  term.write("\x1b]4;1;rgb:00/ff/00\x1b\\\x1b]10;rgb:11/22/33\x1b\\\x1b]11;rgb:44/55/66\x1b\\\x1b]12;rgb:77/88/99\x1b\\")
  term.write(reset)
  term.write("\x1b]4;1;?\x1b\\\x1b]10;?\x1b\\\x1b]11;?\x1b\\\x1b]12;?\x1b\\")
  term.destroyRecursively()
  const answers = replies.join("").split("\x1b\\").filter(Boolean)
  return answers.map((answer) => answer.replace("\x1b]", "").replace(/(\w{2})\w{2}/g, "$1")).join("   ")
}

console.log("no reset".padEnd(34), await colorsAfter(""))
console.log("RIS (ESC c)".padEnd(34), await colorsAfter("\x1bc"))
console.log("DECSTR (CSI ! p)".padEnd(34), await colorsAfter("\x1b[!p"))
console.log("OSC 104, 110, 111, 112".padEnd(34), await colorsAfter("\x1b]104\x1b\\\x1b]110\x1b\\\x1b]111\x1b\\\x1b]112\x1b\\"))
renderer.destroy()

Output on main (c46fc7ee8):

no reset                           4;1;rgb:00/ff/00   10;rgb:11/22/33   11;rgb:44/55/66   12;rgb:77/88/99
RIS (ESC c)                        4;1;rgb:00/ff/00   10;rgb:11/22/33   11;rgb:44/55/66   12;rgb:77/88/99
DECSTR (CSI ! p)                   4;1;rgb:00/ff/00   10;rgb:11/22/33   11;rgb:44/55/66   12;rgb:77/88/99
OSC 104, 110, 111, 112             4;1;rgb:cc/66/66   10;rgb:ff/ff/ff   11;rgb:00/00/00   12;rgb:ff/ff/ff

Expected

xterm's result for each reset (see "What xterm does"):

For the RIS row the embedded terminal should print
4;1;rgb:cc/66/66 10;rgb:11/22/33 11;rgb:44/55/66 12;rgb:77/88/99, and the DECSTR row
the same.

What xterm does

I read xterm patch 411 (the tag xterm-411 of ThomasDickey/xterm-snapshots; the
installed xterm -version is 411). Both resets go through one function,
ReallyReset(xw, full, saved)
(charproc.c#L14358-L14566). The dispatcher calls it
with full = True for RIS and full = False for DECSTR
(charproc.c#L6149-L6157).

The only code that restores OSC 10 to 19 colors is ResetColorsRequest. Its one
caller is the OSC dispatcher for OSC 110 to 119
(misc.c#L4276-L4292). RIS has one more color-related line:
if (xw->flags & REVERSE_VIDEO) ReverseVideo(xw)
(charproc.c#L14507-L14508). It undoes DECSCNM
(CSI ? 5 h) by swapping the current foreground and background. It does not
restore the defaults, so colors that a program set with OSC 10 and 11 stay.

This settles the disagreement between the two reviews of the color-redraw fix:
only the palette part of the report is a defect. The review that said xterm resets
OSC 10, 11, and 12 on RIS cited an xterm FullReset that calls ResetColors. xterm
has neither function (a search of charproc.c, misc.c, main.c, util.c, and
screen.c finds no FullReset and no ResetColors). Current Ghostty agrees with
the other review. Its fullReset calls self.colors.palette.resetAll() and leaves the
dynamic colors alone
(Terminal.zig#L4954-L4996). Its softReset has the comment
"xterm resets the palette for DECSTR as well as RIS ... Dynamic colors (OSC 10-19) are
kept" (Terminal.zig#L5004-L5047).

I did not run the xterm binary: it needs an X display, and I did not want to open a window on a shared desktop.

Cause

EmbeddedTerminal passes the VT actions to Ghostty's stream handler unchanged
(main.zig#L33-L42).
For full_reset it calls Terminal.fullReset in the vendored Ghostty (version
1.3.2-dev, in src/vendor/zig-deps.tar.gz). That version resets the screens, modes,
tab stops, and cursor style, but not colors.palette. The vendored Ghostty also has no
DECSTR: its CSI p handler logs "ignoring unimplemented CSI p" for CSI ! p.
Upstream Ghostty has both fixes today (links above).

Suggested fix

The fix belongs in Ghostty: update the vendored tarball to a version that has
colors.palette.resetAll() in fullReset and softReset. That also gives DECSTR.
A RIS-only workaround in StreamHandler.vt (reset the palette when the action is
full_reset) would not cover DECSTR, because Ghostty's stream gives the handler no
action for it.

Regression test

In packages/native/src/embedded-terminal/tests.zig, add rows to the existing reset
or color table: set OSC 4 color 1, OSC 10, OSC 11, and OSC 12, then write RIS, and
check that the 4;1;? reply is the default and that the OSC 10, 11, and 12 replies
are unchanged. Add the same row for DECSTR (CSI ! p).