Repository navigation
ci(verify): provision a Java toolchain for sdk-java PRs - #13763
Conversation
The verify lane could not execute Java unless the agent spent its own time downloading a JDK. Install JDK 21 and Maven before the agent starts, and restore a Maven repository that does not contain this repo's own jars, only when the diff touches packages/sdk-java.
|
â
Qwen Triage finished â CI landed green on â
Qwen Triage å·ēåŪæ ââ |
|
Thanks for the PR! Template looks good â â every heading present, Risk & Scope filled in honestly (the "cannot be shown until the producer has run on main" line is exactly the right thing to volunteer), and the design doc ships in both languages with reciprocal links and a 1:1 heading structure. Problem: observed, not theoretical. #13741 names five specific verify rounds (#13335, #13554, #13550, #13572, #12754) with links to the reports, including one where all 18 changed Java files went unexecuted â a regression against the previous round on the same PR. That is documented, recurring nondeterminism in a lane maintainers use for A/B verdicts, not a hypothesis. Direction: aligned. This is the repo's own CI tooling, the issue was filed by a maintainer, and the shape follows an existing precedent instead of inventing one â Size: no core paths ( Approach: the scope feels right and I could not find anything worth cutting. The obvious smaller version â an unconditional Risk: no elevated risk signals â none of the changed files match the revert-correlated path list. Moving on to code review. ð äļæčŊīææč°ĒčīĄįŪïž æĻĄæŋåŪæī â ââ åæ éĒé―å ĻïžRisk & Scope ååūåūåŪåĻïžäļŧåĻčŊīæãįžåå―äļčĶįįäš§č åĻ main äļč·čŋæč―įå°ãïžčŋæĢæŊåščŊĨčŪēæļ æĨįéĻåïžïžčŪūčŪĄææĄĢåčŊé―åĪãäšįļéūæĨãæ éĒįŧæäļäļåŊđåšã éŪéĒïž æŊå·ēč§æĩå°įïžäļæŊįčŪšæ§įã#13741 įđåäšäšč―Ūå ·ä―į verifyïž#13335ã#13554ã#13550ã#13572ã#12754ïžåđķéäšæĨåéūæĨïžå ķäļäļč―Ū 18 äļŠæđåĻį Java æäŧķåŪå ĻæēĄæ§čĄââįļåŊđåäļ PR įäļäļč―ŪæŊåéãčŋæŊįŧīæĪč įĻæĨå A/B įŧčŪšįéééå·ēčĒŦčŪ°å―ãååĪåšį°įäļįĄŪåŪæ§ïžäļæŊåčŪūã æđåïž åŊđé―ãčŋæŊäŧåščŠå·ąį CI å·Ĩå
·ïžissue įąįŧīæĪč
æåšïžčäļåæģæēŋįĻäšå·ēæįå
äūčäļæŊåĶé äļåĨââ č§æĻĄïž æēĄæč§Ķåæ ļåŋč·Ŋåūïž æđæĄïž čåīåįïžææēĄæūå°åŊäŧĨį æįéĻåãæįīč§įæīå°įæŽââæ æĄäŧķįĻ éĢéĐïž æ åįš§éĢéĐäŋĄå·ââæđåĻæäŧķé―æēĄæå―äļäļ revert įļå ģįč·ŊåūåčĄĻã čŋå ĨäŧĢį åŪĄæĨ ð â Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
Code reviewI wrote down what I would have done before reading the diff: gate on the changed-file list, download a checksum-pinned Temurin + Maven before the agent's clock starts, warm
No critical blockers. Correctness, the supply-chain surface, and the failure paths all hold up. Specifics I verified rather than assumed:
Two non-blocking things I'd want you to consider, neither of which I think should hold the merge:
Files changed (8)
TestingThis is an unattended CI run, so per the lane's rules I did not build or execute anything from this PR â the evidence below is the PR's own CI, read through the API for the reviewed commit. 61 check-runs in total: 10 reported a real conclusion, 50 were skipped (the platform and event matrices that this diff does not select), 1 is queued and 2 are still in progress, and none failed or were cancelled. The two that matter most for this change â Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / æŊäļŠæĢæĨåäļčĄïžåææ°äļæŽĄčŋčĄïžïžįįĨ skippedïžåĪąčīĨéĄđæåĻæåã Because those two are unfinished, I am explicitly recording what this review does not carry: no local execution of either workflow test suite, and no observation of the producer or the provisioning path running. The author's rehearsal numbers in the PR body (Lima Sandboxed verification would settle part of this, and it is worth being precise about which part: äļæčŊīæäŧĢį åŪĄæĨ åĻčŊŧ diff äđåïžæå
åäļäščŠå·ąäžæäđåïžææđåĻæäŧķåčĄĻéĻæ§ãåĻäŧĢįčŪĄæķåžå§åäļč――éįåđķæ ĄéŠį Temurin äļ Mavenãįą main äļįåŊäŋĄįäš§č
éĒį
æēĄæéŧææ§éŪéĒã æĢįĄŪæ§ãäūåšéūéĒãåĪąčīĨč·Ŋåūé―įŦåūä―ãäŧĨäļæŊæåŪé æ ļåŊđčŋãčäļæŊæģå―įķįįđïž
äļĪįđäļéŧæãä―æåļæä― ččįäšïžæčŪĪäļšé―äļčŊĨæĶä―ååđķïž
æĩčŊ čŋæŊäļæŽĄæ äššåžåŪį CI čŋčĄïžæééč§åææēĄææåŧšææ§čĄæŽ PR įäŧŧä―äŧĢį ââäļéĒįčŊæŪæŊčŋäļŠ PR čŠå·ąį CIïžéčŋ API éåŊđčĒŦåŪĄæäšĪčŊŧåįãå ą 61 äļŠ check-runïž10 äļŠįŧåšäšįåŪįŧčŪšïž50 äļŠčĒŦč·ģčŋïžæŽ diff æŠéäļįåđģå°äļäšäŧķįĐéĩïžïž1 äļŠæéã2 äļŠäŧåĻčŋčĄïžæēĄæåĪąčīĨæåæķã åŊđčŋäļŠæđåĻæå
ģéŪįäļĪäļŠââ å äļščŋäļĪäļŠå°æŠåŪæïžææįĄŪåäļæŽæŽĄåŪĄæĨäļå
åŦäŧäđïžæēĄæåĻæŽå°æ§čĄäŧŧä―äļåĨå·Ĩä―æĩæĩčŊïžäđæēĄæč§åŊå°įäš§č
æäūįŧč·ŊåūįæĢčŋčĄãPR æĢæéä―č
įæžįŧæ°æŪïžLima æēįŪąéŠčŊč―ååŪå
ķäļäļéĻåïžåžåūčŊīæļ
æŊåŠäļéĻåïž â Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
|
Confidence: 4/5 â solid and better thought through than my own baseline; the two nits in Stage 2 are questions about a cold-cache path and a cache-key collision, neither of which can corrupt a verdict. Going back to what I wrote down before reading the diff: the PR matches it and then beats it. My version would have installed the toolchain somewhere convenient, exported the flag, and called it done. It would have left three holes this one closes â the The thing I keep coming back to as the strongest signal is Is every change necessary? Yes. I looked for scope creep specifically and found none â the one edit outside the feature that is not a mechanical baseline bump is a skill paragraph this change made untrue, and leaving it would have told the agent the image ships no JDK. Six months from now I would thank the author, not curse them. The producer/consumer split mirrors What I am not confident about, stated plainly: the positive path has never been observed end to end, and cannot be before merge. The provisioning only runs for a diff touching I would merge this. The unit suites and lint were still in flight when I finished, so I am not approving against a result that does not exist yet â approval is deferred until CI lands green on äļæčŊīæConfidence: 4/5 ââ æåŪïžčäļæŊæčŠå·ąįåšįšŋæđæĄæģåūæīåĻå ĻïžStage 2 éįäļĪįđæŊå ģäšå·įžåč·Ŋåūåįžå key åēįŠįįéŪïžäļĪč é―äļäžæąĄæįŧčŪšã åå°æåĻčŊŧ diff äđååäļįæđæĄïžčŋäļŠ PR äļåŪäļčīïžåđķäļæīåĨ―ãæįįæŽäžæå·Ĩå
·éūčĢ
åĻäļäļŠéĄšæįä―į―ŪãåŊžåšåžå
ģãå°ąįŪåŪäšãéĢäžįäļäļäļŠčŋäļŠ PR čĄĨäļįæīââ æååĪæģå°ãäđčŪĪäļšæč―čŊīæéŪéĒįäļįđæŊ æŊäļåĪæđåĻé―åŋ čĶåïžæŊįãæäļéĻæūčŋčåīčåŧķïžæēĄæūå°ãįđæ§äđåĪåŊäļäļæŊæšæĒ°åšįšŋæīæ°įæđåĻïžæŊčĒŦčŋæŽĄæđåĻæđåäļšäļåŪįéĢæŪĩæč―čŊīæââįįåŪå°ąäžåčŊäŧĢįéåéæēĄæ JDKã å
äļŠæåæäžæč°Ēä―č
ïžčäļæŊéŠäŧãįäš§č
/æķčīđč
įæåäļ æäļįĄŪåŪįå°æđïžįīčŊīïžæĢåč·ŊåūäŧæŠčĒŦįŦŊå°įŦŊč§åŊå°ïžååđķåäđæ æģč§åŊãäūįŧéŧčūåŠåĻ diff įĒ°å° æäžååđķåŪãæįåŪæķåæĩå lint čŋåĻč·ïžæäŧĨæäļäžéåŊđäļäļŠå°äļååĻįįŧæåŧæđåââæđåæĻčŋå° CI åĻ â Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
|
@qwen-code /verify |
qwen-code-review-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship â CI landed green after the review. â
|
Sandboxed verification: â not passed â findings reported (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers â not a review, an approval, or a CI check. Scripted assertions: 160 passed · 0 failed · 160 total Flakiness gate: â 2 changed test file(s) x 5 identical rounds, no divergence äļæ â åĪåŪïžâ äļéčŋ · æĨåäšåį°ïžagent åĪåŪïžæēįŪąéŠčŊåĻéįĶŧãæ åčŊįåŪđåĻäļæ§čĄäščŊĨ PR įäŧĢį ïžäļ base æåŧš A/B åŊđį §ãæ mock harness æčĻãåŪåéĻįĶïžãäŧ ä―äļščŊåŪĄčŊæŪïžäļææčŊåŪĄãæđåæ CI æĢæĨã čæŽæčĻïž160 éčŋ · 0 åĪąčīĨ · 160 æŧčŪĄ æåĻéĻïžâ 2 changed test file(s) x 5 identical rounds, no divergence Verification reportPR #13763 deep verification â
|
| fixture | base java |
head java |
head decision=na |
|---|---|---|---|
| this PR's real file list (no sdk-java) | false | false | â |
packages/sdk-java/qwencode/âĶ + pom |
false | true | â |
| sdk-java only, deeply nested test file | false | true | â |
packages/core/src/index.ts |
false | false | â |
docs-only (README.md, docs/index.md) |
false | false | yes |
near-miss prefixes (packages/sdk-java-docs/, sdk-java/, packages/sdk-javax/) |
false | false | â |
packages/sdk-java/README.md only |
false | false | yes (docs-only wins) |
| 401 files, sdk-java entry last (SIGPIPE shape) | false | true | â |
16/16 cells as predicted: head flips on exactly the sdk-java inputs, base never provisions for any input. The last row is the one the code comment claims to defend (grep <<< rather than a pipe, which under pipefail previously misclassified code PRs as docs-only) â it holds with the match at the end of a 401-line list.
Toolchain step â verbatim, against the real published archives
The step's run: block was executed as extracted, with a stub curl serving the real archives downloaded from the two pinned URLs, and real sha256sum/sha512sum/tar/find/java/mvn.
| cell | exit | ready |
prefix dir | oracle |
|---|---|---|---|---|
| success (real archives) | 0 | written | present | openjdk version "21.0.12.1", Apache Maven 3.9.11, 2 curl calls, no warning |
| JDK archive corrupted | 0 | absent | removed | ::warning::Java toolchain install failed |
| Maven archive corrupted | 0 | absent | removed | same warning |
| download fails (curl exit 7) | 0 | absent | removed | same warning |
The three failure cells are the load-bearing proof that the checksum gate is not decorative: a single flipped byte in either archive removes the marker, which is what the agent's env injection keys on. All four exit 0, so a Java failure can never fail a TypeScript verification.
Agent env injection â verbatim QWEN_ENV region, both arms
| cell | head entries | QWEN_VERIFY_JAVA |
PATH entries |
base arm |
|---|---|---|---|---|
ready present |
27 | 1 |
2, java one last | no injection |
ready absent |
23 | absent | 1 | no injection |
The base arm was run in both cells, including the positive one, per the symmetric-control rule. The comment's claim that "env applies QWEN_ENV left to right, so this PATH overrides the one above" was measured independently: env -i PATH=/first PATH=/second printenv PATH â /second.
Sibling module install â credential stripping measured at the real seam
The mvn stub dumps its own environment, i.e. what survives runuser -u node -- env -u âĶ. ACTIONS_RUNTIME_TOKEN, ACTIONS_RUNTIME_URL and ACTIONS_CACHE_URL were deliberately planted in the parent with sentinel values so a missing -u would be observable rather than assumed absent.
| cell | exit | mvn calls | last line of java-prepare.log |
planted credential in child |
|---|---|---|---|---|
| toolchain missing | 0 | 0 | toolchain=missing |
n/a |
| ready, mvn succeeds | 0 | 2 | qwencode=0 runtime-broker=0 |
none |
| ready, mvn fails | 0 | 1 each, both fail | qwencode=1 runtime-broker=1 |
none |
Both installs use the restored repo as -Dmaven.repo.local, qwencode gets -Dgpg.skip + -Dmaven.javadoc.skip, runtime-broker gets -Dspotbugs.skip, and the step summary carries verify Maven repo: hit=âĶ key=âĶ in all three cells including the cache-miss one.
Live-lane negative arm (production measurement, no harness)
This PR touches .github/workflows/, .qwen/skills/, docs/ and scripts/ â not packages/sdk-java/ â so the gate was off for this very run, and the container is a live sample of the lane:
- no
QWEN_VERIFY_JAVA,JAVA_HOMEorMAVEN_ARGSin the agent's/proc/self/environ; nojava/mvnon PATH /opt/verify-javanever created;/__w/_temp/verify-maven-reponever created; nojava-prepare.logbeside$QWEN_VERIFY_CONTEXT$RUNNER_TEMPis/__w/_temp(derived fromQWEN_VERIFY_CONTEXT=/__w/_temp/verify-context/pr.json), and/home/runnerdoes not exist in the container
Corrections to the PR description
Labelled as corrections to the description, not requests to change code.
- "The node:test file also has pre-existing macOS failures âĶ the timeout assertion itself passed." On this Linux lane the whole file is green:
# tests 125 / # pass 125 / # fail 0. The macOS caveat does not apply here and could mislead a reader into discounting a rednode --testrun on Linux. - "The pinned x64 Temurin archive checksum matched the published file." Confirmed independently, and extended: the Temurin sha256 and the Maven sha512 both match the files the pinned URLs actually serve, byte for byte. Both archive layouts also satisfy the step's own post-extract checks (
jdk-21.0.12.1+1/,apache-maven-3.9.11/). - "
qwencodeplusruntime-brokerinstalled from the merge ref" is accurate for the consumer, but the producer populates the repo from all four modules (managed-agent-serverandclientviaverify). The strip list correctly covers all four artifactIds, so an agent testing those two modules has their third-party dependencies but not their own artifacts â which the HEAD skill text does tell the agent to reinstall. Not a defect; the description just understates what the producer warms.
Findings
1. Medium â the job budget line understates the capped worst case by ~31 minutes
.github/workflows/qwen-triage.yml, the timeout-minutes comment block:
# java toolchain + Maven repo
# + sibling modules âĪ ~20m, and only when the diff touches
# packages/sdk-java/. curl --max-time caps
# each download; `timeout 300` caps each
# mvn install
# worst case ~200m â 210 leaves 10m of headroom.
curl --max-time caps each attempt, and --retry 3 retries on a timeout, so the cap multiplies. Measured in this image (curl 7.88.1) against a server that accepts and never replies, with the flags scaled down 100Ã (curl-retry-measurement.txt):
--retry 3 --retry-delay 2 --max-time 3 â elapsed=18s (= 4Ã3 + 3Ã2 exactly)
--retry 3 --retry-delay 2 --max-time 3 --retry-max-time 7 â elapsed=8s
At the step's real values:
| item | cap |
|---|---|
one download (1+3)Ã300 + 3Ã2 |
1206 s = 20.1 m |
| two downloads (JDK + Maven) | 40.2 m |
two timeout -k 30 300 sibling installs |
660 s = 11.0 m |
| Java worst case | ~51.2 m (line says âĪ ~20 m) |
| cache restore of the Maven repo | not priced at all |
A single download's cap already exceeds the whole Java budget line. Against the file's own pre-PR items (120 m agent + 15 m install/build + 40 m gate + 5 m misc = 180 m) the worst case is ~231 m versus a 210 m limit â i.e. the 20-minute raise does not cover the addition it was raised for. The consequence is the one the comment itself names: the job timeout kills the container mid-run and bypasses the ship-what-ran path, so a Java PR in a network-degraded window loses its report entirely. The timeout-minutes >= 210 assertion in .github/scripts/qwen-triage-workflow.test.mjs encodes the understated figure, so the existing guard cannot catch this.
This is a budget-robustness finding, not a correctness bug: it needs a black-hole network (packets dropped, not refused) to reach, and the ordinary failure mode â a fast 404 or DNS error â is cheap. But that regime is exactly what --max-time --retry exists for, and this file's convention elsewhere is to sum the caps (the flakiness-gate line itemises -k 10 30, -k 30 120, -k 30 600).
Measured candidate fix (one flag per download)
--retry-max-time exists in this image's curl 7.88.1 and caps the whole retry loop. Adding --retry-max-time 300 to both curl invocations makes each download's cap 300 s, so the Java worst case becomes ~5 m + ~5 m + 11 m â 21 m â the stated "âĪ ~20m" becomes true and no limit raise is needed. Verified direction: the scaled probe above drops from 18 s to 8 s when the flag is added, with the same exit code 28 and the same warning path. Not applied to the PR; the arithmetic above is what a maintainer should re-check.
- if curl -fsSL --retry 3 --retry-delay 2 --max-time 300 -o "$work/jdk.tar.gz" "$TEMURIN_URL" \
+ if curl -fsSL --retry 3 --retry-delay 2 --max-time 300 --retry-max-time 300 -o "$work/jdk.tar.gz" "$TEMURIN_URL" \2. Low â the PR's cited test command cannot pass inside the verify lane
The Reviewer Test Plan gives:
npx vitest run --config ./scripts/tests/vitest.config.ts scripts/tests/qwen-triage-workflow.test.js \
-t "qwen-triage verify Java toolchain|strips GitHub command files"
Run verbatim in this lane: 1 failed | 8 passed | 168 skipped. The failure is expect(verifySkill).toContain('QWEN_VERIFY_JAVA=1'). Run in a scratch worktree at HEAD content: 9 passed.
The cause is the lane, not the PR: Pin agent inputs from base replaces .qwen with git archive HEAD^1 -- .qwen and re-pins it root-owned and read-only (dr-xr-xr-x root root, verified), with the workflow's own note that "A PR that edits the skill is verified with the base version by design". So a PR that edits the verify skill is, by construction, unable to satisfy its own skill-content assertions in this lane.
Two consequences worth naming:
- The Test Plan step is not performable here. A reviewer following it inside a verify run will see red and has no way to tell the lane artifact from a real regression without repeating the run against unpinned content.
- The flakiness gate's signal for this file is degraded.
scripts/tests/qwen-triage-workflow.test.jsis a changed test file, so the gate re-runs it in the pinned tree across identical rounds. It will see the same failure every round â "no divergence", so the gate stays quiet â but the file can no longer distinguish a genuine break from the pinning artifact.
Not a request to weaken the pin, which is a real security property. A one-line note in the Test Plan ("these four assertions require an unpinned .qwen; inside the verify lane they fail by design") would close the gap, or the skill-content assertions could read the file from HEAD via git rather than from the working tree.
3. Suggestion â the new suite does not pin the Maven sha512 verification
Mutation matrix: 17 caught / 2 survived / 19, unmutated control green (9 passed), and a positive control (timeout-minutes: 210 â 190) caught in the same file as the survivors. Two mutants needed re-anchoring after their first anchors matched 2Ã and 0Ã; both were caught once applied.
| survivor | classification | evidence |
|---|---|---|
delete sha512sum --check --status |
coverage gap | suite stays 9 passed; driving the mutant build with a well-formed but substituted Maven distribution and the real pin still in force writes ready (ready=true), where head refuses it (ready=false + warning) |
delete "$prefix/jdk/bin/java" -version |
redundant defence | driving the mutant with a broken JVM gives ready=false, identical to head, because mvn -version fails first â and that line is pinned, via expect(javaHomeAt).toBeLessThan(toolchain.indexOf('"$prefix/maven/bin/mvn" -version')) |
The asymmetry is the finding: the Temurin sha256 check is pinned by an explicit toBeGreaterThan(-1), while the Maven sha512 check is only pinned by an ordering assertion of the shape indexOf(x) < readyAt, which -1 satisfies. The same shape leaves java -version unpinned, though there a sibling assertion covers the hazard. Head's behaviour is correct â the tamper cell in the A/B table above proves the sha512 gate fires â so this is a test to write, not code to fix. The fixture that would go red: assert expect(toolchain.indexOf('sha512sum --check --status')).toBeGreaterThan(-1) alongside the existing sha256 line.
Caught mutants, for completeness: timeout floor; sha256 removal; mvn -version removal; unanchored grep; herestringâpipe; cache/restoreâcache; dropped continue-on-error; dropped acp-sdk from the strip list; removed QWEN_VERIFY_JAVA=1; removed MAVEN_ARGS; dropped timeout -k 30 300; dropped the ACTIONS_RUNTIME_* strip; producer cache path changed; producer moved to ubuntu-latest; dropped -Dspotbugs.skip; exit 1 on download failure; if true in place of the ready-marker guard.
Nit â one java-prepare.log state the skill contract does not describe
Read from the code, not measured: if chown -R node:node "$repo" fails, the step logs toolchain=chown-failed and exits before any mvn, so the log never gets the qwencode=<n> runtime-broker=<n> last line â yet /opt/verify-java/ready still exists, so the agent does receive QWEN_VERIFY_JAVA=1. The HEAD skill text tells the agent the exit codes are the last line and to reinstall "if either code is non-zero", which does not cover a missing line. Low likelihood (needs a non-root step or a read-only $RUNNER_TEMP).
Not covered
- Any real
mvnexecution. The gate is off for this PR, so no JDK was provisioned in this lane andQWEN_VERIFY_JAVAis unset â per the base-pinned skill, the Java side is not covered. Themvninvocations were verified as command lines and environment through a stub at the realrunuserseam; no module was compiled or tested. The PR description's ownmvn testcounts (208 / 754 / 1185) were not reproduced. - A real cache hit. Requires the producer to have saved on
mainfirst; the PR says so too. What was proven instead is that the mechanism can fire:getCacheVersionin@actions/cache@4.1.0(the version the pinned action SHA depends on) hashes the literal resolved paths + compression method + salt; producer and consumer shareruns-on, container image andoptions: '--init --user node'with the already-in-productionpnpm-store.ymlpair; neither image ships azstdbinary, so both fall back identically; and no/home/runnerhost path exists here, so$RUNNER_TEMPand${{ runner.temp }}are the same directory (/__w/_temp). The restore-only entrypoint's own manifest has nopost:step while the full action's haspost: 'dist/save/index.js'â read from the manifests at the pinned SHA, not from the PR's description. Residual not measured: an actual round-trip through the cache service. - Database-backed Java tests, the hosted harness, the Java 11/17 matrix, and re-running Java tests through the flakiness gate â all declared out of scope by the PR; not attempted.
actionlintandyamllint. Not installed in this container andnode scripts/lint.js --setupwas not run for budget reasons.bash -nwas run on all three producerrun:blocks (all OK), and the consumer's changed blocks were executed, which is stronger. The size ratchet was run and its liveness proven (planting100forverify-maven-repo.ymlyields exit 1 withgrew to 5575 bytes); as committed both entries are byte-exact (362377, 5575) and the gate exits 0.- Producer end-to-end.
verify-maven-repo.ymlwas parsed, syntax-checked and its strip list checked against the real coordinates (acp-sdk,qwen-managed-agent-server,qwen-managed-runtime-broker,qwencode-sdk, allcom.alibaba, all fixed non-SNAPSHOT versions â which is what makes the strip necessary). It was never executed: that needs a cold JDK plus a full Spring Boot resolve. Its day-one cost was not modelled either; the trigger is narrow (packages/sdk-java/*/pom.xmlor the workflow itself, onmain), so arrival rate is low, but confirming that needsgh apievent history, which this token-free environment does not have. - One harness limitation, stated for the record. The toolchain and modules scripts were executed with
prefix=/opt/verify-javarewritten to a sandbox path, because/optis root-owned and this agent is uid 1000 with noCAP_SYS_ADMIN(unshareâOperation not permitted). Everything else is byte-identical to the extracted step. The security property the rewrite gives up was measured directly against the live lane instead:mkdir -p /opt/verify-javaasnodeâPermission denied, whilerun:steps in this job do execute as root (/etc/apt/sources.list.d/github-cli.listis root-owned), so the step can write/optand PR code cannot forge the marker.
Methodology
Everything ran inside the lane's own node:22-bookworm container at the merge ref (head 1d81dfa7, base 4360b003), as uid 1000, with the pre-built tree already present. Three harnesses, all kept in this directory as .mjs so a maintainer can rerun them: ab-java-toolchain.mjs (103 assertions â arm shape, gate A/B over 8 fixtures à 2 arms, 4 toolchain cells against the real published archives, agent-env injection in both arms, sibling-install cells, cache identity) parses both workflow revisions from git show with the yaml package, extracts the run: blocks verbatim and executes them under bash --noprofile --norc keeping each step's own set line, stubbing only external tools (curl, runuser, mvn, chown) and asserting on their recorded call logs and child environments; followups.mjs (11) re-applies the two mis-anchored mutants and adjudicates both survivors by building and driving the mutant scripts against stand-in archives; gates.mjs (46) covers the supply-chain pins against the real downloaded files, the actions/cache manifests and getCacheVersion source at the pinned SHA, the strip list against the real POM coordinates, the repo's own gates and the size ratchet plus its liveness, and the live-lane negative arm. Repo gates were run separately: node --test .github/scripts/qwen-triage-workflow.test.mjs (125/125) and the full scripts/tests/qwen-triage-workflow.test.js + scripts/tests/workflow-size.test.js at HEAD content in a scratch worktree (400/400), plus the PR's cited filter both in the pinned lane tree and unpinned. Raw logs are ab-java-toolchain.log, mutation-matrix.log, followups.log, gates.log, gate-*.log, curl-retry-measurement.txt; third-party manifests and the real archive hashes are under vendor/.
Flakiness gate log
rounds=5 files=2 skipped=0
file .github/scripts/qwen-triage-workflow.test.mjs: (cd .) node --test ./.github/scripts/qwen-triage-workflow.test.mjs
file scripts/tests/qwen-triage-workflow.test.js: (cd .) npx --no-install vitest run --config ./scripts/tests/vitest.config.ts ./scripts/tests/qwen-triage-workflow.test.js
per-file results (P=pass F=fail I=infra-exit, one letter per run):
.github/scripts/qwen-triage-workflow.test.mjs: PPPPP
scripts/tests/qwen-triage-workflow.test.js: PPPPP
verdict: pass
summary: 2 changed test file(s) x 5 identical rounds, no divergence
--- per-invocation detail (full copy in the artifact) ---
round 1 · .github/scripts/qwen-triage-workflow.test.mjs: P (exit 0)
round 1 · scripts/tests/qwen-triage-workflow.test.js: P (exit 0)
round 2 · .github/scripts/qwen-triage-workflow.test.mjs: P (exit 0)
round 2 · scripts/tests/qwen-triage-workflow.test.js: P (exit 0)
round 3 · .github/scripts/qwen-triage-workflow.test.mjs: P (exit 0)
round 3 · scripts/tests/qwen-triage-workflow.test.js: P (exit 0)
round 4 · .github/scripts/qwen-triage-workflow.test.mjs: P (exit 0)
round 4 · scripts/tests/qwen-triage-workflow.test.js: P (exit 0)
round 5 · .github/scripts/qwen-triage-workflow.test.mjs: P (exit 0)
round 5 · scripts/tests/qwen-triage-workflow.test.js: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
â Qwen Code · sandboxed verification
yiliang114
left a comment
There was a problem hiding this comment.
Reviewed at head f4841fe â LGTM.
The trust model is the load-bearing part of a verify-lane change, and it holds up:
- Cache poisoning is closed on both sides: the consumer is restore-only (a save step in a job that executes PR code would let a PR write the shared cache), and the producer strips this repo's own artifacts before saving â necessary because the module versions are fixed, not SNAPSHOT, so a main-built jar would otherwise be what Maven resolves for the PR tree.
- The ready marker lives under root-owned /opt, so the node user (which runs PR code) cannot plant it, and the toolchain install happens pre-checkout with no PR tree on disk â the same supply-chain surface as the existing runner-tools step.
- Credential hygiene in the sibling-install step is explicit: GITHUB_TOKEN/GH_TOKEN blanked, ACTIONS_* cache tokens unset under runuser, each mvn capped by timeout, and the step always exits 0 so a sibling failure degrades Java to Not covered instead of failing a TypeScript verdict.
- The
^packages/sdk-java/gate is deliberately narrower than sdk-java.yml's path list, and the comment says why (those paths are mostly core/CLI, and this lane cannot host the database-backed ITs anyway) â other PRs download nothing. - The SKILL.md contract tells the agent the non-SNPAPSHOT A/B trap (reinstall siblings from the base worktree, or a copied repo) and pins the failure semantics (
QWEN_VERIFY_JAVAunset with an sdk-java diff means the toolchain failed â report Not covered, do not self-provision). - The 190â210 timeout raise states its cost in the comment (an ECS slot held ~3.5h instead of 1h), which is the right way to land it.
Producer/consumer runner and image are matched so cache entries are restorable, and the workflow-test pins (including the strip markers) keep the two copies from drifting. CI on this head: no failures; several lanes still running.
âĶe guidance (QwenLM#13791) (QwenLM#13792) PR QwenLM#13763 provisioned a JDK 21 + Maven toolchain for sdk-java diffs and pinned the verify-pr skill to never claim the verify image ships no JDK, but the "Java-centred PRs" bullet added by the concurrently merged QwenLM#13756 still made exactly that claim, failing the qwen-triage workflow test on main. Repoint the bullet at the QWEN_VERIFY_JAVA=1 gate in the environment contract and keep the probe-then-Not-covered guidance for a failed toolchain install. Co-authored-by: qwen-code-dev-bot <qwen-code-dev-bot@users.noreply.github.com> Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
Released in v0.25.1. |


What this PR does
The sandboxed verify lane now prepares a Java toolchain before the agent's clock starts, and only when the pull request diff touches
packages/sdk-java. The agent gets Temurin JDK 21, Maven 3.9.11, a restore-only Maven repository, andqwencodeplusruntime-brokerinstalled from the merge ref. Other pull requests download nothing.The repository is filled by a trusted producer on main and restored read-only. This repo's own jars are removed before the cache is saved, because the module versions are fixed and a jar built from main would otherwise be what Maven resolves. The skill's environment contract tells the agent what the lane provides, including that an A/B must reinstall the sibling modules for the base side, and that database-backed integration tests stay not covered.
Why it's needed
Whether a Java pull request was actually executed in this lane depended on whether that round's agent spent its own time budget downloading a JDK. Some rounds did, some did not, and a later round could silently stop executing Java that an earlier round had executed.
sdk-java.ymlalready tests Java on pull requests. The gap is the verify agent's own A/B verdict.Reviewer Test Plan
How to verify
Confirm a diff outside
packages/sdk-javadoes not download a JDK or restore the Maven repository. Confirm a Java diff reaches the agent withQWEN_VERIFY_JAVA=1,JAVA_HOME, andMAVEN_ARGSset, or, when the download fails, without those variables and with a warning, while the agent still runs. Confirm the verify job never saves the Maven cache.The cache hit itself can only be observed after this merges and the producer has run on main: comment
@qwen-code /verifytwice on the same Java pull request. The second step summary should containverify Maven repo: hit=true, the report should execute the Java side, and the agent should not download a JDK.Unit coverage:
npx vitest run --config ./scripts/tests/vitest.config.ts scripts/tests/qwen-triage-workflow.test.js -t "qwen-triage verify Java toolchain|strips GitHub command files"andnode --test .github/scripts/qwen-triage-workflow.test.mjsfor the timeout floor (timeout-minutes >= 210). The node:test file also has pre-existing macOS failures in the flakiness-gate shell fixtures (/usr/bin/rmmissing under that harness); the timeout assertion itself passed.Evidence (Before & After)
N/A
Tested on
Environment (optional)
macOS: the workflow unit tests above. Linux: Lima
node:22-bookworm(aarch64). The pinned x64 Temurin archive checksum matched the published file. On that arm64 guest the x64 binary cannot execute, and the script correctly leaves no ready marker. The same Temurin release's aarch64 build, afterJAVA_HOMEis exported, writes the marker: Java21.0.12.1, Maven3.9.11. Sibling installs ofqwencodeandruntime-brokersucceeded as thenodeuser.mvn testwith only the agent environment:qwencode208 tests, 0 failures.runtime-broker754 tests with 1 failure and 1 error.managed-agent-server1185 tests with 1 error. Those two modules include tests that start a Node fixture or a built CLI, and this rehearsal did not runnpm run buildfirst. The verify lane builds the TypeScript app before the agent starts.Risk & Scope
/verifyof the same PR cannot be shown until the producer has saved on main.QWEN_VERIFY_JAVAis unset.Design: docs/design/verify-java-toolchain.md and docs/design/verify-java-toolchain.zh-CN.md.
Linked Issues
Closes #13741
äļæčŊīæ
čŋäļŠ PR åäŧäđ
æēįŪą verify ééį°åĻäžåĻäŧĢįčŪĄæķåžå§äđåååĪ Java å·Ĩå ·éūïžåđķäļåŠåĻ PR diff įǰå°
packages/sdk-javaæķæåãäŧĢįäžæŋå° Temurin JDK 21ãMaven 3.9.11ãåŠæĒåĪį Maven äŧåšïžäŧĨåæååđķåžįĻčĢ åĨ―įqwencodeåruntime-brokerãå ķäŧ PR äļäžäļč――čŋäšäļčĨŋãäŧåšįą main äļįåŊäŋĄįäš§č åĄŦåĨ―ïžæķčīđäū§åŠæĒåĪãäŋååäžå ææŽäŧåščŠå·ąį jarïžå äļšæĻĄåįæŽæŊåšåŪįïžåĶå Maven äžč§Ģæå° main äļæåŧšåšæĨįéĢäļäŧ―ãæč―įįŊåĒįšĶåŪåæééæäūäŧäđïžå æŽ A/B æķåŋ éĄŧäļš base äļäū§éčĢ åįš§æĻĄåïžäŧĨåäūčĩæ°æŪåšįéææĩčŊäŧįķįŪæŠčĶįã
äļšäŧäđéčĶ
äļæĄ Java PR åĻčŋäļŠéééææēĄæčĒŦįæĢæ§čĄïžååģäšéĢäļč―ŪäŧĢįčŊäļčŊčąčŠå·ąįæķéīåŧäļč―― JDKãæįč―ŪæŽĄäžäļč――ïžæįäļäžïžåäļč―ŪčŋåŊč―ææåæåäļč―Ūå·ēįŧæ§čĄčŋį Javaã
sdk-java.ymlå·ēįŧåĻ PR äļæĩčŊ JavaãįžšåĢæŊ verify äŧĢįčŠå·ąį A/B įŧčŪšãåŪĄé č æäđéŠčŊ
įĄŪčŪĪ
packages/sdk-javaäŧĨåĪį diff äļäžäļč―― JDKïžäđäļäžæĒåĪ Maven äŧåšãįĄŪčŪĪ Java diff å°čūūäŧĢįæķå·ēčŪūį―ŪQWEN_VERIFY_JAVA=1ãJAVA_HOMEåMAVEN_ARGSïžäļč――åĪąčīĨæķåæēĄæčŋäšåéïžåđķæ warningïžåæķäŧĢįäŧįķäžč·ãįĄŪčŪĪ verify äŧŧåĄäŧäļäŋå Maven įžåãįžåå―äļåŠč―åĻååđķäđåãįäš§č å·ēåĻ main äļč·čŋæč―įå°ïžåŊđåäļäļŠ Java PR čŋįŧäļĪæŽĄčŊčŪš
@qwen-code /verifyãįŽŽäšæŽĄįæĨéŠĪæčĶåšå åŦverify Maven repo: hit=trueïžæĨååšæ§čĄ Java äū§ïžäŧĢįäļåšåäļč―― JDKãåæĩïž
npx vitest run --config ./scripts/tests/vitest.config.ts scripts/tests/qwen-triage-workflow.test.js -t "qwen-triage verify Java toolchain|strips GitHub command files"ïžäŧĨånode --test .github/scripts/qwen-triage-workflow.test.mjséįčķ æķäļéïžtimeout-minutes >= 210ïžãčŋäļŠ node:test æäŧķåĻ macOS äļčŋææĒæįæåĻéĻ shell åĪđå ·åĪąčīĨïžčŊĨįŊåĒéæūäļå°/usr/bin/rmïžïžčķ æķæčĻæŽčšŦæŊéčŋįãčŊæŪ
äļéįĻã
æĩčŋįįŊåĒ
macOS äļč·äšäļéĒįå·Ĩä―æĩåæĩãLinux äļįĻ Lima į
node:22-bookwormïžaarch64ïžãéæŧį x64 Temurin å―æĄĢæ ĄéŠåäļååļæäŧķäļčīãåĻčŋå° arm64 åŪĒæ·æšäļ x64 äščŋåķæ æģæ§čĄïžčæŽæĢįĄŪå°äļå readyãåäļ Temurin įæŽį aarch64 æåŧšåĻåŊžåšJAVA_HOMEäđåäžå readyïžJava21.0.12.1ïžMaven3.9.11ãäŧĨnodeįĻæ·åŪčĢqwencodeåruntime-brokeré―æåãåŠįĻäŧĢįįŊåĒč·mvn testïžqwencode208 äļŠæĩčŊã0 åĪąčīĨãruntime-broker754 äļŠæĩčŊéæ 1 äļŠåĪąčīĨå 1 äļŠéčŊŊãmanaged-agent-server1185 äļŠæĩčŊéæ 1 äļŠéčŊŊãåäļĪäļŠæĻĄåå åŦåŊåĻ Node fixture æå·ēæåŧš CLI įæĩčŊïžčŋæŽĄæžįŧæēĄæå č·npm run buildãverify ééåĻäŧĢįåžå§åäžå æåŧš TypeScript åšįĻãéĢéĐäļčåī
/verifyįįžåå―äļïžčĶįįäš§č åĻ main äļäŋåäđåæč―įå°ãQWEN_VERIFY_JAVAäļååĻæķæ Java č§äļšæŠčĶįãčŪūčŪĄææĄĢïždocs/design/verify-java-toolchain.md äļ docs/design/verify-java-toolchain.zh-CN.mdã
å ģč Issue
Closes #13741