feat(coevolution): PR 5a â wall-time instrumentation + pretrain-arm smoke (pre-pilot polish) - #151
Merged
Merged
Conversation
Two new pre-pilot PR sections inserted ahead of PR 6 (pilot run): - **8a (PR 5a) â Pre-pilot polish**: wall-time instrumentation in `_run_one_k_block` (8a.1) so PR 6's logbook reconciliation row is populated automatically; pretrain-arm smoke validation (8a.2) to verify the heuristic-imitation pretrain path doesn't crash before launching the ~3.5-7 hour pilot arm A. - **8b (PR 5b) â Probe body wiring**: real per-opponent fitness in `_probe_one_opponent` (currently NaN). Begins with a design conversation resolving the prey-side held-out semantic ambiguity (8b.1), then implementation (8b.2), then OpenSpec sync (8b.3). PR-splitting reference table updated: 9 PRs â 11 PRs. PR 5a and PR 5b ship in either order (independent); both are pre-flight for PR 6.
âĶension + pretrain smoke Pre-pilot polish (tasks 8a.1 + 8a.2). Decoupled from PR 5b's probe wiring because these items are mechanical and dependency-free. **Wall-time instrumentation (8a.1):** - `CoevolutionLoop.__init__` initialises top-level `walltime.csv` with header `(scope, side, generation, index, parallel_workers, wall_seconds)`; pattern mirrors `generality_probe.csv`. - `_run_one_k_block` brackets `_evaluate_in_worker` dispatch with `time.perf_counter()`. Sequential path records true per-eval wall; pool path records `batch_wall / population_size` (amortised). Worker ABI unchanged â preserves the spec's "11-tuple worker tuple ABI does NOT change for co-evolution" contract. - `_record_walltime` helper writes one evaluation row per child plus a generation-aggregate row. - Aggregator extension: `_load_walltime_csv` + `_walltime_summary` (per-side mean eval/gen walls + total run wall + modal `parallel_workers_used` + n_eval/n_gen counts). - New output `walltime_summary.csv` (one row per seed) emitted alongside `verdict.csv`. - New "Wall-time reconciliation" section in `summary.md` with a markdown table per task 9.5. PR 6's reconciliation row is now populated automatically. **Pretrain-arm smoke (8a.2):** ran the heuristic-imitation pretrain bootstrap path end-to-end (smoke pop=4, K=2, gen_pairs=1 with `predator_gen0_bootstrap: heuristic_imitation_pretrain`). EXIT=0; ~141s wall (vs ~138s for cold-start). Confirms pilot arm A path is healthy before the ~3.5-7 hour campaign launches. No code change landed; documented as a pre-flight check in tasks.md 8a.2. **Smoke wall numbers** (cold-start, smoke pop=4, K=2, gen_pairs=1, parallel_workers=1): - Prey eval: 5.75s mean (15 episodes/eval = 0.38s/episode). - Predator eval: 11.49s mean (5 episodes/eval = 2.30s/episode). - Total run: 137.94s. Both sides within ~2x of design.md D4's 0.75-1.5s/episode envelope at smoke compute (parallel_workers=1; pilot will run at =4). **Tests** (5 new, 73 total in PR 5+5a tests): - `TestWalltimeInstrumentation` (2 cases): walltime.csv initialised with header at __init__; run writes the right row count (population_size eval rows + 1 gen row per side per K-block). - `TestWalltimeSummary` (4 cases): empty rows return NaN summary; per-side means + total computation; modal parallel_workers when split; end-to-end main() emits walltime_summary.csv + reconciliation table in summary.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
âĶstring drift) 3 self-review should-fix items applied; 6 informational notes left documented in the review report. **S1 â `walltime.csv` and `generality_probe.csv` truncated on resume.** `CoevolutionLoop.__init__` opened both CSVs with `"w"` mode unconditionally, wiping any prior data. On resume (which re-enters `__init__`), every prior wall-time row + every prior probe row was silently lost â `total_run_wall_seconds` for resumed seeds would understate the campaign total. Pilot is single-shot so this wouldn't bite PR 6, but the multi-day full run (PR 7) is plausible to resume. Fix: write the header only when the file doesn't already exist; on resume the prior run's rows are preserved. Added a regression test that re-instantiates the loop and verifies forged data rows in both CSVs survive the second `__init__`. **S2 â tasks.md 8a.2 wording mismatch with smoke config.** Task said "1-gen prey + 1-gen predator K-blocks" but the smoke config runs `K_per_block: 2, generation_pairs: 1` (2 generations per K-block, 1 K-block per side). Tightened the task wording to match what was actually run, including the explicit `predator_gen0_bootstrap` override and the ~3s pretrain delta vs cold-start. **S3 â aggregator docstring drift.** Top-of-file docstring listed the 5 emitted artefacts but omitted the new `walltime_summary.csv`. Added it for parity; clarified that summary.md now includes the wall-time reconciliation table, and that the verdict list includes INCONCLUSIVE. Smoke pilot still passes; 70/70 PR 5+5a tests pass; pre-commit clean; openspec strict still validates. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. ð âđïļ Recent review infoâïļ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ð Files selected for processing (1)
ð§ Files skipped from review as they are similar to previous changes (1)
ð WalkthroughWalkthroughThis change implements wall-time instrumentation for the coevolution training loop. The CoevolutionLoop now measures and logs per-evaluation and per-generation wall-clock durations to a checkpoint-safe walltime.csv. The aggregator reads these logs, computes per-seed rollups (side-specific and total wall-time metrics, modal worker count), and outputs walltime_summary.csv plus a reconciliation table in summary.md. ChangesWall-time instrumentation and aggregation
Estimated code review effortðŊ 3 (Moderate) | âąïļ ~25 minutes Possibly related PRs
Poem
ðĨ Pre-merge checks | â 5â Passed checks (5 passed)
âïļ Tip: You can configure your own custom pre-merge checks in the settings. âĻ Finishing Touchesð Generate docstrings
𧊠Generate unit tests (beta)
Comment |
chrisjz
marked this pull request as ready for review
May 9, 2026 01:26
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ðĪ Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/campaigns/aggregate_m5_pilot.py`:
- Around line 195-204: The current modal-selection of parallel_workers_used uses
Counter(...).most_common(1) which makes ties order-dependent; change the logic
in aggregate_m5_pilot.py where parallel_workers_used is set: build a Counter
from workers, determine the maximum frequency, collect all worker-values with
that frequency, and if there is a tie prefer 1 (i.e., set parallel_workers_used
= 1 when 1 is among the tied candidates); otherwise pick a deterministic value
(e.g., the single candidate). Update the block referencing Counter(workers) and
parallel_workers_used to implement this tie-break rule.
ðŠ Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
âđïļ Review info
âïļ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: d555f075-ace3-48c8-940d-31cd5cd3df0d
ð Files selected for processing (5)
openspec/changes/add-coevolution-arms-race/tasks.mdpackages/quantum-nematode/quantumnematode/evolution/coevolution.pypackages/quantum-nematode/tests/quantumnematode_tests/evolution/test_aggregate_m5_pilot.pypackages/quantum-nematode/tests/quantumnematode_tests/evolution/test_coevolution.pyscripts/campaigns/aggregate_m5_pilot.py
Codecov Reportâ Patch coverage is
ðĒ Thoughts on this report? Let us know! |
âĶllel_workers (PR 5a review) PR 5a review finding: `Counter(workers).most_common(1)[0][0]` made the modal-selection of `parallel_workers_used` order-dependent on dict iteration when multiple values shared the highest frequency. The docstring already promised "1 if mixed" (conservative sequential interpretation), but the code didn't enforce it. Fix: build counts, find max frequency, collect all tied values; if 1 is among the tied â return 1; else return `min(tied)` for deterministic output across runs. 2 new tests cover the tie-break rule (tie-with-1-prefers-1, tie-without-1-picks-smallest); the existing unambiguous-modal test stays passing. 31/31 aggregator tests pass; pre-commit clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 2
ð§đ Nitpick comments (1)
scripts/campaigns/aggregate_m5_pilot.py (1)
148-231: ðïļ Heavy liftModel the walltime payload with a typed schema instead of nested
dict[str, Any].The new walltime path relies on unstructured nested dicts, which makes key/shape regressions easy to miss. Please introduce a dedicated model (e.g.,
WalltimeSummarywith nested side metrics) and use it across parse/reduce/write/render.As per coding guidelines,
Use comprehensive type annotations in all codeandUse Pydantic BaseModel for data structures.Also applies to: 547-603, 753-765
ðĪ Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/campaigns/aggregate_m5_pilot.py` around lines 148 - 231, The _walltime_summary function currently returns and manipulates untyped nested dicts (dict[str, Any]) which is error-prone; define a Pydantic model (e.g., WalltimeSideMetrics and WalltimeSummary) to represent mean_eval_wall_seconds, mean_gen_wall_seconds, total_run_wall_seconds, parallel_workers_used, n_eval_rows and n_gen_rows, change _walltime_summary signature to return WalltimeSummary instead of dict, replace local eval_walls/gen_walls/workers with typed structures and convert computed values into the Pydantic model before returning, and update any callers (parse/reduce/write/render code paths that consume _walltime_summary) to accept the WalltimeSummary type and serialize (.dict() or .json()) when writing output; ensure all relevant functions have full type annotations referencing WalltimeSummary so the schema is enforced across the pipeline.
ðĪ Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/campaigns/aggregate_m5_pilot.py`:
- Around line 166-173: When walltime_rows is empty the summary currently sets
"parallel_workers_used": 0 which contradicts the desired "N/A" for older runs;
change the no-data return to set "parallel_workers_used" to a sentinel string
"N/A" instead of integer 0 (update the block that checks if not walltime_rows
and any other identical block around lines 753-765), and ensure any callers that
consume parallel_workers_used (or formatting code) can handle the string value
or convert it for display; reference the variable walltime_rows and the JSON key
"parallel_workers_used" when making the change.
- Around line 153-160: The docstring for the return value parallel_workers_used
is inaccurate: update the text for the "parallel_workers_used" entry (and the
duplicate doc text around the other docstring instance) to match the selection
logic used later in the file â i.e., state it is the modal value across rows; if
multiple values tie for highest frequency choose 1 if 1 is among the tied
values, otherwise choose the smallest tied value. Locate the two docstring
occurrences that describe parallel_workers_used and replace the old "1 if mixed"
wording with this precise tie-breaking description so the docs match the
selection behavior implemented in the code.
---
Nitpick comments:
In `@scripts/campaigns/aggregate_m5_pilot.py`:
- Around line 148-231: The _walltime_summary function currently returns and
manipulates untyped nested dicts (dict[str, Any]) which is error-prone; define a
Pydantic model (e.g., WalltimeSideMetrics and WalltimeSummary) to represent
mean_eval_wall_seconds, mean_gen_wall_seconds, total_run_wall_seconds,
parallel_workers_used, n_eval_rows and n_gen_rows, change _walltime_summary
signature to return WalltimeSummary instead of dict, replace local
eval_walls/gen_walls/workers with typed structures and convert computed values
into the Pydantic model before returning, and update any callers
(parse/reduce/write/render code paths that consume _walltime_summary) to accept
the WalltimeSummary type and serialize (.dict() or .json()) when writing output;
ensure all relevant functions have full type annotations referencing
WalltimeSummary so the schema is enforced across the pipeline.
ðŠ Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
âđïļ Review info
âïļ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: bd69840e-e1b6-4076-a3fb-71cb5eb4f610
ð Files selected for processing (2)
packages/quantum-nematode/tests/quantumnematode_tests/evolution/test_aggregate_m5_pilot.pyscripts/campaigns/aggregate_m5_pilot.py
ð§ Files skipped from review as they are similar to previous changes (1)
- packages/quantum-nematode/tests/quantumnematode_tests/evolution/test_aggregate_m5_pilot.py
âĶrs_used` (PR 5a review)
Two PR 5a review findings applied; one nitpick skipped with reason.
**Finding 1 â N/A sentinel for missing walltime data.** The empty-rows
return path of `_walltime_summary` set `parallel_workers_used` to
int 0, contradicting the markdown renderer's intent ("N/A" via
`wt.get('parallel_workers_used', 'N/A')` â fallback only fires when
the key is missing, not when it's 0). Older runs without
instrumentation rendered "0" in summary.md / verdict.csv, indistinct
from a real 0-worker run (impossible â `parallel_workers >= 1` per
`EvolutionConfig` schema). Switch to the string sentinel "N/A" in:
- `_walltime_summary([])` empty-rows return path.
- `_walltime_summary([...])` defensive branch where every
`parallel_workers` column was unparseable.
- `_write_walltime_summary_csv`'s fallback dict for rows lacking a
`walltime` entry.
CSV `DictWriter` round-trips strings unchanged; markdown f-string
renders "N/A" verbatim. Updated the empty-rows test assertion +
added an end-to-end test that runs main() with a header-only
walltime.csv (older-run shape) and verifies `walltime_summary.csv`
+ summary.md both show "N/A" for `parallel_workers_used`.
**Finding 2 â docstring drift.** The `_walltime_summary` return-shape
docstring still said "1 if mixed", which was the pre-tie-break-fix
wording. Updated to reflect the precise tie-break rule (modal across
rows; on ties prefer 1 if among the tied else smallest tied; "N/A"
sentinel for no data).
**Skipped â Pydantic model nitpick.** Reviewer suggested replacing
the dict-with-`Any` returns with `WalltimeSummary` Pydantic models.
Real modernisation but: the dict surface is local to this script +
test fixture; pydantic adoption here is inconsistent with the rest
of the script's idioms (other returns are also untyped dicts); the
refactor would be 200+ LoC for cosmetic typing benefit, not
contract-breaking. Defer to a future cross-script typing refactor.
32/32 aggregator tests pass; pre-commit clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 2
ðĪ Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/campaigns/aggregate_m5_pilot.py`:
- Around line 194-197: The code currently appends 1 on ValueError/TypeError when
parsing row.get("parallel_workers", 1); change this to only append to the
workers list when int(row["parallel_workers"]) succeeds (i.e., try to read raw =
row.get("parallel_workers") and if raw is not None/empty attempt int(raw) and
append the parsed int), and on exception do not append anything (treat as
missing). Also update the downstream modal/aggregate logic that computes the
modal worker value to return "N/A" when workers is empty. Apply the same change
to the other identical block around lines 219-223 so both parsing sites (the
workers list population for parallel_workers) follow this behavior.
- Around line 148-163: Replace the untyped dict schema returned by
_walltime_summary with a Pydantic model: define a BaseModel (e.g.,
WalltimeSummary) with typed fields mean_eval_wall_seconds: dict[str, float],
mean_gen_wall_seconds: dict[str, float], total_run_wall_seconds: float,
parallel_workers_used: int | Literal["N/A"], n_eval_rows: int, n_gen_rows: int
(or use explicit prey/predator submodel if preferred), update _walltime_summary
signature to return that model (or the model instance .dict() only where
strictly needed), change internal construction to instantiate WalltimeSummary
instead of building raw dicts, and update the other usages mentioned (the blocks
around lines ~224-243 and ~559-615) to accept/consume the typed model rather
than dict[str, Any], adjusting any indexing like ["prey"] to attribute access or
typed dict access as appropriate.
ðŠ Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
âđïļ Review info
âïļ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ba2d6056-0178-4433-af32-2fa98b555aa3
ð Files selected for processing (2)
packages/quantum-nematode/tests/quantumnematode_tests/evolution/test_aggregate_m5_pilot.pyscripts/campaigns/aggregate_m5_pilot.py
ð§ Files skipped from review as they are similar to previous changes (1)
- packages/quantum-nematode/tests/quantumnematode_tests/evolution/test_aggregate_m5_pilot.py
Comment on lines
+148
to
+163
| def _walltime_summary(walltime_rows: list[dict[str, Any]]) -> dict[str, Any]: | ||
| """Reduce walltime CSV rows to a per-seed summary dict. | ||
|
|
||
| Returns | ||
| ------- | ||
| `{ | ||
| "mean_eval_wall_seconds": {"prey": float, "predator": float}, | ||
| "mean_gen_wall_seconds": {"prey": float, "predator": float}, | ||
| "total_run_wall_seconds": float, | ||
| "parallel_workers_used": int | "N/A" (modal value across rows; on | ||
| ties, prefer 1 if 1 is among the tied | ||
| values, else the smallest tied value; | ||
| "N/A" sentinel when no walltime data), | ||
| "n_eval_rows": int, | ||
| "n_gen_rows": int, | ||
| }` |
There was a problem hiding this comment.
ð ïļ Refactor suggestion | ð Major | ðïļ Heavy lift
Replace wall-time nested dicts with a typed model.
The new wall-time flow is still string-keyed (dict[str, Any]), which weakens schema safety and pushes contract errors to runtime. A dedicated model for wall-time summary would make this path safer and clearer.
As per coding guidelines, "Use Pydantic BaseModel for data structures" and "Use comprehensive type annotations in all code".
Also applies to: 224-243, 559-615
ðĪ Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/campaigns/aggregate_m5_pilot.py` around lines 148 - 163, Replace the
untyped dict schema returned by _walltime_summary with a Pydantic model: define
a BaseModel (e.g., WalltimeSummary) with typed fields mean_eval_wall_seconds:
dict[str, float], mean_gen_wall_seconds: dict[str, float],
total_run_wall_seconds: float, parallel_workers_used: int | Literal["N/A"],
n_eval_rows: int, n_gen_rows: int (or use explicit prey/predator submodel if
preferred), update _walltime_summary signature to return that model (or the
model instance .dict() only where strictly needed), change internal construction
to instantiate WalltimeSummary instead of building raw dicts, and update the
other usages mentioned (the blocks around lines ~224-243 and ~559-615) to
accept/consume the typed model rather than dict[str, Any], adjusting any
indexing like ["prey"] to attribute access or typed dict access as appropriate.
âĶabricating 1 (PR 5a review)
PR 5a review finding: `_walltime_summary` previously appended `1` to
the workers list whenever `parallel_workers` was missing
(`row.get("parallel_workers", 1)`) or unparseable (`except: append(1)`).
That polluted the modal counter and masked truly-missing
instrumentation â a row without a `parallel_workers` column would
look like a sequential run.
Fix: read `raw = row.get("parallel_workers")`; skip when None / empty
string; otherwise attempt `int(raw)` inside a `contextlib.suppress`
and append on success only. When every row's column is
missing/malformed, `workers` ends up empty and the existing
"N/A" sentinel branch fires (per the prior review's tightening).
New test covers the missing / empty-string / unparseable cases all
together: 3 rows with valid wall_seconds but no parseable
parallel_workers â mean_eval_wall_seconds populates correctly,
parallel_workers_used returns "N/A".
**Skipped (Pydantic model nitpick).** Reviewer raised this for the
second time. Same rationale as before: dict-with-Any is consistent
with the rest of the script (`# pragma: no cover` glue layer); 200+
LoC refactor for cosmetic typing benefit; risks breaking working
code. Defer to a future cross-script typing refactor.
33/33 aggregator tests pass; pre-commit clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
ðĪ Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@packages/quantum-nematode/tests/quantumnematode_tests/evolution/test_aggregate_m5_pilot.py`:
- Around line 477-497: The test
test_main_walltime_summary_renders_na_without_walltime_csv is intended to
exercise the "walltime.csv missing" path but the helper _build_synthetic_session
always creates a walltime.csv; after calling _build_synthetic_session in that
test, remove the created walltime.csv (e.g., delete seed42_session /
"walltime.csv") so the code exercises the true-missing-file branch;
alternatively, update _build_synthetic_session to not create the file when
walltime_rows is None so tests that pass walltime_rows=None get a genuinely
absent file.
ðŠ Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
âđïļ Review info
âïļ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 96bcd44e-bae2-4d0d-ad7b-aba2e7124f4f
ð Files selected for processing (2)
packages/quantum-nematode/tests/quantumnematode_tests/evolution/test_aggregate_m5_pilot.pyscripts/campaigns/aggregate_m5_pilot.py
ð§ Files skipped from review as they are similar to previous changes (1)
- scripts/campaigns/aggregate_m5_pilot.py
Round 4 review: `_build_synthetic_session` always wrote walltime.csv (header-only when `walltime_rows is None`), so `test_main_walltime_summary_renders_na_without_walltime_csv` was exercising the empty-data branch rather than the truly-missing-file branch the docstring claimed. The outcome happened to match because `_load_walltime_csv` returns `[]` for both cases. Update the helper so `walltime_rows=None` skips creating the file entirely (genuinely-missing case) and `walltime_rows=[]` writes a header-only file (instrumented run that aborted before recording). The existing test now exercises the missing-file branch and gains a sanity-check assertion that the file is absent. Add a sibling test for the header-only branch so coverage of both code paths is explicit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Caution Docstrings generation - FAILED An unexpected error occurred while creating a local commit: Not Found - https://docs.github.com/rest/git/refs#get-a-reference |
Merged
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 5a of the M5 co-evolution arms-race milestone (pre-pilot polish, ahead of the pilot run in PR 6). Two mechanical, dependency-free items:
CoevolutionLoop._run_one_k_blockâ master-sidetime.perf_counter()brackets around_evaluate_in_workerdispatch. Per-eval timing for the sequential path; per-batch amortised timing for the pool path (batch_wall / population_size). Worker ABI unchanged. Per-eval + per-generation aggregate rows written to top-levelwalltime.csvwith schema(scope, side, generation, index, parallel_workers, wall_seconds).scripts/campaigns/aggregate_m5_pilot.py): readswalltime.csv, emitswalltime_summary.csv(per-seed roll-up: mean eval/gen walls per side, total run wall, modalparallel_workers_used), and adds a "Wall-time reconciliation" markdown table tosummary.md. PR 6's task 9.5 reconciliation row is now populated automatically rather than reverse-engineered from session logs.predator_gen0_bootstrap: heuristic_imitation_pretrain); EXIT=0; ~141s wall (vs ~138s for cold-start; ~3s delta covering the inline pretrain). Confirms pilot arm A path is healthy before the ~3.5-7 hour campaign launches. No code change.Why PR 5a is decoupled from PR 5b: PR 5b wires the deferred
_probe_one_opponentbody. That depends on a design conversation about prey-side held-out semantics. PR 5a's items are mechanical and have no dependency on that conversation, so they can ship first.Test plan
uv run pre-commit run --all-filesclean.openspec validate add-coevolution-arms-race --strictvalid.walltime_summary.csvwith real numbers; reconciliation table renders insummary.md.phase5_m5_coevolution_pilot.sh(out of PR 5a scope; PR 6).Reviews completed
walltime.csvandgenerality_probe.csvwere truncated on resume because__init__opened them in"w"mode unconditionally. Fixed: write header only when file doesn't exist; regression test added.1-gen K-blocksvs actualK=2).walltime_summary.csv.Deferred / out of scope
parallel_workers=1. Pool-path is exercised in real production but lacks unit coverage. Documented as a Note in the review; cheap to add (~10 LoC test) post-merge.parallel_workers, the modal value inwalltime_summary.csvwill silently mask the change. Not blocking; documented.ðĪ Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes
Tests
Documentation