Refactor: Consolidate duplicated RolloutBuffer implementations - #83
Merged
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughExtracted duplicated PPO rollout logic into a new shared Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
packages/quantum-nematode/quantumnematode/brain/arch/_ppo_buffer.py (1)
21-22: Match the repo's docstring format for the shared buffer.This class is now the shared PPO buffer API, so its docstring should follow the project's NumPy-style convention.
As per coding guidelines, "Use NumPy-style docstrings".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/quantum-nematode/quantumnematode/brain/arch/_ppo_buffer.py` around lines 21 - 22, The RolloutBuffer class docstring currently is a single-line summary; replace it with a NumPy-style docstring for the shared PPO buffer API (class RolloutBuffer) that includes a short summary, Parameters (e.g., buffer_size, observation_space, action_space, device), Attributes (e.g., observations, actions, rewards, dones, ptr, size), and a brief Notes or Methods section listing key methods (add, sample, reset) so it matches the repo's NumPy-style convention and documents the public API.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/quantum-nematode/quantumnematode/brain/arch/_ppo_buffer.py`:
- Around line 103-119: The loop can produce minibatch_size==0 when batch_size <
num_minibatches (causing range() to error or produce wrong number of
minibatches); change the minibatch splitting to ensure a positive,
well-distributed size by computing minibatch_size = max(1, math.ceil(batch_size
/ num_minibatches)) (or equivalently compute start/end offsets using integer
division to produce exactly num_minibatches chunks) and then use that
minibatch_size for the for start in range(0, batch_size, minibatch_size) loop
while still indexing mb_indices = indices[start:end]; update imports if needed
and keep references to batch_size, num_minibatches, minibatch_size, indices and
the for loop so small buffers no longer yield zero-sized minibatches or extra
chunks.
---
Nitpick comments:
In `@packages/quantum-nematode/quantumnematode/brain/arch/_ppo_buffer.py`:
- Around line 21-22: The RolloutBuffer class docstring currently is a
single-line summary; replace it with a NumPy-style docstring for the shared PPO
buffer API (class RolloutBuffer) that includes a short summary, Parameters
(e.g., buffer_size, observation_space, action_space, device), Attributes (e.g.,
observations, actions, rewards, dones, ptr, size), and a brief Notes or Methods
section listing key methods (add, sample, reset) so it matches the repo's
NumPy-style convention and documents the public API.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: e94998e6-ee03-4532-90a6-00e62d546cef
📒 Files selected for processing (4)
packages/quantum-nematode/quantumnematode/brain/arch/_hybrid_common.pypackages/quantum-nematode/quantumnematode/brain/arch/_ppo_buffer.pypackages/quantum-nematode/quantumnematode/brain/arch/_reservoir_hybrid_base.pypackages/quantum-nematode/quantumnematode/brain/arch/mlpppo.py
Extract the PPO RolloutBuffer (duplicated across 3 files) into a single shared module at brain/arch/_ppo_buffer.py. The three near-identical copies (_reservoir_hybrid_base, mlpppo, _hybrid_common) now import from the shared module. Key changes: - Use reshape(-1) for value flattening (safe for single-step buffers) - Add len(advantages) > 1 guard for advantage normalization - Fix mlpppo's bare .squeeze() which could collapse batch dim QLIFLSTMRolloutBuffer and QSNNRolloutBuffer are left as-is since they store additional per-step data (LSTM states, spike caches) with fundamentally different interfaces. Closes #77 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chrisjz
force-pushed
the
refactor/consolidate-rollout-buffer
branch
from
March 21, 2026 05:17
23c84a3 to
b369f05
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RolloutBufferinto a shared module atbrain/arch/_ppo_buffer.py_reservoir_hybrid_base.py,mlpppo.py, and_hybrid_common.py.squeeze()which could collapse the batch dimension for single-step buffers (now uses.reshape(-1))len(advantages) > 1guard to prevent degenerate single-step advantage normalizationQLIFLSTMRolloutBufferandQSNNRolloutBufferleft as-is (different interfaces with extra per-step data)Closes #77
Test plan
🤖 Generated with Claude Code
Summary by CodeRabbit