Repository navigation
ci: deploy the documentation to the gh-pages branch - #4323
Conversation
9a3b2a3 to
74e4e3b
Compare
|
The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR4323 |
|
(The preview url is still the old one because that workflow uses the workflow file on main) |
There was a problem hiding this comment.
🟡 Changes recommended
The release version parsing can produce an empty target that risks overwriting doc/ on gh-pages, and the deploy action needs basic target-path validation to prevent path traversal deletion.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR migrates documentation deployment from S3/CloudFront to GitHub Pages served from the gh-pages branch, introducing a reusable composite action to publish (or delete) site paths and updating CI workflows to use it with appropriate GitHub token permissions.
Changes:
- Added a
deploy-to-gh-pagescomposite action that publishes a file/directory into a target path ongh-pages, with retry-on-push-race handling. - Replaced AWS-based docs deployments (S3 sync/cp + CloudFront invalidation) with GitHub Pages deployments for main, releases, and previews.
- Added workflows to clean up PR previews on close and to optionally squash
gh-pageshistory into a single commit.
File summaries
| File | Description |
|---|---|
| .github/workflows/reusable-docs.yml | Switch deploy job from AWS to gh-pages publishing; constrain build jobs to contents: read. |
| .github/workflows/docs-version.yml | Deploy docs/switcher.json to gh-pages instead of S3/CloudFront. |
| .github/workflows/docs-squash.yml | Add a manual workflow to squash gh-pages history (force-push). |
| .github/workflows/docs-preview.yml | Deploy PR preview artifacts to gh-pages and comment the new preview URL. |
| .github/workflows/docs-preview-cleanup.yml | Add PR-close cleanup to remove doc/pr/<number> from gh-pages. |
| .github/workflows/ci.yml | Update reusable docs workflow invocation to use contents: write (no AWS secrets). |
| .github/actions/deploy-to-gh-pages/action.yml | New composite action implementing the gh-pages publish/delete operation. |
Review details
- Files reviewed: 7/7 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The site moved off the S3 bucket and CloudFront distribution to GitHub Pages, which serves `awkward-array.org` from the `gh-pages` branch. Add a `deploy-to-gh-pages` composite action that publishes a file or a directory to a path on that branch through a worktree, and use it in place of the `aws s3 sync`/`cp` and cache invalidation steps. The layout of the site is unchanged, except that pull request previews now live at `/doc/pr/<number>/` rather than on a separate bucket. Previews were never removed from the bucket, but the branch is fetched by every deployment, so remove a preview once its pull request closes, and add a manually dispatched workflow that squashes the history of the branch when it grows too large. Assisted-by: claude-code:claude-opus-5
74e4e3b to
132cff7
Compare
ianna
left a comment
There was a problem hiding this comment.
@ariostas - thanks for taking care of it! Since I didn't attend the meeting where it was discussed I'd ask @TaiSakuma to double check if that is what we've agreed upon. Thanks!
|
@ariostas - thank you for the PR. |
|
@TaiSakuma yeah, let's keep talking about it to see if there's a better alternative. There would only be a new doc set per new minor version, and I'm guessing that minor version bumps won't be as frequent. The more likely issue is with PRs, but hopefully the PR removal workflow keeps that under control. I'll merge now so that the docs stay up to date and we can preview PRs, but let's definitely keep discussing this. |
* ci: deploy the documentation to a Cloudflare R2 bucket The site moves off GitHub Pages to an R2 bucket served through Cloudflare, reached over the S3 interface, so revert the `gh-pages` deployment added in #4323. Add a `purge-cloudflare-cache` composite action in place of the CloudFront invalidation, and drop the `deploy-to-gh-pages` action along with the workflow that squashed the branch history, which only existed because every deployment committed a full copy of the rendered site. Pull request previews return to their own bucket at `/PR<number>`. They are replaced in place on every push, so upload them with a `Cache-Control` that makes the edge revalidate rather than purging the whole zone for each one; Cloudflare only offers purge-by-prefix on Enterprise plans. Previews are still removed once their pull request closes, since nothing else would ever overwrite them. R2 needs an access key rather than OIDC, so the deployment jobs take `R2_ACCOUNT_ID`, `R2_ACCESS_KEY_ID` and `R2_SECRET_ACCESS_KEY`, and the purge takes `CLOUDFLARE_ZONE_ID` and `CLOUDFLARE_API_TOKEN`. Assisted-by: claude-code:claude-opus-5 * fix(ci): link the deployed documentation with a trailing slash An R2 custom domain serves objects by exact key and has no index document, unlike the S3 website endpoint it replaces, so `/doc/main` and `/PR<number>` are both a 404. Only the directory itself can be rewritten to the `index.html` it holds, and that rewrite keys off the trailing slash. Assisted-by: claude-code:claude-opus-5 * fix(ci): name the R2 buckets that hold the documentation A bucket name is no longer the domain it is served on, since R2 bucket names cannot contain a dot. Assisted-by: claude-code:claude-opus-5 * fix(ci): remove stale pages when syncing the documentation The documentation is generated with one page per callable, so a renamed or deleted function used to leave an orphan behind that the site kept serving and that counted against the storage of the bucket. Sync with `--delete`, which also restores what the `gh-pages` deployment did by rewriting its target. A preview now ends in a slash as well, so that the prefix it deletes under cannot reach the previews of other pull requests. Retry the cache purge, since it is idempotent and a transient failure would otherwise fail a deployment whose upload had already succeeded, and report the reason the API gives for a refused purge. Assisted-by: claude-code:claude-opus-5 --------- Co-authored-by: Ianna Osborne <ianna.osborne@cern.ch>
🤖 AI text below 🤖
The documentation site has moved off the S3 bucket and CloudFront distribution:
awkward-array.orgis now served by GitHub Pages from thegh-pagesbranch, which already holds a copy of the previous bucket contents. This replaces the deployment steps in CI accordingly.What changed
A new
deploy-to-gh-pagescomposite action publishes a file or a directory to a path ongh-pages. It fetches the branch shallowly, adds it as a detached worktree, replaces the target path, and commits and pushes asgithub-actions[bot]. An emptysourcedeletes the target instead, and because several workflows write to the branch, a rejected push is retried up to three times, re-applying the change on the new tip each time.The
aws s3 sync/aws s3 cpandaws cloudfront create-invalidationsteps are replaced by that action. The layout of the site is unchanged:mains3://awkward-array.org/doc/main/doc/maindoc/$version/,doc/stable/,doc/switcher.jsondocs/switcher.jsonchangeddoc/switcher.jsonpreview.awkward-array.org/PR<number>doc/pr/<number>/The only user-visible change is the preview URL, which is now
https://awkward-array.org/doc/pr/<number>/.Two workflows are added:
docs-preview-cleanup.ymlremovesdoc/pr/<number>when a pull request closes. Previews were never removed from the bucket, but every deployment now fetches this branch, and a published GitHub Pages site may be no larger than 1 GB.docs-squash.ymlis dispatched manually and squashesgh-pagesinto a single commit holding the same tree, so that superseded copies of the docs can be dropped from its history. It requires typinggh-pagesin a confirmation field, and no-ops if the branch is already a single commit.Deploy jobs now request
contents: writerather thanid-token: write, and theAWS_ACCOUNT_IDandAWS_DEPLOY_ROLEsecrets are no longer referenced anywhere and can be removed. Because a called workflow inherits the permissions of its caller, the jobs ofreusable-docs.ymlthat build code from the pull request are pinned tocontents: read, leavingcontents: writeto the deploy job alone, which only runs onpushandrelease. The PR number read from the preview artifact is now validated, since it is used as a path on a branch of this repository rather than a key in a separate bucket.Notes for review
.nojekyllhas already been pushed togh-pages. GitHub Pages builds that branch with Jekyll, which excludes the_staticand_imagesdirectories that Sphinx generates. The composite action keeps the file in place on every deployment.gh-pages, which need agit reset --hard origin/gh-pages.Previews now share an origin with the site
Previews used to be served from the separate
preview.awkward-array.orgbucket, and are now served fromawkward-array.org/doc/pr/<number>/. Their contents come from the artifact built by the pull request, so anyone able to run CI on a pull request can serve arbitrary HTML and JavaScript from the same origin as the documentation. That was equally true of the bucket, but the origin was a different one, so this is a deliberate trade-off rather than a side effect and is worth agreeing on before merging. Modifyingdocs/is enough to do it; modifying a workflow is not required.What a pull request cannot reach is unchanged. The token of a pull request from a fork is read-only and receives no secrets, whatever
permissionsthe pull request declares.docs-preview.ymlanddocs-preview-cleanup.ymlrun the copy of themselves on the default branch, and check out the default and base branch respectively, so the composite action they call is always the reviewed one. The target path isdoc/pr/<validated integer>and the artifact is copied into it, so a preview cannot touchdoc/main,doc/stable, the root of the site, or the repository.The exposure is therefore a convincing page on a trusted domain rather than access to anything. Two settings bound it, neither of them part of this pull request:
first_time_contributorspolicy, so a contributor who has already landed a pull request runs CI, and so publishes a preview, without approval.all_external_contributorswould require a maintainer to approve each run.default_workflow_permissionsfor the repository iswrite. Every workflow here declarespermissionsexplicitly, so nothing relies on it today, but a workflow added without one would get a write token by default.