Skip to content

ci: add stdlib matrix testing (libc++, stdc++, dynamic-stdc++) - #726

Merged
helly25 merged 9 commits into
bazel-contrib:masterfrom
helly25:feat/libcpp-testing
May 10, 2026
Merged

helly25 merged 9 commits into
bazel-contrib:masterfrom
helly25:feat/libcpp-testing

Conversation

@helly25

@helly25 helly25 commented May 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Extend external_test in .github/workflows/tests.yml to cover all supported stdlib implementations by adding a stdlib matrix dimension and corresponding test toolchains.

What changed

CI matrix (.github/workflows/tests.yml)

The external_test job now includes a stdlib dimension with four values:

stdlib behaviour
default No override — auto-config picks builtin-libc++
libc++ System/sysroot-hosted libc++
stdc++ Static libstdc++
dynamic-stdc++ Dynamic libstdc++

Exclusions to keep the matrix manageable:

  • macOS + non-default stdlib — libstdc++ is not available on macOS by default (would need homebrew install)
  • bzlmod=false + non-default stdlib — only test the default builtin-libc++ in legacy mode
  • bazel latest + bzlmod=false — not needed

Bazel versions: restricted to 8.* and latest (the old "rules_rust bzlmod support is experimental" comment from 2023 is stale).

Test toolchains (tests/MODULE.bazel + tests/WORKSPACE)

Three new toolchain configurations added to both bzlmod and WORKSPACE modes:

  • llvm_toolchain_libcxx — stdlib = "libc++"
  • llvm_toolchain_stdcpp — stdlib = "stdc++"
  • llvm_toolchain_dynamic_stdcpp — stdlib = "dynamic-stdc++"

Test script (tests/scripts/run_external_tests.sh)

  • Platform detection — uses uname to auto-detect the correct toolchain label suffix (aarch64-darwin, x86_64-linux, etc.) instead of hardcoding x86_64-linux
  • TEST_STDLIB env var — selects which extra toolchain to pass via --extra_toolchains
  • targets array — all test targets collected into a single array, passed as one -- "${targets[@]}" argument to bazel test
  • Conditional rust/go tests — rules_rust and rules_go cgo prep+targets only run on bazel 9+ with bzlmod enabled (rules_rust bzlmod support is unreliable on older Bazel versions)

WORKSPACE boringssl fix

Added missing patch_args and patches attributes to the boringssl http_archive in tests/WORKSPACE to match the bzlmod version in tests/MODULE.bazel.

Extend external_test to cover all supported stdlib implementations:
- default: auto-config (builtin-libc++)
- libc++: system/sysroot-hosted libc++
- stdc++: static libstdc++
- dynamic-stdc++: dynamic libstdc++

Restrict bazel to 8.* and latest; exclude bzlmod=false for latest
and for non-default stdlibs to keep the matrix manageable.

Add llvm_toolchain_libcxx, llvm_toolchain_stdcpp, and
llvm_toolchain_dynamic_stdcpp to both MODULE.bazel and WORKSPACE.
Update run_external_tests.sh to select toolchain via TEST_STDLIB env var.
helly25 added 6 commits May 9, 2026 15:39
â€Ķtch to WORKSPACE

- Only run rules_rust and rules_go cgo prep/tests on bazel 9+
  with bzlmod enabled, since older versions don't support them reliably.
- Add missing boringssl-20260413.0.patch to tests/WORKSPACE to match
  the bzlmod version in tests/MODULE.bazel.
â€Ķts.sh

Use uname to auto-detect os/arch and build the correct cc-toolchain
label (e.g. aarch64-darwin on macOS ARM, x86_64-linux on Linux)
instead of hardcoding x86_64-linux everywhere.
@helly25

helly25 commented May 9, 2026

Copy link
Copy Markdown
Collaborator Author

@fmeum we need to change the required tests since the two external tests now have an additional parameter. It is not clear which ones we want to require.

@helly25

helly25 commented May 9, 2026 •

Copy link
Copy Markdown
Collaborator Author

@phlax take a look if the matrix for the external tests make sense to you please.

@fmeum fmeum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modifying required checks on GH is an unfortunate experience - I'll do my best

@helly25
helly25 merged commit e46e811 into bazel-contrib:master May 10, 2026
69 checks passed
@helly25
helly25 deleted the feat/libcpp-testing branch May 10, 2026 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants