Repository navigation
tests: extend the conformance completeness test and close known gaps - #953
Merged
Merged
Conversation
The completeness test only walked the service attributes, and only checked that the declared test file exists. Two gaps in that guarantee: - the top-level elements of the schema (services, networks, volumes, secrets, configs, models, jobs, include, name, version) could be added or removed without any test being declared for them, and so could the properties of the structured definitions behind them (network, volume, secret, config, model, include, job, schedule, healthcheck); - a declaration could point at a file that has nothing to do with the element, since only the existence of the file was checked. The test now covers the top-level elements and the properties of those definitions, each either declared with its file or listed in a known-gaps table with the reason (services, version, healthcheck.disable). It also requires the declared file to use the element as a YAML key, which catches a typo, a moved file or a copy-pasted entry. The check is shallow on purpose and says so; it stops at the listed definitions and does not descend into deeper structures such as deploy or build. Known gaps for service attributes become a map carrying the reason, as the other tables do. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
mem_limit, mem_reservation, mem_swappiness and memswap_limit had no conformance file of their own: they were only exercised together in a catch-all test. Each now has its file, with the spec section it locks. The byte-valued ones (mem_limit, mem_reservation, memswap_limit) check every notation of the specification's byte values, interpolation, override by a later file and rejection of invalid values; mem_limit and mem_reservation also check that they must agree with deploy.resources.limits.memory and deploy.resources.reservations.memory. memswap_limit locks the two special values of the specification, -1 for unlimited swap and 0 which is treated as unset. mem_swappiness locks the percentage notations, and states in its header what is deliberately not locked. The vocabulary shared by these files (the byte value notations, a one-service document builder, a load with the consistency checks enabled) is added to the test helpers. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
read_only and shm_size had no conformance file of their own. read_only locks the boolean and its string form, interpolation, override by a later file in both directions, and the rejection of values that are not booleans. shm_size locks every notation of the specification's byte values, interpolation, override and rejection of invalid values. pull_refresh_after stays the only service attribute without a conformance test: the specification does not describe it and compose-go passes the string through without interpreting it, so there is no behavior to lock yet. The known-gaps table now says so. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
TESTING.md still said the completeness test only walks the service attributes. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does, in one sentence
Makes the conformance completeness test guard the top-level elements of the schema and verify that a declared test file really uses the element, and closes six of the seven known gaps with new conformance files.
Context
loader/tests/completeness_test.gois what keeps the conformance matrix honest, but its guarantee was narrower than it looked. It only walked the service attributes, sonetworks,volumes,secrets,configs,models,jobs,include,name... and the properties of their structures (a network'sinternal, a secret'sdriver...) could change in the schema without anything asking where they are tested. And for the attributes it did walk, it only checked that the declared file exists: a typo, a moved file or a copy-pasted entry went unnoticed. Seven service attributes were listed as known gaps:mem_limit,mem_reservation,mem_swappiness,memswap_limit,pull_refresh_after,read_onlyandshm_size.What the PR brings
network,volume,secret,config,model,include,job,schedule,healthcheck). Each is either declared with its file or listed in a known-gaps table with the reason:services(no test of the map itself) andversion(obsolete, only exercised by a concurrency regression test), plushealthcheck.disable. The check deliberately stops there: it does not descend into deeper structures such asdeployorbuild, and says so in its header.mem_limit,mem_reservation,mem_swappiness,memswap_limit,read_onlyandshm_sizeget a file each, following the repository convention (spec link and quote, YAML-in/YAML-out, interpolation, override, invalid values; thedeployconsistency rule formem_limit/mem_reservation; the-1and0values ofmemswap_limit).pull_refresh_afterremains: it is absent from the specification and compose-go passes the string through, so there is no behavior to lock.Only tests change: no production code is modified.
🤖 Generated with Claude Code