Repository navigation
Conversation
GET /runs only filters on statuses that map directly to the latest TestProgress row (queued, running, canceled). pass, fail, error and incomplete are not stored in the database: they are computed per request from every sample's TestResult rows once a run completes, so they cannot be used as a query filter. The RunStatus parameter advertised all six values (with example: pass), so spec-conforming clients got a 400. Narrow the enum and document why. Run.status and RunSummary.status were missing "error", which the API already returns when a completed run reported no results or fewer samples than configured. Add it to both enums.
A full pass over all 79 operations found more places where the API
rejects a request the spec documents as valid:
- GET /runs/{run_id}/progress advertised status=queued, but TestStatus
has no queued value, so it always returned 400. A queued run has no
progress events to filter. Drop it from the enum, and build the 400
message from TestStatus instead of listing queued and error, which
were both rejected.
- GET /runs/{run_id}/samples advertised status=skipped and running,
which derive_sample_status never produces and the route rejects.
Narrow the enum to the four statuses the history filter already uses.
- GET /runs/{run_id}/samples/{sample_id}/logs documents cursor
pagination, but the route used the offset decorator, which rejects
any cursor. Switch it to validate_cursor_pagination.
Smaller mismatches in the same direction:
- validate_path_id rejects IDs above 2147483647, but no path ID in the
spec had a maximum. Add it to all of them, and to the sample_id filter
on /regression-tests. Apply validate_path_id to the int-converted IDs
that skipped it (output_id, variant_id, extra_id, token_id, blocked
user_id), where oversized values reached the database and raised.
- Document the minLength: 1 the schemas enforce on several body fields,
and the configured 10-500 password bounds.
- Document the 400 for an unknown ccx_version.
- Add actual_output to the artifacts type filter; the API returns and
filters on it.
Responses that broke the spec: restart returns action=restart, which
RunActionResult did not allow, and /auth/me returns seven scopes against
a maxItems of 6.
Add tests/api/test_openapi_contract.py, which calls every GET operation
with each documented enum, boolean and pagination bound value and fails
on a 400 or 5xx. It fails on all three bugs above against the previous
commit, and on the original /runs status bug against the one before.
kaihere14
requested review from
canihavesomecoffee and
thealphadollar
as code owners
November 6, 2026 12:33
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



In raising this pull request, I confirm the following (please check boxes):
My familiarity with the project is as follows (check one):
Replaces #1242, which was pointed at the outdated
GSoC-22-devbranch by mistake. This PR targetsmaster, rebased on the current tip, and contains only the 2 intended commits.Problem
Several parts of
openapi-ci-api.yamldocumented request values that the API rejects with a 400, so any client following the spec, including the spec's own examples, got errors. It started withGET /runs?status=, and a full pass over all 79 operations found the same mismatch elsewhere.Filters the spec advertised but the API rejected
GET /runs?status=pass,fail,incompletequeued,runningandcanceledcan be filtered on.GET /runs/{run_id}/progress?status=queuedTestStatushas noqueuedvalue.GET /runs/{run_id}/samples?status=skipped,runningderive_sample_statusnever produces either.GET /runs/{run_id}/samples/{sample_id}/logs?cursor=Cannot mix cursor and offset pagination.). The route used offset pagination.Smaller mismatches
validate_path_idrejects IDs above 2147483647, but no path ID in the spec had a maximum.minLength: 1in the schemas) without the spec saying so. The password fields enforce the configured 10–500 length range, which was also undocumented.ccx_versionreturns 400, which was not documented.GET /runs/{run_id}/artifacts?type=left outactual_output, although the API returns it and filters on it.Responses that violated the spec
Run.statusandRunSummary.statusdid not listerror, although the API returns it (_check_completed_run_status).POST /runs/{run_id}/restartreturnsaction: restart, butRunActionResult.actiononly allowedcancel.GET /auth/mereturns 7 scopes, butscopeshadmaxItems: 6.Why mostly the spec, not the code
pass,fail,errorandincompleteare not stored in the database. They are computed per request from every sample'sTestResultrows once a run completes, so they cannot be expressed as a SQL filter without deriving status for every run before pagination. Per-sample statuses work the same way. In each case the code already deliberately limits filters to values it can actually match, so the spec was the side that was wrong.The code changed in three places:
/runs/{run_id}/logsalready does./progress400 message is now built fromTestStatus. It used to listqueuedanderroras allowed, and both were rejected.validate_path_idnow covers the int-converted path IDs that skipped it, so oversized values return 400 instead of reaching the database.Changes
openapi-ci-api.yamlRunStatusenum narrowed to[queued, running, canceled], withexample: runningand a description explaining why outcome statuses cannot be filtered on.Run.statusandRunSummary.status: addederror./runs/{run_id}/progressstatus: droppedqueuedand added a description./runs/{run_id}/samplesstatus: narrowed to[pass, fail, missing_output, not_started], matching the/historyfilter.maximum: 2147483647added to all 17 integer path IDs and to thesample_idfilter on/regression-tests.minLength: 1added toPATCH /auth/menameandcurrent_password,PATCH /samples/{sample_id}tag items andversion,POST /tagsname, finalizeversion, and the password-resetmac. Both password fields now document the 10–500 length range.ccx_version: documented the 400 for unknown versions.typefilter: addedactual_output.RunActionResult.action: addedrestart.CurrentUser.scopes:maxItemsraised from 6 to 7.Routes
mod_api/routes/errors_logs.py:get_sample_logsusesvalidate_cursor_pagination.mod_api/routes/runs.py: the/progressstatus error lists the realTestStatusvalues.mod_api/routes/{regression_tests,samples,auth,system}.py:validate_path_idadded foroutput_id,variant_id,extra_id,token_idand the blocked-usersuser_id. An ID of0or one above 2147483647 now returns 400, matching every other ID route.Tests
tests/api/test_openapi_contract.py. It reads the spec, calls every GET operation with each documented enum value, boolean and pagination bound against the dataBaseTestCaseseeds, and fails on any 400 or 5xx. It runs with the existing nose2 suite, so it needs no new dependency.Testing
Run on Python 3.14 after rebasing onto
masterat02d0297:/progress,/samplesand sample-logs cases. Againstmasterit also fails on/runs?status=pass|fail|incomplete.openapi-spec-validator.Not changed
ProgressEvent.status,RunSample.status,SampleHistoryEntry.status). Clients cannot break on these, so I left them alone./runs/{run_id}/logsdefaults to 100 lines while the sharedLimitparameter documents a default of 50. Nothing is rejected, so this is left for a separate change.