Repository navigation
ci: benchmark ARM64 runners for bottleneck test suites - #8203
Closed
nicohrubec wants to merge 2 commits into
Closed
nicohrubec wants to merge 2 commits into
nicohrubec wants to merge 2 commits into
Conversation
Co-Authored-By: GPT-6 <codex@openai.com>
Contributor
Coverage Results 📊✅ 137221 passed | ⏭️ 7456 skipped | Total: 144677 | Pass Rate: 94.85% | Execution Time: 442m 22s 📊 Comparison with Base Branch
✨ Test counts unchanged from base. All tests are passing successfully. ✅ Patch coverage is 100.00% (no changed executable lines found; target 80%). Generated by Coverage Action |
Co-Authored-By: GPT-6 <codex@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Move AI, AI Workflow, and Web 1 (28 matrix jobs) to
ubuntu-22.04-armto measure CI performance. Other jobs keep their existing runners. The initial one-line README wording change triggered the x86 baseline before the runner-only commit.Issues
CI performance experiment; no linked issue.
Validation
Both first-attempt PR runs passed all 154 Test workflow jobs (152 matrix jobs, the required gate, and coverage reporting). Both used base commit
d565fed3932d83a04731cecc0a7910a916888de3, the same SDK code, and the same configured test matrix. Workflow regeneration passes--fail-on-changes; mypy reports no issues in 203 source files.Suite completion times run from the Test workflow start until the last job in that suite finishes, including runner setup and queue delays. The critical path changed from AI/Python 3.13 to Web 1/Python 3.6.
Average individual job runtime increased: AI 7m03s to 7m15s (+2.8%), AI Workflow 7m51s to 8m55s (+13.4%), and Web 1 5m40s to 6m07s (+7.9%). Total job runtime across the 28 changed jobs increased 7.4%. This single paired run shows a shorter completion tail, but does not establish a consistent ARM speedup; unchanged x86 jobs also varied between runs.
Baseline run · ARM run