Skip to content

ci: add on-demand Claude routing evals - #64

Draft
teallarson wants to merge 2 commits into
fix/claude-directory-compliancefrom
codex/claude-routing-evals-ci
Draft

teallarson wants to merge 2 commits into
fix/claude-directory-compliancefrom
codex/claude-routing-evals-ci

Conversation

@teallarson

Copy link
Copy Markdown
Contributor

Run Claude Code routing evals on a reviewed PR head with Sonnet 5.5 and synthetic tool responses. Choose the full MCP and CLI comparison or the GitHub CLI case. GitHub retains a summary and runner metadata.

After merge, restrict the routing-evals-claude environment to main, require an approver, and add an environment secret named ANTHROPIC_API_KEY. Dispatch Claude routing eval from main with the PR number and exact head SHA.

Full verification passed with 215 tests and two PowerShell skips. In a live Sonnet 5.5 smoke test, both builds answered the GitHub fixture through Arcade. A GitHub-hosted run still needs the environment key.

— Strider 🐦‍⬛

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant