Skip to content

docs(skill-evals): fix eval README links that 404 on GitHub - #2157

Merged
KatalKavya96 merged 3 commits into
apache:mainfrom
KatalKavya96:docs/fix-eval-readme-links
Nov 18, 2026
Merged

KatalKavya96 merged 3 commits into
apache:mainfrom
KatalKavya96:docs/fix-eval-readme-links

Conversation

@KatalKavya96

@KatalKavya96 KatalKavya96 commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Six eval READMEs linked to skill files in a way that 404s on GitHub:
    • two used ../../skills/…, which resolves under tools/skill-evals/skills/ and does not exist (good-first-issue-sweep, skill-reconciler);
    • four went through the skills/<name> symlink, which GitHub's web view does not follow (contributor-calibrate, contributor-candidate-screen, contributor-to-committer, release-verify-rc).
  • All six now link the real plugins/magpie-<family>/skills/<alias>/… path, as release-announce-draft/README.md already does. That path resolves on GitHub, locally, and in lychee.
  • A check of all 127 relative links in eval READMEs found no other broken ones.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • prek run --files on the changed files passes
  • Other: clicked all six links on this branch on GitHub and each opens its target file (before the fix, each returned 404).
  • Other: every new link target exists under plugins/, and no eval README links through skills/ any more.

RFC-AI-0004 compliance

Not applicable (docs-only link fix).

Linked issues

Closes #2156

Notes for reviewers (optional)

  • lychee didn't catch the two missing targets because .lychee.toml excludes all of tools/skill-evals/evals, which is meant for fixtures. It can't catch the symlink 404s either, because it reads the local filesystem, where symlinks resolve.
  • Separately, about 429 links in docs/ go through skills/ and also 404 on GitHub. That's a convention question (human docs vs agent-read files), so I'll raise it in its own issue rather than change it here.

Both linked to ../../skills/, which resolves under tools/skill-evals/. Use ../../../../skills/ like the other eval READMEs. A scan of all tracked Markdown found no other broken relative links or anchors.

Generated-by: Claude Code (Opus 5.5)
@KatalKavya96
KatalKavya96 requested a review from Kaap10 November 17, 2026 11:43
@github-actions github-actions Bot added family:docs Docs, MISSION.md, READMEs capability:triage Sweep + classify + propose disposition capability:review Deep per-item code review or contributor mentoring capability:reconciliation Compare tracker state against external inventory; surface drift family:mentoring mentoring skills family:utilities utilities skills labels Nov 17, 2026
@KatalKavya96
KatalKavya96 marked this pull request as draft November 17, 2026 12:26
@KatalKavya96
KatalKavya96 marked this pull request as draft November 17, 2026 12:26
@KatalKavya96
KatalKavya96 removed the request for review from Kaap10 November 17, 2026 12:26
@KatalKavya96 KatalKavya96 changed the title docs(skill-evals): fix broken skill links in two eval READMEs docs(skill-evals): fix eval README links that 404 on GitHub Nov 17, 2026
@github-actions github-actions Bot added capability:stats Read-only dashboards, metrics, governance evidence family:release-management release-* skills family:contributor-growth contributor-growth skills labels Nov 17, 2026
@KatalKavya96
KatalKavya96 marked this pull request as ready for review November 17, 2026 12:44

@Kaap10 Kaap10 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KatalKavya96 Thanks for catching and cleaning these up!

Fixing these six eval README links resolves #2156. Navigating through directory symlinks (or pointing at non-existent sibling tools/skill-evals/skills/ paths) leads to immediate 404s in GitHub's web interface.

correctness: plugin alias resolution (tools/skill-evals/evals/*/README.md)

No source rule covers this. It is a path resolution check across plugin family aliases.

Verified the relative paths and directory depth:

  • tools/skill-evals/evals/<suite>/ has directory depth 4, so ../../../../ correctly returns to the repo root before entering plugins/.
  • Crucially, this accurately accounts for shortened plugin folder names where the directory under plugins/magpie-<family>/skills/ differs from the framework symlink alias (e.g. calibrate instead of contributor-calibrate, candidate-screen instead of contributor-candidate-screen, verify-rc instead of release-verify-rc).
  • Tested all six target files locally; all exist and resolve cleanly.

Smaller observations

  • tooling:ci — Lychee / CI blindspot: Good explanation in the PR description regarding why lychee did not catch these: .lychee.toml excludes tools/skill-evals/evals to protect test fixtures, and local filesystem link checkers traverse symlinks transparently whereas GitHub's web tree viewer does not.
  • scope — Docs-wide symlink issue: Deferring the ~400 symlink occurrences in docs/ to a separate issue/convention discussion is the right call to keep this PR minimal and immediately mergeable.
  • Grepped across all eval READMEs: there are no remaining broken relative links in tools/skill-evals/evals/*/README.md.

LGTM!


This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
Contributing guide.

@KatalKavya96
KatalKavya96 merged commit f35629d into apache:main Nov 18, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:reconciliation Compare tracker state against external inventory; surface drift capability:review Deep per-item code review or contributor mentoring capability:stats Read-only dashboards, metrics, governance evidence capability:triage Sweep + classify + propose disposition family:contributor-growth contributor-growth skills family:docs Docs, MISSION.md, READMEs family:mentoring mentoring skills family:release-management release-* skills family:utilities utilities skills

Projects

None yet

Development

Successfully merging this pull request may close these issues.

skill-evals: six eval READMEs have skill links that 404 on GitHub

2 participants