Repository navigation
feat(harness): verify frozen preparation before registration - #561
zengchuang-hw wants to merge 4 commits into
Conversation
Adapt the authorized pool draft and freeze the null-timeout binding vector. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Verify candidate and worker declarations against fixed preparation bytes. Preserve ordinary command behavior and leave evaluation startup disabled until entry verification is implemented. Agent-assisted-by: Codex (author) Agent-assisted-by: Unknown PR Pool agent (draft contributor) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Dong1017
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 10ba7a5 and head d67b413 (full diff, diffComplete: true). prior-reviews.md has no formal reviews and no outstanding CHANGES_REQUESTED on this head, so this Approve is the first formal review. It is one vote and not a merge-rule substitute.
Verified by reading the diff and the callers at the head commit:
- Ordering.
execution_service.validateruns static shape checks first, including the new_evaluation_errorsand the V32-N2 "evaluation step needs a non-nullexpected_code_sha256" rule. Only after those pass does it callprepared_binding.verify(spec, artifact_reader), and only then theUnsupportedOperationchecks.executepassesartifact_readerthrough, andHarness.executeforwards its configured reader (api.pyL50-53). Every refusal therefore happens beforeRunStore.claim. Valid evaluation specs still end inUnsupportedOperation, so no evaluation attempt can start. - Bounded read.
prepared_binding.readrejects a malformed Ref, copies the Ref beforereader.open, closes the stream, and reads at mostsize_bytes + 1bytes. It re-checks size and SHA-256 itself.OSErrorand a missing reader becomeArtifactUnavailable,PublicErroris re-raised, and other programming errors propagate. - Error codes.
IdentityMismatch,UnsupportedRuleVersion,ArtifactUnavailable,ArtifactIntegrityErrorandUnsupportedSchemaare all inPUBLIC_ERROR_CODES(errors.py). The helpers it relies on (run_store._has_fields,_is_id,_describe,input_binding.RELATIVE_PATH,json_v1.MAX_SAFE_INTEGER) exist. - Binding logic.
- Identity is compared first, then declarations.
- Worker checks cover a single step, entry kind/name/revision, outputs,
cwd=".", frozen capabilities, whole-argument{input:}/{output:}expansion, and path overlap. - The overlap check sorts path tuples and compares neighbours, which is sufficient: if one path is a prefix of another, every path sorted between them shares that prefix.
- The
expected-sha256@1andcaptured-sha256@1rules and the frozen expected-candidate checks match the contract doc. - Unknown rule revisions are refused and not downgraded.
- Isolation.
prepared_bindingimports onlycontracts.json_v1andharness.*.check_installed.pyadds a cold-import assertion that it pulls in noapps,benchmarkorknowledgemodules. - Registry. The
apps-module-migrations.jsonconsumer entry is needed because the new test importsinferrsi.apps.handoff.bridge.test_active_consumers_use_canonical_paths_and_are_registeredis set-based, so the entry's position does not matter. - Tests and docs. The new tests assert an empty
attemptsandrecordstable and patchclaimand process start to fail on use. They cover null and wrong code, identity mismatch, worker drift, malformed projection, reader bounds, reader mutation and propagation of reader programming errors. The updated expectation intest_harness_execute.py(InvalidExecutionSpecinstead ofUnsupportedOperationfor the null-digest evaluation spec) follows from V32-N2. The contract doc, operations row and fixture README state the scope honestly: T37 and T40 are later slices and there is no production acceptance claim.
Not verified here: I did not run the test suite or CI. In particular I did not confirm that the D023 vector's bytes (2069 bytes, digest sha256:62ce11e6…6fe9) match bind_execution output. I also did not audit the golden t108-v4 fixtures or the Apps binder behaviour. Please confirm CI is green before merge.
Non-blocking nit: the new consumer entry in docs/apps-module-migrations.json sits between docs/apps-output.md and docs/architecture.md, while the rest of the list is alphabetical. It could move after src/... / before tests/check_installed.py for tidiness.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
|
当前 head @Dong1017 技术复审已收到。清单排序是非阻塞建议,登记和校验为集合语义;暂不为排序单独推新 head 重启 CI,后续实际修改该清单时再归并顺序。你作为 parent / 草稿材料贡献者的 Approve 不计独立审批。 @amy-why-3459 @david6666666 请对同一最终 head 独立检视,有问题我先修复;两份有效独立批准与检查通过后再 squash。#429 PR-2/3 和生产 CPU 验收仍保留,#371 不关闭。 |
|
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5). Nothing is needed from me beyond my existing Approve. That Approve is on the current head
This reply is from Dong-bot (authorized delegate). |
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 10ba7a5 and head d67b413.
Verified against the accepted #429 PR-1 scope in 7061372237 (pre-registration refusals with no RunStore attempt/record, a Harness-owned bind port, and the D023 null-timeout vector). This is PR-1 only; T37 entry verification and T40 remain later slices, and valid evaluation bindings still end in UnsupportedOperation.
- Ordering:
execution_service.validatecollects static shape errors first, including_evaluation_errorsand the V32-N2 rule that an evaluation step must have a non-nullexpected_code_sha256. Only a clean spec reachesprepared_binding.verify, and only then theUnsupportedOperationchecks.Harness.executeforwards the configuredartifact_reader. All of this runs beforeRunStore.claim. Ordinary command mode never reads a preparation. - Bounded read:
prepared_binding.readcopies the ArtifactRef beforereader.open, closes the stream, reads at mostsize_bytes + 1bytes, and re-checks size and SHA-256. A missing reader orOSErrorisArtifactUnavailable;PublicErroris re-raised; other host programming errors propagate. The mutation test shows the original Ref is unchanged. - Binding: identity (prepared id, protocol digest, operation, scope) is compared before declarations. Worker checks cover a single step, entry kind/name/revision, outputs,
cwd=".", frozen capabilities on the spec, whole-argument{input:}/{output:}expansion, and prefix-overlap of materialized paths.expected-sha256@1requires a declared input hash even without a frozen candidate; unknown rule revisions areUnsupportedRuleVersionand are not downgraded. The new error codes are inPUBLIC_ERROR_CODES. - Isolation:
prepared_bindingimports onlycontracts.json_v1andharness.*.check_installed.pyadds an isolated-interpreter probe that the public port does not import Apps, Benchmark, or Knowledge. - D023: I independently hashed
tests/fixtures/harness-prepared-binding/d023-null-timeout.jsonat this head: 2069 bytes, digestsha256:62ce11e6d458b5c581245d0988189e1bd834f2ed1e1880d15242faf70c5b6fe9, matching the accepted vector. The tests also assert agreement withbind_executionandrun_store.request_digest; I did not execute those binders myself. - Tests and docs: refusals assert empty
attempts/recordstables and patchclaim/ process start to fail if used. The null-digest evaluation case intest_harness_execute.pycorrectly becomesInvalidExecutionSpecunder V32-N2. The contract and operations row state the remaining PR-2/PR-3 work honestly.
lint-and-smoke, declarations, pr-references, installed-package (3.10/3.11), dashboard-browser and host-compatibility were green on this head when I checked; full-check (3.10/3.11) in https://github.com/InferMatrixLab/InferRSI/actions/runs/44900831192 was still running and should pass before merge.
This is an independent write+ review of the accepted PR-1 behavior. The parent/material-contributor Approve does not count toward the two required independent approvals.
|
@jiangkuaixue123 补请你独立检视当前 head 当前 head Ready CI 已全部通过,细项本地回归与 fresh wheel 结果见正文。Amy 已正式批准;原 David 请求仍保留。Dong 是本切片的 accountable parent / owner,他的批准不计独立票,作者也不自审。请按实际 diff 和接受范围给出 PASS / DO NOT PASS;符合时正式 Approve 的首个非空行请独立为 |
|
@hsliuustc0106 补请你独立检视本 PR。当前 head 的 Ready CI 全部通过(3.10 / 3.11 各 5980 passed、31 skipped、674 subtests),核心/边界回归与 fresh wheel 结果见正文;Amy 已同 head 正式批准。现有 David / jiangkuaixue123 的请求仍保留。请按实际差异给出结论,有意见会先处理。 本切片 parent 是 Dong1017,他的技术批准不计独立审批;作者不自审。需要另一位独立 write+ 评审正式批准后才可按既定顺序合入。 |
Intercept the supervisor used by Harness rather than the legacy execution wrapper in zero-start binding tests. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
66fef98
|
修正了一处验证缺口,当前 head 本 PR 当前 78 项核心用例通过,Ruff/whitespace 通过;生产及打包源码未改。既有表计数/固定记录断言保留。旧头测试和安装证据按原 revision 记账,新的 CI 及正式批准必须对应新 head。此前旧头 Approve 不计新头票。 @amy-why-3459 @david6666666 @jiangkuaixue123 @hsliuustc0106 请复审最终头,有意见先处理。@Dong1017 请保留 owner 技术检视及既定前置顺序;你的票仍不计独立批准。 |
Dong1017
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 10ba7a5 and head 66fef98. I read the full diff (diffComplete: true), not only the d67b413→66fef981 delta. This is a technical review by Dong-bot (authorized delegate). The #429 parent/material-contributor role makes it one vote that does not count as independent. Your merge rule still needs the two independent write+ approvals on this head and passing checks.
State: my earlier Approve was dismissed with the push, so this one renews it on the current head. prior-reviews.md shows no formal review on 66fef981 and no outstanding CHANGES_REQUESTED.
Verified from the diff and the head source
- Ordering.
validateruns the static shape checks first:_evaluation_errors, the V32-N2 "evaluation step needs non-nullexpected_code_sha256" rule, and the existing input/step checks.- It then calls
prepared_binding.verify(spec, artifact_reader). - It then raises the
UnsupportedOperationerrors. executeforwardsartifact_readerintovalidate(L991) and only callsstore.claimafterwards (L995). Every refusal therefore happens before registration. A valid evaluation spec still ends inUnsupportedOperation.
- Bounded read.
readvalidates the ArtifactRef shape, then callsreader.open(dict(ref))on a copy and closes the stream.- It reads at most
size_bytes + 1bytes in chunks and re-checks size and SHA-256 itself. OSErrorand a missing reader map toArtifactUnavailable.PublicErroris re-raised. Other host exceptions propagate. Tests cover each of these cases, including a reader that mutates the ref.
- Identity and declarations.
- Prepared id, protocol digest, operation and scope are compared first and give
IdentityMismatch. - Worker checks cover the single step, runtime role, entry kind/name/revision, outputs,
cwd="."and frozen capabilities. - Whole-argument
{input:}/{output:}expansion is checked. - The input/output path overlap check compares only adjacent sorted tuples. That is sufficient, because under lexicographic order a prefix path forces every path between the two to share that prefix. Equal paths are caught too.
- Unknown candidate identity rules or revisions give
UnsupportedRuleVersion.expected-sha256@1requires a declared input hash.
- Prepared id, protocol digest, operation and scope are compared first and give
- Test correction in this head.
- The zero-launch sentinel now patches
execution_service.local_process._superviseandstore.claim, which are the real call path. The_argvguard stays. - Each refusal test still asserts that the
attemptsandrecordstables are empty. The changed assertion intest_harness_execute.pyis consistent with V32-N2.
- The zero-launch sentinel now patches
- Packaging and vector.
- The
check_installed.pyprobe runs in an isolated interpreter (-I -B) and asserts that no Apps, Benchmark or Knowledge modules are imported. prepared_bindingis added toNEW_H_CLOSURE, and the new test consumer is registered inapps-module-migrations.json.- The D023 vector is asserted at 2069 bytes with digest
sha256:62ce11e6…6b5c581245d0988189e1bd834f2ed1e1880d15242faf70c5b6fe9. The test checks it against bothrequest_digestand the Appsbind_execution/verify_protocol. - The docs state that the code digest is an expected declaration, not an observed fact, and that T37, T40 and CPU acceptance are not covered.
- The
Not done: I did not run the tests, the wheel build or CI. The 78-test result and Ruff/whitespace results are the author's. Please wait for current-head checks to pass before the squash. The consumers ordering in apps-module-migrations.json remains a non-blocking style point.
Scope: #429 PR-2/PR-3 and the #371 closure are unchanged and stay open. Nothing here should be read as completing them.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed the increment d67b41312a2b61cfe81ae1dec1fcda715edd4d62 → 66fef981f6893038f3128595ffcdfd12b65bfba2 against base 10ba7a5500bcd95949219b138f9ada5033b922b0.
The only file change is in tests/test_harness_prepared_binding.py: the zero-start sentinel now patches execution_service.local_process._supervise instead of the legacy execute_command. That is the launch path execution_service._run_steps actually uses. Patching _supervise also covers the legacy execute_command wrapper, which calls _supervise in the same module. The _argv and store.claim guards remain. Production and packaging source are unchanged in this increment.
Ready lint-and-smoke, declarations, pr-references, installed-package (3.10/3.11), dashboard-browser and advisory host-compatibility were green on this head when I checked; full-check (3.10/3.11) in https://github.com/InferMatrixLab/InferRSI/actions/runs/44907170850 was still running and should pass before merge. This is an independent write+ review of the test-hook correction on the previously accepted PR-1 behavior; the parent/material-contributor Approve does not count toward the two required independent approvals. Accuracy/performance comparison evidence is not applicable (test-only change).
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 10ba7a5500bcd95949219b138f9ada5033b922b0 and head 66fef981f6893038f3128595ffcdfd12b65bfba2.
Identity: cursor[bot] independent third-party / technical verify only (collaborator permission none). This is not a write+ eligible Approve, dual-tip satisfaction, or merge-ready claim under docs/agent-collaboration.md. Merge still needs write+ independent Approves + integrator checks.
Clean-code / repo style: PASS
- Layering:
prepared_bindingimports onlycontracts+ harness (input_binding,run_store,errors) — no Apps/Benchmark/Knowledge imports (docs/architecture.mddependency direction). - Naming & APIs:
FixedPreparation,read/project/verify, PublicError codes matchdocs/contracts/harness-prepared-binding.mdand operations/migrations consumers. - Error handling: identity/declaration failures raise before
store.claim; bounded artifact reads (size+1) with independent SHA/size re-check; host errors mapped honestly. - No dead code / no forwarder shim; ordinary command path unchanged; valid evaluation still refused until later T37 slice (as documented).
- Tests/docs aligned (
test_harness_prepared_binding.py, D023 fixture,check_installedNEW_H_CLOSURE probe). Non-blocking:apps-module-migrations.jsonconsumers list order is slightly off-alpha.
Verified: tip Dong1017 + amy Approves present; Ready required checks green (declarations/full-check/installed-package). Read tip diff for prepared_binding.py, execution_service.validate ordering, and binding tests — not CI-green alone.
Refs #429
Refs #371
Harness now verifies an evaluation request against its fixed preparation before registering an attempt. Malformed or inconsistent bindings raise declared errors with no RunStore attempt or record. Valid evaluation bindings still raise UnsupportedOperation until the separate T37 implementation lands; ordinary command execution remains enabled.
Accepted scope: Dong1017's PR-1 delegation. This is PR-1 of the three accepted Harness delivery units, not completion of #429 or #371. Dong1017 remains the accountable Harness parent; the later PR-2/PR-3 implementation slices are also explicitly delegated to this Codex session in #429. This uses and strengthens the owner-supplied, previously unreviewed PR Pool draft; that agent's exact identity is unknown. zengchuang-hw / Codex is the acting implementer. Dong's parent/material-contributor review does not count as independent approval.
Base: main
10ba7a5500bcd95949219b138f9ada5033b922b0, including #556. Head:66fef981f6893038f3128595ffcdfd12b65bfba2. Initial incrementbfa4bc23was pushed before completing the slice, as requested.Delivered behavior
prepared_binding.read / project / verify: read-only stream capability, bounded reads, independent SHA/size checks and canonical v1 projection. Unknown host programming exceptions propagate. No Apps/Benchmark/Knowledge imports and no grading-rule interpretation.sha256:62ce11e6d458b5c581245d0988189e1bd834f2ed1e1880d15242faf70c5b6fe9. The canonical Apps binder and Harness digest agree; cross-checks callinferrsi.apps.handoff.bridge.verify_protocoldirectly. Historical/frozen fixtures are unchanged.Validation
66fef981: 78 binding tests passed, including the corrected zero-launch guard onlocal_process._supervise, the actual entry used by Harness. The earlier guard intercepted the legacy execute_command wrapper and did not directly intercept Harness launches. The no-registration/empty-table assertions remain in place.d67b4131: final binding + ordinary execution + recovery group 127 passed, 2 POSIX skips; earlier consumer/import group 205 passed, 43 subtests passed, 2 skips. Fresh sdist→wheel→new environment outside checkout, including the independent port probe, passed.Remaining work and review
PR-2 supplies registered T37 refusal records and actual entry verification. PR-3 supplies attempt-level deadline/budget and T40 evaluation replay, including the separately owned #552 launch prerequisite. This PR does not register a real evaluation worker, produce a production execution record, prove a service gain or complete shared CPU acceptance.
@Dong1017 please review the owner boundary and use this PR as the PR-2 base, as recorded. @amy-why-3459 @david6666666 @jiangkuaixue123 please independently review the accepted PR-1 behavior, exact D023 bytes, zero-registration failures and ordinary execution compatibility. Integration requires two independent eligible write+ formal approvals with
LGTMas the first nonempty line and passing checks; author, parent/material contributor and non-write bot approvals do not count.