Skip to content

fix(harness): recheck execution limits before worker launch - #552

Draft
Inference1 wants to merge 1 commit into
InferMatrixLab:mainfrom
Inference1:fix/harness-prelaunch-cancellation
Draft

Inference1 wants to merge 1 commit into
InferMatrixLab:mainfrom
Inference1:fix/harness-prelaunch-cancellation

Conversation

@Inference1

Copy link
Copy Markdown

Governing Issue and delivery slice

Refs #429

This proposed prerequisite strengthens the shared local CPU launch path used by the deadline and cancellation work in the accepted Harness scope.

Problem

The runner checks its deadline before creating the process domain and opening log files. Cancellation is checked by the caller before setup and by the supervisor after launch. If cancellation is observed or the deadline expires during log setup, the supervisor can still start a worker.

What

Recheck the cancellation handle and monotonic deadline immediately before Popen. Record the refused step and subsequent steps as not_started when no process was launched.

Why needed

Even a short-lived worker can write output or acquire resources. Checking the limits at the launch boundary keeps those effects from occurring when the cancellation or expiry is already observable.

Design and implementation

The existing supervisor returns the cancellation timestamp and source or its timeout result with an empty PID. The existing cleanup path closes the opened streams and any empty Windows Job. The execution service uses its unstarted-step representation and records cancellation independently from process cleanup. The fix adds no schema fields, clocks or dependencies.

Checks and dependencies

  • Both regression cases launch a worker on 9466bc8. With the patch they commit schema-valid, unstarted step records with no process lease or runtime binding.
  • Linux, Python 3.12. Harness execution, legacy local executor, task protocol, generation seal and Apps fixed-record views. 68 passed, 1 skipped. The skip is the Windows Job Object test. Full Ruff and git diff --check passed.
  • Upstream base 9466bc8 already fails test_project_explanation.py::test_recovered_attempt_does_not_trust_a_rewritable_declaration due to its import of the removed inferrsi.infra.artifacts module.

Remaining work and Issue closure

This draft requests maintainer scope confirmation and formal review of the local launcher change. Production evaluation-mode execution remains tracked by the governing Issue.

Recheck the caller cancellation handle and monotonic deadline after log setup. Commit an unstarted step and preserve the cancellation request when no worker was launched.

Agent-assisted-by: Codex (author)
Signed-off-by: Inference1 <3617237115@qq.com>

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NO PASS

首轮独立检视:9466bc8d910c9fef55d4a9e23ee1a82950280c30 → f331999b2dcc0d7f9d9eabed99483bcb3e935f12。这是 #429 的本地启动前置修正,不是 T37/T40 验收。

批准前必修:日志打开期间耗尽步骤限时会被记录成整次执行超期。 execution_service.py:622–640 将 min(整次截止时间, 步骤截止时间) 传入 supervisor。这个 PR 新增的 启动前检查 会正确拒绝启动,但无 PID 的 timeout 分支一律记 deadline_exhausted。用本提交的两个函数及受控时钟验证:整次限时 60 秒、步骤限时 5 秒、打开日志耗时 7 秒,没有启动进程,却报整次期限用尽。请区分步骤限时和整次期限,同时保留 not_started 和零进程启动;补一个非空 timeout_seconds 的回归。

实际验证:独立执行变更后的函数,用启动哨兵确认取消和整次期限在日志打开后均拒绝启动,且日志流关闭;另复现上述有限步骤限时的误报。未运行完整 Harness 测试、原生 Windows Job Object 或生产 T37/T40,未看 CI。

Copy link
Copy Markdown
Collaborator

Thanks for closing the launch-boundary gap. I reviewed 9466bc8d910c9fef55d4a9e23ee1a82950280c30 → f331999b2dcc0d7f9d9eabed99483bcb3e935f12.

Rechecking the cancellation handle and monotonic deadline after the log files are opened and before Popen is the right place. The early return stays inside the _supervise try/finally, so opened streams and an unattached Windows Job are still closed, and _run_steps now treats a pre-launch cancelled result like the existing no-PID timeout path so later steps stay not_started. The new parametrized test covers the cancellation and whole-run deadline cases.

I also checked the current-head review from @zengchuang-hw and agree it is still blocking:

  • _run_steps passes min(attempt deadline, step deadline) into _supervise. The new pre-launch timeout check in local_process.py is correct to refuse the start, but a no-PID timeout is always recorded as deadline_exhausted, and the supervisor error text always says the whole-run deadline was exhausted. If log setup overruns a finite timeout_seconds while the attempt deadline is still in the future, the step is misreported as a whole-run expiry.

Please distinguish a step-timeout refusal from a whole-run deadline refusal, keep the step not_started with zero process starts, and add a regression with a non-null timeout_seconds for that path.

I am not approving this head.

@Dong1017 Dong1017 added area:harness Harness module: profile, A/B loop, quality gates, evidence reports kind:task Bounded delivery task with a delegated slice labels Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:harness Harness module: profile, A/B loop, quality gates, evidence reports kind:task Bounded delivery task with a delegated slice

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants