[ParidelPooya]

Redesign of the state check (rule 8)

Two review findings hit the same check, so this section replaces the earlier design of the state check.

Problem

After a sign that the MicroVM may not run (an HTTP 502 with x-aws-proxy-error: MICROVM_CONNECT_FAILED, or a request without an answer), the job must learn whether the MicroVM has ended. Two findings showed cases where it does not:

1. A denied GetMicrovm kept retrying. The check caught AccessDeniedException as a transient failure and the job kept sending. A dead MicroVM then made about 55 denied calls and stayed on the roughly six-minute failure path. (Fixed in the previous round.)
2. A short retry window skips the check. With retryWindow: { seconds: 1 } (retry_window in Python), one delayed 502 from a dead endpoint (measured 3 to 8 s) uses up the window. The window exit runs before the check, so the attempt ends with MicrovmEndpointUnavailableError after zero GetMicrovm calls, and the default strategy repeats that.

Cause (shared)

The state check is an optional hook inside the delivery retry loop, not a step with its own rule:

• the loop decides first (window exit, then the window budget for the check), and calls the check only if the window still has time;
• the loop also decided what a check failure meant (keep retrying).

So the check's outcome depends on the retry window, and its failures were folded into the loop's own retry rule.

Options

Option	Result
A. Patch: skip the window exit when a check is due	Fixes finding 2 only; keeps the hook design that caused both findings.
B. Give the check its own budget inside the loop	Still two rules interleaved in one loop.
C. Simplify: after a sign, run the state check first, as its own step of the attempt, budgeted only by the invocation reserve. Its result decides before any window or backoff rule runs. Chosen.	One order, one rule. Removes the window budget for the check and the hook-specific branches.

Chosen order

On every sign:

1. Run one GetMicrovm call, at most once per 2 seconds, limited only by the invocation reserve (10 s before the Lambda times out). The retry window does not limit it.
2. TERMINATING, TERMINATED or not found: fail now with MicrovmNotRunningError.
3. Denied: fail now (see the error model below).
4. Any other state: continue with the normal window and backoff rules.
5. Any other GetMicrovm error (throttling, 5xx, timeout, connection): one warning per step attempt, then continue with the normal window and backoff rules.
6. If the invocation reserve is already reached, no call starts, and the attempt ends as it does today (retryable MicrovmEndpointUnavailableError).

The order does not depend on the retry window, so the terminal state is always checked after a sign while invocation time remains.

Error model

Both internal denial errors (denied GetMicrovm, denied ResumeMicrovm) have the same retry behavior (not retried by the default strategy) and the same public error (MicrovmDeliveryError). So they become one internal type, MicrovmPermissionDeniedError. Its message names the denied action and the permission, for example ... GetMicrovm was denied. The function's execution role needs lambda:GetMicrovm. GetMicrovm request ID: <id>. MicrovmGetDeniedError and MicrovmResumeDeniedError are removed. A custom retry strategy that matches by name uses the one name.

Tests

• A delayed 502 (3 s) with a 1-second retry window ends in MicrovmNotRunningError after one GetMicrovm call (fails on the current head).
• The same with no answer instead of a 502.
• The invocation reserve still applies: a reserve already reached makes no GetMicrovm call.
• Denied GetMicrovm and denied ResumeMicrovm both raise MicrovmPermissionDeniedError with their own permission in the message.
• Each branch fails without it (mutations: check after the window exit; window-limited budget; denied treated as transient; one class per action).

What it removes

The window budget for the check, the check's branch inside the window logic, and one internal error class.

Rule 8 commits

The fifth and sixth commits implement the redesign above. They apply as a fast-forward onto the fourth commit.

Fifth commit: fix(extras): check MicroVM state before the window.

• In send_job, the state check (suspend_check) now runs right after a sign, before the window exit and before the backoff. It still runs at most once per 2 seconds.
• Only the invocation reserve limits the check. It uses the existing _call_budget helper with the reserve as the deadline. With less than 1 second before the reserve, no call starts, and the tier raises MicrovmEndpointUnavailableError, as at the reserve today. The check receives the time until the reserve, or infinity when the compute reports no deadline.
• Removed: the window budget of the check (left = deadline() - now) and its window exit branch (if left <= 0: raise unavailable()).
• The session recheck path (resume mode) does not change.
• Docs: SendJobOptions.suspend_check, send_job, deliver, and README "Suspending after a job" item 2 describe the order.

Sixth commit: refactor(extras): use one MicroVM denial error.

• MicrovmGetDeniedError and MicrovmResumeDeniedError are replaced by one internal MicrovmPermissionDeniedError. The default retry strategy does not retry it, and the delivery raises MicrovmDeliveryError, as before.
• One helper, _denied, builds the message for both calls. The message text does not change: MicroVM <id> gave a sign that it does not run. GetMicrovm was denied, so the operation cannot check its state. The function's execution role needs lambda:GetMicrovm. GetMicrovm request ID: <id>. and MicroVM <id> is SUSPENDED. ResumeMicrovm was denied, so the operation cannot resume it. The function's execution role needs lambda:ResumeMicrovm. ResumeMicrovm request ID: <id>.
• default_microvm_retry_strategy lists neither old name, so it does not change. The README names neither class. The package has no CHANGELOG.

Tests of the rule 8 commits.

Test	Commit	What it pins
operation_test.py::test_checks_the_state_after_a_sign_that_uses_up_the_retry_window[connect-failed]	5	A 502 with MICROVM_CONNECT_FAILED that takes 1.5 s, with a 1-second retry_window: MicrovmNotRunningError after 1 request and 1 GetMicrovm call.
operation_test.py::test_checks_the_state_after_a_sign_that_uses_up_the_retry_window[no-answer]	5	The same with no answer instead of the 502.
request_test.py::test_starts_no_suspend_check_within_1_second_of_the_reserve	5	Less than 1 second before the reserve: no check, MicrovmEndpointUnavailableError after 1 attempt.
request_test.py::test_runs_the_suspend_check_when_the_sign_used_up_the_window	5	A sign after the window ended still runs the check, with the reserve as its only limit.
operation_test.py::test_a_custom_strategy_sees_one_error_type_for_a_denial[get, resume]	6	A custom retry strategy receives MicrovmPermissionDeniedError for both denials, with its own permission in the message.

The tests use a 1.5-second delay instead of the measured 3 to 8 seconds. Any delay longer than the 1-second window shows the bug, and the shorter delay keeps the suite fast.

Edited tests (rule 5.4). The check now runs before the delay, so it runs at the failure time instead of after the delay:

• request_test.py::test_checks_for_a_suspend_after_a_connect_failure_or_no_answer: the check budgets are the time until the reserve ([90, 86.25]) instead of the rest of the window ([59.75, 52.25]). The test now sets a reserve.
• request_test.py::test_checks_for_a_suspend_at_most_once_per_2_seconds: the check times are [0, 3.75] instead of [0.25, 3.75, 7.75].
• operation_test.py::test_keeps_sending_the_job_with_one_warning_when_get_microvm_fails and session_test.py::test_keeps_sending_after_a_failed_state_check_with_one_warning: 5 failed requests instead of 4, so that two checks still run (at 0 and 3.75 s). The job succeeds on request 6.
• request_test.py::test_ends_the_tier_when_the_suspend_check_has_no_time_left is removed: it asserted that the window skips the check, which is finding 2. test_starts_no_suspend_check_within_1_second_of_the_reserve replaces it.
• Commit 6 renames the class in 2 imports and 6 assertions of lifecycle_test.py, operation_test.py, and session_test.py. No expected message changes.

The window test fails on the fourth commit (bee5570a). Both cases end with MicrovmDeliveryError, after 1 request and no GetMicrovm call:

FAILED operation_test.py::test_checks_the_state_after_a_sign_that_uses_up_the_retry_window[connect-failed]
  {'error': 'MicrovmDeliveryError'} != {'error': 'MicrovmNotRunningError'}
  'MicroVM "build": the job could not be delivered: The MicroVM did not accept the job at /job after 1 attempts in 1504 ms. Last failure: HTTP 502.'
FAILED operation_test.py::test_checks_the_state_after_a_sign_that_uses_up_the_retry_window[no-answer]
  {'error': 'MicrovmDeliveryError'} != {'error': 'MicrovmNotRunningError'}
  'MicroVM "build": the job could not be delivered: The MicroVM did not accept the job at /job after 1 attempts in 1501 ms. Last failure: URLError: <urlopen error connection reset>.'
2 failed

Mutation check of the fifth commit, against the 22 state check tests of request_test.py and operation_test.py:

Mutation	Failing tests
The check runs after the window exit (the old order)	3 (both window cases in operation_test.py, test_runs_the_suspend_check_when_the_sign_used_up_the_window)
The check budget is limited by the window (deadline() instead of the reserve)	4 (the 3 above, and test_checks_for_a_suspend_after_a_connect_failure_or_no_answer)
The check budget ignores the reserve	2 (test_starts_no_suspend_check_within_1_second_of_the_reserve, test_checks_for_a_suspend_after_a_connect_failure_or_no_answer)

Mutation check of the sixth commit, against the 9 denial tests of lifecycle_test.py, operation_test.py, and session_test.py:

Mutation	Failing tests
MicrovmPermissionDeniedError retryable (a subclass of MicrovmEndpointUnavailableError)	4 (microvm() 2, session 2)
Wrong permission text (lambda:GetMicrovm for both calls)	5 (every ResumeMicrovm denial test)
One class per action (a ResumeMicrovm subclass)	1 (test_a_custom_strategy_sees_one_error_type_for_a_denial[resume])
A denied GetMicrovm treated as transient (warn and keep sending)	4 (lifecycle 1, microvm() 2, session 1)

Extras package after the sixth commit: 489 passed, coverage 100% (1093 statements, 228 branches). mypy: no issues in 24 files. hatch fmt --check: no new findings (the one remaining finding, in tests/microvm/aws_errors_test.py, is on the fourth commit too). lintcommit.py --range origin/main..HEAD: all commits pass.

Decisions not covered by the design note.

• (Changed in commit d73fe6c9.) The resume and the wait for RUNNING after the check get a new retry_window, and end earlier at the reserve when one exists. So without a Lambda context the wait ends after about one window with the retryable MicrovmEndpointUnavailableError, as in the JavaScript operation. The first GetMicrovm call stays limited only by the reserve.
• The reserve guard reuses _call_budget, the guard of every other call in the tier. So the check starts only with at least 1 second before the reserve, not at any time before it. The error is the tier's own MicrovmEndpointUnavailableError message, as at the reserve today.
• The session recheck path keeps its order. It runs before every job, and the findings did not involve it.
• The window tests use a fake opener with a 1.5-second delay instead of the local endpoint, because the local endpoint cannot delay an answer.

---

Issue #, if available

None. Port of aws-durable-execution-sdk-js#1178.

Description of changes

microvm() over HTTP now fails at once with MicrovmNotRunningError when its MicroVM has ended before the job arrives. Before, the request step kept sending the job to the dead endpoint until both retry tiers ended, and the job failed with a plain MicrovmDeliveryError about 6 minutes later.

The change removes the suspend_check flag of the internal deliver(). So every job over HTTP without the resume mode runs the state check that a session job already runs. Session behavior does not change: a session set the flag for every job.

The second commit ports the follow-up of the JS PR. The ended-state message now removes one trailing period of stateReason, so it reads TERMINATED: Success. instead of TERMINATED: Success... Sessions share _raise_if_ended, so the session message changes the same way. The docs now say that the check calls GetMicrovm after each sign, at most once per 2 seconds, and logs one warning per step attempt, so up to 5 with the default retry strategy. Two new tests cover the resume path of microvm().

New permissions for microvm() with HTTP delivery: lambda:GetMicrovm and lambda:ResumeMicrovm. lambda:GetMicrovm is required.

The third commit answers the review: a denied GetMicrovm in the state check now fails the request step at once. It raises the new internal MicrovmGetDeniedError, in the style of MicrovmResumeDeniedError. The default retry strategy does not retry it, so the job fails with MicrovmDeliveryError, and the message reads MicroVM <id> gave a sign that it does not run. GetMicrovm was denied, so the operation cannot check its state. The function's execution role needs lambda:GetMicrovm. GetMicrovm request ID: <id>. A throttled, 5xx, timed-out, or failed-to-connect GetMicrovm still logs one warning per step attempt and keeps sending the job. Sessions share this code, so a session job without lambda:GetMicrovm also fails at once when its request gets the dead or suspended sign. The commit removes the denied note on the final MicrovmDeliveryError message, the unavailable_note option of send_job that carried it (it has no other caller), and the denied attribute of MicrovmStateCheckError.

Effect for a role without lambda:GetMicrovm: a job whose request gets a 502 with MICROVM_CONNECT_FAILED, or no answer, fails at once with MicrovmDeliveryError, and the message names the permission. A job whose MicroVM answers makes no GetMicrovm call and is not affected. A role without lambda:GetMicrovm now fails a job whose request gets no answer, even when the MicroVM runs, for example a worker that takes more than 10 seconds to listen. Before, such a job logged a warning and succeeded.

Docs: the MicrovmNotRunningError, MicrovmStateCheckError, resume_if_suspended, SendJobOptions.suspend_check, send_job, SendJobOptions.window_started_at, deliver, _STATE_CHECK_WARNING and microvm() docstrings, and the README sections Errors, Permissions, Suspending an idle MicroVM, Suspending after a job and Session permissions. The errors.py note "Whether that 502 carries MICROVM_CONNECT_FAILED is not verified" now states the measured answer. The package has no CHANGELOG.

The fourth commit answers the second review. The denied GetMicrovm message is now the same text as in the JS SDK. The README states the effect for a healthy job without lambda:GetMicrovm (above), and says "that answer, or no answer" in Permissions and Session permissions. The MicrovmDeliveryError docs of the README Errors table, MicrovmSession.invoke and microvm() name the denied GetMicrovm and ResumeMicrovm. One new test row pins the narrow match: a GetMicrovm UnrecognizedClientException stays on warn-and-retry.

Design

See Rules for every change.

Design note (rule 3).

Problem. A microvm() job over HTTP whose MicroVM has ended keeps sending the job to the same dead endpoint until both retry tiers end. The job then fails with a plain MicrovmDeliveryError, minutes later, and each attempt bills Lambda compute.

Cause.

1. The launch uses clientToken = sha256(callback_id). So every retry reaches the same MicroVM.
2. The request step retries connection errors, 429 and 5xx as a MicroVM that is still starting.
3. The endpoint of a terminated MicroVM answers HTTP 502 with x-aws-proxy-error: MICROVM_CONNECT_FAILED and an empty body. CreateMicrovmAuthToken still issues a token for it. Both were measured live in us-east-1 (see the JS PR and the table below).
4. A session job already calls GetMicrovm after each such answer, at most once per 2 seconds (resume_if_suspended). deliver() turned the check on only through its suspend_check flag, and only a session set it. So microvm() made no state check.

Options.

1. Simplification: run the existing check for every job over HTTP, and remove the suspend_check flag. The trigger stays the same: a 502 with MICROVM_CONNECT_FAILED, or no answer.
2. Add a second check for microvm() that only raises for an ended MicroVM, and never resumes. It adds a code path next to an existing one that does the same job.
3. Check after any 5xx. A plain 500 or 503 comes from a worker that answers, so the MicroVM runs. The extra GetMicrovm call would add nothing.

Chosen. Option 1, the same as the JS SDK. It fixes the bug by removing a condition. The measured dead answer is exactly the answer that the existing check reacts to.

Removes (first commit). The suspend_check parameter of deliver() and its condition (elif suspend_check: becomes else:), and the suspend_check=True argument in session.py.

• Simpler options considered (rule 2): see Options. This change removes logic.
• Root cause (rule 5): deliver() ran the state check only for sessions, so microvm() treated the dead endpoint's 502 as a booting MicroVM.
• Net change in source lines and in public API members, for the first commit: code: 2 lines removed, 1 line changed (elif suspend_check: to else:); the rest is docstrings. Public API: no change (deliver() is internal).

Before and after, live in us-east-1

Scenario: microvm() with request=MicrovmRequestConfig(path="/job"). The boto3 client terminates the MicroVM in an after-call handler of RunMicrovm, and waits for TERMINATED (about 2 s). The job then goes to the dead endpoint. A urllib handler on the opener logs each request and its answer. The function's role has lambda:GetMicrovm.

	Before (main)	After (this branch)
Dead endpoint's answer	502, x-aws-proxy-error: MICROVM_CONNECT_FAILED, every request	same (1 request)
Error class	MicrovmDeliveryError	MicrovmNotRunningError
Message	MicroVM "job": the job could not be delivered: The MicroVM did not accept the job at /job after 11 attempts in 61130 ms. Last failure: HTTP 502.	MicroVM "job": the job could not be delivered: MicroVM <id> is TERMINATED: Success. GetMicrovm request ID: <id>. (the live run at the first commit read Success..; the second commit removes one trailing period of stateReason)
Time to fail (execution start to end)	341.6 s	9.0 s (about 2 s of it is the scenario's own wait for TERMINATED)
Requests to the endpoint	55 (5 step attempts x 11)	1

This change does not cover run hook delivery. A job in the run hook has no request step. So a MicroVM that ends before it runs the job still fails it with MicrovmTimeoutError, at its heartbeat_timeout or timeout. The repo has no Python cloud harness for this scenario, so the live run used a throwaway script outside the repo.

Tests

New tests in operation_test.py, 14 cases for microvm() over HTTP:

• 502 with MICROVM_CONNECT_FAILED or no answer, GetMicrovm reports TERMINATED or TERMINATING: MicrovmNotRunningError at once, with the stateReason (Success., one period) and the request ID, 1 request, 1 GetMicrovm call, and the MicroVM is terminated (4 cases).
• GetMicrovm answers ResourceNotFoundException: MicrovmNotRunningError with the request ID (1).
• GetMicrovm reports RUNNING or PENDING: the job is sent again and succeeds, with no resume (2).
• (third commit) GetMicrovm is denied: MicrovmDeliveryError at once, naming lambda:GetMicrovm and the request ID, after 1 request and 1 GetMicrovm call; the step error type is MicrovmGetDeniedError (1). It replaces the old denied test.
• (third commit) GetMicrovm is throttled, answers 5xx, or fails to connect: the job keeps its retries and succeeds, with one warning for two failed checks (3).
• A plain 500 or 503: no GetMicrovm call (2).
• A 400 or 404: MicrovmDeliveryError at once, with no GetMicrovm call (2).
• GetMicrovm reports SUSPENDED: the MicroVM is resumed (get:SUSPENDED, resume, get:RUNNING, terminate), and the job succeeds after 2 requests (1).
• GetMicrovm reports SUSPENDED and ResumeMicrovm is denied: MicrovmDeliveryError at once, naming lambda:ResumeMicrovm, after 1 request (1).

On main, 8 of the first 12 fail (all GetMicrovm cases). The 4 others pass (plain 5xx and 4xx). With the old behavior restored by mutation (no state check for microvm()), 10 of the 14 fail: the 8, plus the 2 resume tests. With the fix, all pass.

Third commit, in other test modules:

• session_test.py: test_fails_at_once_when_the_state_check_is_denied (new; one request, no retry, names lambda:GetMicrovm, step error type MicrovmGetDeniedError) and test_keeps_sending_after_a_failed_state_check_with_one_warning (new; throttled and 5xx, 2 cases) replace test_keeps_sending_after_a_denied_state_check_and_names_it_at_the_end.
• lifecycle_test.py: test_check_fails_at_once_when_get_microvm_is_denied and test_check_names_no_get_request_id_for_an_error_of_another_call (new). test_check_reports_a_failed_get_microvm_without_failing_the_job now runs for ThrottlingException and a 500 instead of AccessDeniedException and ThrottlingException, and no longer asserts denied. The fourth commit adds an UnrecognizedClientException (403) row to it. test_check_reports_a_get_microvm_connection_failure no longer asserts denied.
• request_test.py: test_adds_the_note_to_the_final_error and test_adds_no_note_when_the_note_is_none are removed with unavailable_note.

Extras package after the fourth commit: 484 passed, coverage 100% (statements and branches). mypy: no issues. hatch fmt --check: no new findings.

Mutation check (rule 4). The first 8 rows ran against the first 12 new tests, the last 3 against all 14:

Mutation	Failing tests
No state check for microvm() (the old behavior)	8
TERMINATING/TERMINATED does not raise	4
ResourceNotFoundException from GetMicrovm treated as a failed check	1
Every GetMicrovm state forced to TERMINATED (so RUNNING/PENDING are treated as ended)	4 (2 running cases, and the 2 TERMINATING messages)
A denied GetMicrovm fails the job	1
A warning on every failed check	1
Check after any 5xx	2
4xx retried instead of rejected	2
No state check for microvm() (the old behavior), all 14 tests	10
One trailing period of stateReason not removed	4
resume_if_suspended does not resume a suspended MicroVM	2 (both resume tests)

Mutation check of the third commit, against the 17 denied and failed-check tests of lifecycle_test.py, operation_test.py, session_test.py, and request_test.py:

Mutation	Failing tests
A denied GetMicrovm treated as a transient failure (warn and retry)	4 (lifecycle 2, microvm() 1, session 1)
MicrovmGetDeniedError retried by the default strategy	2 (microvm() 1, session 1)
Every GetMicrovm ClientError treated as denied	6 (throttled and 5xx in lifecycle, microvm(), and session)

Mutation check of the fourth commit: a GetMicrovm UnrecognizedClientException treated as denied fails 1 test (the new UnrecognizedClientException row of test_check_reports_a_failed_get_microvm_without_failing_the_job), of 202 tests in lifecycle_test.py, operation_test.py, and session_test.py.

Edited tests (rule 5.4): the third commit removes or changes 5 tests from main, because the denied behavior they asserted is removed (a denied GetMicrovm no longer warns, keeps sending, and adds a note to the final error):

• request_test.py: test_adds_the_note_to_the_final_error and test_adds_no_note_when_the_note_is_none are removed with unavailable_note.
• session_test.py: test_keeps_sending_after_a_denied_state_check_and_names_it_at_the_end is replaced by test_fails_at_once_when_the_state_check_is_denied and test_keeps_sending_after_a_failed_state_check_with_one_warning.
• lifecycle_test.py: test_check_reports_a_failed_get_microvm_without_failing_the_job changes its parameters (AccessDeniedException and ThrottlingException become ThrottlingException and a 500) and drops the denied assert. test_check_reports_a_get_microvm_connection_failure drops the denied assert.

The fourth commit adds one row to test_check_reports_a_failed_get_microvm_without_failing_the_job and changes the quoted denied message in 3 tests that the third commit added. No existing test asserted that microvm() makes no GetMicrovm call. The second commit changes one expectation of a test that the first commit added: test_fails_at_once_when_the_microvm_has_ended expected Success.. and now expects Success., because of the period fix. Without the fix, its 4 cases fail.

Assumptions

• A terminated MicroVM's endpoint answers 502 with x-aws-proxy-error: MICROVM_CONNECT_FAILED: measured live in us-east-1 (table above, and the probe in the JS PR).
• CreateMicrovmAuthToken issues a token for a terminated MicroVM whose record still exists: measured live in us-east-1 (JS PR probe; the Python run above got past the token call too).
• The stateReason after TerminateMicrovm is Success.: measured live (message above).
• A MicroVM that the service itself fails (for example "Internal service error") answers the same 502: unverified. Only TerminateMicrovm was measured.
• The CreateMicrovmAuthToken answer for a removed MicroVM record (not only terminated): unverified, as before.

Decisions not covered by the issue or the design note

• The trigger stays a 502 with MICROVM_CONNECT_FAILED or no answer, not any 5xx. A plain 5xx comes from a running worker. A test pins it.
• A microvm() MicroVM that something outside the operation suspended is now resumed, as in a session. Before, the job retried until both tiers ended. Without lambda:ResumeMicrovm, it now fails at once with MicrovmDeliveryError. microvm() sets no idle policy, so its MicroVM suspends only through an outside call.
• microvm() and sessions now need lambda:GetMicrovm for HTTP delivery, and microvm() should have lambda:ResumeMicrovm. Without GetMicrovm, a job whose request gets the dead or suspended sign fails at once with an error that names the permission. README Permissions says so.
• (Replaced by the sixth commit.) The third commit added MicrovmGetDeniedError next to MicrovmResumeDeniedError. The rule 8 error model replaces both with MicrovmPermissionDeniedError, because both have the same retry behavior and the same public error.
• The warning text stays GetMicrovm failed, so the operation could not check whether the MicroVM is suspended. ... for microvm() too, so that log filters keep matching. Same as the JS SDK.
• The period fix in _raise_if_ended changes the session message too, because sessions share the function. The JS SDK made the same change.
• The new tests live in operation_test.py, because the changed branch is in operation.py (CONTRIBUTING: one test module per source module).

Checklist

• Every new branch has a test that fails without it (rule 4). The first commit adds no branch; the first mutation table covers the branches it now runs for microvm(). The third commit adds the AccessDeniedException branch in resume_if_suspended; the mutation table of the third commit covers it.
• For a bug fix: a test fails on the current code (rule 5). Rule 5.2 allows the description to show the test failing before the fix: the Tests section shows 8 of the first 12 new tests failing on main.
• Every edited test expectation is listed in the description, with the reason (rule 5). The third commit lists its replaced and removed tests under Tests.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[github-actions]

Codex AI review · Finding arf_v1_6ft4kjt3rh3nf5oyujf4tnmon7

[P2] Follow the package's required testing approach

The extras README's Custom clients and tests, rule 3 requires tests to wrap a real lambda-microvms client in botocore's Stubber. This factory and the added denial/resume tests instead synthesize responses and errors through scripted clients, bypassing validation against the API model. CONTRIBUTING rule 9 requires enforcing package rules even when the implementation is correct. Please return to the design phase under rule 8, step 6, and agree a testing approach that provides the required API-model validation.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

[github-actions]

Codex AI review

One unresolved approach finding at packages/aws-durable-execution-sdk-python-extras/tests/microvm/operation_test.py:1228: required API-model validation is bypassed. Tests were not executed under the read-only review constraints.

Reviewed commit 12c4e292a8f036726fc879a4bfba6c689de5f558. Workflow run