[kamal-kaur04]

What is this about?

On the CLI flow, every test in a build that uses accessibilityOptions.includeTagsInTestingScope / excludeTagsInTestingScope is reported as an accessibility test, including tests outside the tag scope. The binary sets each test's product_map.accessibility from the platform check alone (accessibility on + supported browser), so every test gets an accessibility test-execution record. The classic flow sends the per-test verdict (Listener.setTestRunAccessibilityVar), so only in-scope tests are recorded there.

This PR sends that verdict on the CLI flow:

• AccessibilityModule.setTestAccessibility sets test_accessibility on the test instance: accessibility on and the test in the tag scope (same shouldScanTestForAccessibility call the scan gate makes, so cucumber uses the gherkin tags and mocha the title match). Auto-scanning is deliberately left out, so tests scanned manually with performScan() keep their accessibility row; it still gates the automatic scans.
• Cucumber scenarios skipped because a BeforeAll failed are built straight into TEST/POST and never pass TEST/PRE, so they are created with test_accessibility: false (they never ran or scanned).
• It is registered on TEST/PRE with a new prepend option (EventDispatcher.registerObserver / TestFramework.registerObserver), so it runs before TestHubModule serializes TestRunStarted. Without that, TestRunStarted would still carry the platform-only value.
• The binary uses test_accessibility when it is a boolean and keeps the platform check otherwise (paired binary PR). Older binaries ignore the extra key.

Effect: in a tag-scoped build, out-of-scope tests no longer get accessibility test-execution records, matching the classic flow. Scanning behaviour is unchanged.

Backport of #359 to v9.

Related Jira task/s

N/A (tracked internally)

Release (mandatory for every PR — required for the ready-for-review label)

Version bump: (required — tick exactly one)

• minor (backwards-compatible feature)
• patch (bug fix or other small change)

Release notes type: (optional)

• New Feature
• Bug Fix
• Other Improvement

Release notes (customer-facing): (optional but encouraged)

• Fixed tests outside includeTagsInTestingScope / excludeTagsInTestingScope being reported as accessibility tests when using the CLI.

Release notes (internal): (required — engineer-facing; what actually changed / why)

• CLI flow: the accessibility module now sends a per-test test_accessibility verdict (a11y on and test in tag scope; auto-scanning excluded so manual-scan tests keep their row) on test events, set by a TEST/PRE observer prepended ahead of TestHubModule so TestRunStarted carries it; cucumber cascade-skipped scenarios send false. Restores tag scoping for product_map.accessibility; requires the paired binary change to take effect.

Checklist

• Ready to review
• Has it been tested locally?

Unit tests: accessibilityModule.test.ts covers the verdict (in scope / out of scope / auto-scanning off keeps an in-scope test / platform unsupported / mocha title match) and cascade-skipped scenarios carrying false, plus the prepended registration. eventDispatcher.test.ts covers prepend ordering (main and v9 only). tsc --noEmit is clean.

End to end (WDIO 9 + cucumber, Chrome latest / Windows 11, includeTagsInTestingScope: ['@accessibility'], CLI flow; 1 tagged scenario + 2 untagged), product_map.accessibility on TestRunStarted / TestRunFinished:

Run	Service	Binary	Tagged	Untagged 1	Untagged 2
Before	9.44.0 (released)	released	true / true	true / true	true / true
After	this branch (v9)	paired fix (local)	true / true	false / false	false / false
Compat	this branch (v9)	released	true / true	true / true	true / true

• After: the tagged scenario is still scanned and its results saved, with the same issue count on the accessibility dashboard as Before.
• Compat: an older binary ignores the new key, so behaviour is unchanged and the run is green.

PR Validations

Run Tests: Comment RUN_TESTS to trigger sanity tests.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto (fallback from Default branch)
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Central YAML (base), Organization UI (inherited), Workspace UI (inherited); Enforced overrides: Organization
• Review profile: ASSERTIVE
• Plan: Enterprise
• Run ID: 9fd1fcb8-177a-4084-b142-380dc089ec15

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

[kamal-kaur04]

RUN_TESTS

[minionhelperappqa]

[SDK Wdio Test] TRA build state: failed | Stability 91% — verdict: failure. Passed: 53, Failed: 5, Aggregate: 58. TRA: https://observability.browserstack.com/builds/5wc8dziwlsedget28eqewor1eztbfhlb8yiaspn5

[kamal-kaur04]

⚠️ Needs human review

File	Confidence
.changeset/pr-360.md	✅ All Clear
packages/browserstack-service/src/cli/eventDispatcher.ts	✅ All Clear
packages/browserstack-service/src/cli/frameworks/constants/testFrameworkConstants.ts	✅ All Clear
packages/browserstack-service/src/cli/frameworks/testFramework.ts	✅ All Clear
packages/browserstack-service/src/cli/frameworks/wdioCucumberTestFramework.ts	✅ All Clear
packages/browserstack-service/src/cli/modules/accessibilityModule.ts	✅ All Clear
packages/browserstack-service/tests/cli/eventDispatcher.test.ts	✅ All Clear
packages/browserstack-service/tests/cli/modules/accessibilityModule.test.ts	✅ All Clear
packages/browserstack-service/tests/service.test.ts	✅ All Clear

Process integrity

The review gate could NOT certify this run — the verdict is held at ⚠️ pending regardless of findings:

• G1 — u004: no findings file — unit never judged
• G1 — u007: no findings file — unit never judged
• G3 — cross-cutting: no full read receipt for sdk-diff.md

Change map (generated deterministically from the diff)

graph LR
  subgraph nwdio_service["wdio-service"]
    npackages_browserstack_service_src_cli_modules_accessibilityModule_ts["accessibilityModule.ts<br/>~49 lines"]
    npackages_browserstack_service_tests_cli_modules_accessibilityModule_test_ts["accessibilityModule.test.ts<br/>~49 lines"]
    npackages_browserstack_service_src_cli_eventDispatcher_ts["⚠ eventDispatcher.ts<br/>~10 lines"]
    npackages_browserstack_service_tests_cli_eventDispatcher_test_ts["⚠ eventDispatcher.test.ts<br/>~10 lines"]
    n_changeset_pr_360_md["pr-360.md<br/>~5 lines"]
    npackages_browserstack_service_src_cli_frameworks_testFramework_ts["testFramework.ts<br/>~5 lines"]
    npackages_browserstack_service_src_cli_frameworks_wdioCucumberTestFramework_ts["wdioCucumberTestFramework.ts<br/>~2 lines"]
    npackages_browserstack_service_src_cli_frameworks_constants_testFrameworkConstants_ts["testFrameworkConstants.ts<br/>~1 lines"]
    npackages_browserstack_service_tests_service_test_ts["service.test.ts<br/>~1 lines"]
  end

Loading

↻ This verdict comment is the review anchor — it's updated in place on each run (the gate posts its status separately).

— SDK PR Review Agent

[github-actions]

🟢 SDK PR Review gate is green — the SDK PR Review Agent has run on the current head commit (verdict: pending).

This gate confirms a review ran on the latest commit. The verdict itself is advisory — read the findings and use your judgement; it does not block merge. A native GitHub reviewer approval is still separately required by branch protection before this PR can merge.