Repository navigation
[#1221] Add authorize display characterization tests - #1237
Conversation
vharseko
left a comment
There was a problem hiding this comment.
1. [P1] Keep credentials and consent responses out of failure artifacts
The setup now sends the administrator credentials through Playwright’s request fixture, and the tests pass complete consent responses to assertions such as expect(html).toContain(...). With the repository configuration retaining traces on failure and the workflow uploading the Playwright report and test results, a failed run can persist credentials, session information, or the consent page’s CSRF token in logs or artifacts. This also affects runs against a server configured through the OPENAM_* environment variables.
Please restore protection for the authentication and consent data in both API steps and assertion failure output, and verify it with synthetic secret markers in failure artifacts.
2. [P2] Apply the HTTPS setting to the independently created request contexts
apiRequest.newContext() creates an independent API request context and does not inherit use.ignoreHTTPSErrors from the Playwright configuration. Consequently, when OPENAM_BASE_URL points to an HTTPS deployment with a self-signed certificate, the administrator setup can succeed through the configured request fixture, but the resource-owner login fails at this context. The context created by sessionContext and the anonymous context have the same issue.
Please pass the appropriate HTTPS setting to all independently created contexts so this spec works with the same deployments as the configured fixture.
|
@vharseko, replies to the review:
|
vharseko
left a comment
There was a problem hiding this comment.
Thanks for addressing the HTTPS context settings. The failure artifacts contain test secrets, so I withdraw my earlier sanitization request. Approved.
57825f6
into
OpenIdentityPlatform:features/1075-remove-restlet
Part of #1221. Add the three device display scenarios from `b02024a4f0:e2e/oauth2/display-test.spec.mjs` on top of merged b3a (#1237). The incremental diff is **141 additions / 1 deletion (142 changed lines)** and touches only `e2e/oauth2/display-test.spec.mjs`: a separate consent-requiring device client, two helpers, and three tests. All nine merged authorize tests remain unchanged, giving twelve display tests in total. Production code, shared fixtures, workflows and existing device-PKCE coverage are unchanged. The consent scenario retains all four display variants (`page`, `popup`, `touch`, `wap`). For the three XUI templates it verifies the client identity, response type, scope, nonempty CSRF, user code and form target. For WAP it verifies the corresponding existing hidden submission fields and form action. The code-entry scenario checks all five display values, including `bogus`, and compares the complete HTML after masking only freshly generated CSRF values. ## Changed expectations from the source | Check | Source | Current Restlet contract | Reason | | --- | --- | --- | --- | | Reach device consent | GET with `user_code` and display in the query, expecting 200 | POST user code and CSRF in the form body with a session cookie; display stays in the query | Source GET returns 303 without processing the code. The POST follows the actual code-entry form. | | Consent post-back target | Includes user code in the query | Contains only display; XUI carries the code as `pageData.userCode` | The body-only user code does not become part of the query target. | | Unknown consent display | 400 `invalid_request`, exact enum description | Non-redirecting 400 HTML `server_error`; error-page title and error code retained, description omitted | The current renderer failure reaches the generic exception handler. #1221 excludes framework error descriptions. | | Entry-page HTML equality | Unmasked byte equality across displays | Equality after masking only the fresh CSRF property; a nonempty token is required on every page | Each render generates a different CSRF token. Display does not change the entry template. | ## Pinned defects - Touch renders the client name as `isplayName` and omits `displayName`. Target: `displayName`. - Unknown consent display returns 400 `server_error`. Target: 400 `invalid_request`. - The WAP consent action carries only display and the template has no hidden `user_code`, so its next consent submission loses the code. Target: a hidden `user_code` body field. This characterization explicitly requires the current absence; it does not fix the template. ## Validation Test commit: `e7f0c535e1f4d1969151a0d2aba77960e070c760`. Integration base: `57825f6732dd04bdd4ae6aafabb2d0a31cd7539b`. Server WAR source: `097005086e3113fcd81c382f4aa080ec2f823cf4`, successful [Build run 43852682205](https://github.com/OpenIdentityPlatform/OpenAM/actions/runs/43852682205), `ubuntu-latest-11` artifact 14222035312. Only e2e files differ between that source and the integration base, establishing production-source equivalence. The complete nested server WAR was verified against its archive entry size and CRC; the full outer archive digest was not checked. WAR SHA256: `39599c6caf586b5e31b4971a76548b135eaaa20b9c99ec57e148f55cf0718cde`. Local validation uses one OpenAM container with embedded OpenDJ. The temporary runner emits login success labels without token bytes; traces and screenshots are disabled and a local reporter suppresses existing secret-bearing diagnostics. New secret/HTML assertions compare booleans and retain useful labels. Five deliberately failing matchers using the actual added assertion lines confirmed that user-code, CSRF and complete-HTML sentinels are absent from captured assertion errors. Source-contract recording with diagnostic-only sanitization preserved all three source requests, payloads and expected contract values. Both runs executed all three scenarios: 0 passed / 3 failed / 0 skipped. The first failures were consent GET expecting 200 but receiving 303, unknown-display GET expecting 400 but receiving 303, and unmasked entry-page equality expecting true but receiving false after five 200 responses. Raw HTML matchers and secret-bearing logs/errors were sanitized without masking contract differences. | Run | Passed | Failed | Skipped | | --- | ---: | ---: | ---: | | Full display spec, each of two runs on one fresh stand | 12 | 0 | 0 | | Restored shared `oauth2 uma xacml`, each of two runs on one fresh stand | 132 | 0 | 0 | | Ignore-display product mutant | 5 | 7 | 0 | | Device-consent-only missing-CSRF product mutant | 11 | 1 | 0 | | Display-dependent code-entry template product mutant | 11 | 1 | 0 | The ignore-display mutant fails the five existing authorize checks for popup, touch, WAP, unknown display and case normalization. Device consent passes its page leg then fails popup's lowercase-doctype predicate; unknown device display fails expected 400 versus received 200. Entry rendering remains green. The missing-CSRF mutant returns 200 consent HTML but fails the exact CSRF-presence boolean; all authorize, unknown-display and entry scenarios stay green. Deployed module hashes were verified against each mutant WAR. The entry-template mutant routes only popup through a separate code-entry template, preserving the default entry page. All five responses retain their expected status and shell markers; the CSRF-masked HTML equality fails. Only the entry scenario is red. This mutates the product routing/template selection rather than inverting a test assertion. After restoring the original WAR, `oauth2 uma xacml` ran twice on one fresh embedded stand: **132 passed / 0 failed / 0 skipped in each run**, including all twelve display tests and all existing device-PKCE tests. The restored deployed OAuth2 module hash matches the original WAR. Discovery confirms 132 tests in nine files, superseding the plan's older counts. `node --check`, test discovery, `git diff --check`, incremental changed-line budget and independent code review passed. No review findings remain. All nine authorize tests are unchanged. No temporary specs or production mutations are present in the repository. The remaining b4/b5 ports and #1221's formal master acceptance condition remain outstanding. This PR completes the device slice of b3; it does not close #1221. PR-specific Build/build-docker validation is pending; no CodeQL result is claimed for this integration base.
Part of #1221.
Scope
Add nine authorize-only display characterization tests, adapting
b02024a4f0:e2e/oauth2/display-test.spec.mjsto the current Restlet implementation. They cover page, empty and absent display, popup composition, touch, WML, unknown display, case normalization (including the separate uppercasePOPUPwrapper selection), and authentication before display validation.The spec owns its consent-requiring public client and login session and sends S256 PKCE. Popup checks parse an inert document and require the consent model inside
#print, the container copied into the opened window. The WAP form action must preserve the complete HTML-escaped authorize query, including display and PKCE.The diff is one new spec, 406 additions / 0 deletions. Shared fixtures, existing tests, production code and workflows are unchanged. Device display coverage remains for b3b; the remaining #1221 slices are still outstanding.
Changed expectations from the source
Demo Democn, read through the user API and compared after inert HTML entity decodingformTargetredirect_uriConsentRequiredResourcepreserves the request query in its post-back target. The leading JavaScript slash escape is retained.invalid_requestserver_errorThe anonymous source assertions already pass. They are strengthened to parse the login redirect and check its exact origin/path,
realm=/, and completegoto; the observed status remains 301.Pinned defects
isplayNameand omitsdisplayName, leaving the client heading blank. Target:displayName, as in the page and popup templates.server_error. Target: 400invalid_requestfor an invalid parameter value (RFC 6749 section 4.1.2.1).Validation
Test revision:
66e4097e895e3fc236b9b4047057110c7b69d21f.Server baseline: integration commit
097005086e3113fcd81c382f4aa080ec2f823cf4.Local validation used one OpenAM container with embedded OpenDJ. A temporary reporter suppressed secret-bearing output from existing specs, and traces/screenshots were disabled. The latest spec passed all nine display scenarios after restoring the baseline module; the shared
oauth2 uma xacmlrun then passed 106 tests / 0 failed / 0 skipped. Syntax, discovery (106 tests in eight files, including nine display tests), andgit diff --checkpassed.Four targeted module mutations demonstrate the additional regression coverage. Each mutation was deployed separately; all runs had zero skipped tests:
equalsIgnoreCasetoequalsPOPUPretains the wrapper title and consent modeldisplay=#printThe Java mutations were compiled with
--release 11; the other two changed only templates. Mutations were packaged into temporary OAuth2-module JARs, with deployed SHA256 checked after every restart. The original module was restored and its hash verified before the final green runs. No mutation is committed.The spec-local Node HTTP/HTTPS transport keeps credentials and consent bodies outside Playwright API steps, sanitizes setup/transport/JSON errors, and does not follow redirects. Response-derived HTML, CSRF, model fields and URL comparisons use boolean assertions. The spec disables retained traces/screenshots and decodes the resource-owner name once through an inert textarea.
Earlier validation at
2c2767115c695e0346f1d83fa2eb173c59b5b3c9also checked five deliberate failure probes with the normal list/HTML reporters: authentication transport, provider transport, authorize transport, a consent assertion, and an invalid login Location. Synthetic secret markers were absent from stdout, report files, embedded report ZIP data, and generated failure artifacts after the safety fixes. Four synthetic name cases covered apostrophes, ampersands, named/numeric Unicode entities, and one-pass decoding; a mismatched name still failed. These safety probes were not rerun for the assertion-only follow-up.Server artifact provenance
The baseline WAR came from successful Build run 43852682205, exact baseline SHA,
ubuntu-latest-11artifact 14222035312, distribution-kit server WAR. The assembly copiesopenam-server:warunchanged. The complete nested WAR was extracted from the downloaded archive prefix and its ZIP entry size/CRC checked; the full outer artifact digest was not checked.WAR SHA256:
39599c6caf586b5e31b4971a76548b135eaaa20b9c99ec57e148f55cf0718cde.Baseline OAuth2-module SHA256:
91504f1640eb5a226cccd44993202e25bafba85e5c079bc84ea528d4fef27ce1.The local results above cover
oauth2 uma xacml; the full CI Playwright suite is a separate check. No CI or CodeQL result is claimed here for the latest test revision.