From 115fe2c5193ff85c3d0e5b079fda520f485367c8 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Tue, 3 Nov 2026 11:11:44 +0800 Subject: [PATCH 1/3] feat(WO13): port the ISC grammar to the parsanol .parg DSL grammar/isc.parg mirrors lib/interscript/isc/grammar/concerns/*.rb rule-for-rule: system/metadata/notes/relations/tests/aliases/ dependencies/stages, both sub forms, before/after/not_* constraints, any(range/set/alias/item), capture/maybe/some, ref(N), the strict comment_item, and the corpus escape rulings (\uXXXX, \U 4..8, lone \U). Root pins full-input consumption with an explicit !utf. --- grammar/isc.parg | 323 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 323 insertions(+) create mode 100644 grammar/isc.parg diff --git a/grammar/isc.parg b/grammar/isc.parg new file mode 100644 index 00000000..c9d4ffc8 --- /dev/null +++ b/grammar/isc.parg @@ -0,0 +1,323 @@ +# Interscript ISC grammar — portable .parg (parsanol grammar language). +# +# A 1:1 port of the canonical Ruby-DSL grammar in +# lib/interscript/isc/grammar/concerns/*.rb (+ core.rb). Every rule in +# this file carries a comment naming its concern rule of origin. The +# port is semantic, not cosmetic — the parsanol .parg surface has no +# `.maybe`/`.repeat`/`.absent?` combinators, so they map as: +# +# Ruby .parg +# ----------------------- -------------------------------- +# x.repeat(0) / repeat *x +# x.repeat(1) 1*x +# x.maybe [x] +# x.absent? !x +# x.present? &x +# a | b a / b (ordered choice, PEG) +# a >> b a b +# expr.as(:name) (expr) as name +# +# Two semantics worth calling out because the corpus depends on them: +# +# * The Ruby DSL's `any` matches ANY character including raw newlines +# (multiline `description { ... }` bodies parse). `chr` here is that +# true any-character matcher. Wherever the concerns stop at a newline +# they do it with an EXPLICIT newline.absent? — mirrored verbatim +# below (line_comment, field_raw, comment_item). +# * Choice is ordered (PEG), exactly like the Ruby alternation — the +# alternative order below is copied verbatim from the concerns and is +# load-bearing (item_atom, metadata_field, stage_item, ...). +# +# Consumed by tools/isc-parg-check (parsanol-rs validation harness). + +grammar InterscriptIsc version "1.0.0" edition "v1" { + + ## Interscript ISC map language, ported from the Ruby-DSL concerns. + + # --------------------------------------------------------------------- + # Concerns::Primitives — whitespace, comments, identifiers, strings + # --------------------------------------------------------------------- + + # The DSL's `any` — every Unicode scalar value, raw newlines included. + utf = %x00-10FFFF + chr = utf + + # rule(:newline) — "\n" | "\r\n" | "\r" + newline = %x0A / %x0D.0A / %x0D + + # rule(:line_comment) — "#" >> (newline.absent? >> any).repeat + # (stops at CR too — the absent? is on the full newline rule) + line_comment = "#" *(!newline chr) + + # rule(:space) — /\s/ is [ \t\r\n\f\v] + space = 1*(%x20 / %x09 / %x0A / %x0B / %x0C / %x0D) + + # rule(:whitespace) / rule(:whitespace?) + whitespace = 1*(space / line_comment) + wsq = [whitespace] + + # rule(:inline_space) / rule(:inline_whitespace) / inline_whitespace? + inline_space = 1*(%x20 / %x09) + inline_whitespace = 1*(inline_space / line_comment) + iwsq = [inline_whitespace] + + # rule(:comma) — "," with trailing whitespace allowed + comma = "," [whitespace] + + # rule(:arrow) — whitespace? >> "->" >> whitespace? + arrow = [whitespace] "->" [whitespace] + + # rule(:identifier) + ident_first = %x41-5A / %x61-7A / %x5F + ident_rest = %x41-5A / %x61-7A / %x30-39 / %x5F + identifier = (ident_first 0*ident_rest) as identifier + + # rule(:escape_sequence) — the corpus rulings are in the concerns: + # \uXXXX (exactly 4), \U with 4..8 hex (gki-bel "\U040E"), and a lone + # \U (un-mar converter corruption). + esc_nl = ("n") as newline + esc_cr = ("r") as carriage_return + esc_tab = ("t") as tab + esc_dq = (%x22) as dquote + esc_bs = (%x5C) as backslash + esc_uni4 = "u" (4hexd) as unicode + esc_uni = "U" (4*8hexd) as unicode + esc_u_lone = ("U") as u_lone + hexd = %x30-39 / %x41-46 / %x61-66 + escape_sequence = %x5C (esc_nl / esc_cr / esc_tab / esc_dq / esc_bs / esc_uni4 / esc_uni / esc_u_lone) + + # rule(:plain_run) — escape-free stretch of a double-quoted string + plain_run = (!%x22 !%x5C chr) + + # rule(:double_quoted_string) — runs captured as slices + dq_part = escape_sequence / (1*plain_run as run) + double_quoted_string = %x22 (*(dq_part) as string) %x22 + + # rule(:single_quoted_string) — no escape interpretation + single_quoted_string = %x27 (*( !%x27 chr ) as string) %x27 + + # rule(:quoted_string) + quoted_string = double_quoted_string / single_quoted_string + + # braced(inner) is a Ruby method (scaffold), not a rule: every use + # site below spells "{" [whitespace] inner [whitespace] "}" inline. + + # --------------------------------------------------------------------- + # Concerns::Items — item expressions + # --------------------------------------------------------------------- + + # rule(:keyword) — the concern's ordered list. One deviation, forced + # by the parg compiler's shadow lint: "any_character" must precede + # "any" (longest-first). Semantically identical — keyword is used + # only under negative lookahead, where any matching branch suffices. + keyword = ("parallel" / "sequence" / "stage" / "compose" / "separate" / "system" / "metadata" / "aliases" / "tests" / "notes" / "description" / "name" / "authority" / "dependency" / "run" / "sub" / "before" / "after" / "not_before" / "not_after" / "any_character" / "any" / "none" / "boundary" / "line_start" / "line_end" / "word_boundary" / "downcase" / "upcase" / "title_case" / "capture" / "maybe" / "some" / "ref") + + # rule(:zero_width_primitive) + zw_primitive = (("boundary" / "line_start" / "line_end" / "word_boundary" / "space" / "non_boundary")) as primitive + + # rule(:function_call) + fn_lit = (("upcase" / "downcase" / "title_case" / "reverse" / "strip" / "swapcase")) as function + + # rule(:any_character) + any_char_lit = ("any_character") as any_char + + # str("none").as(:none) + none_lit = ("none") as none + + # rule(:capture_reference) — ref(N), single digit + digit = %x30-39 + capture_reference = (("ref" "(" [whitespace] (digit as digit) [whitespace] ")")) as ref + + # rule(:alias_reference) — identifier with optional .qualified part, + # rejected when a keyword or followed by "{". + alias_reference = ((!(keyword) identifier [("." (identifier as qualified_name))] !"{")) as alias + + # rule(:range_arg) + range_arg = ((quoted_string) as lo) [whitespace] ".." [whitespace] ((quoted_string) as hi) + + # rule(:set_arg) — quoted single, or [ item, item, ... ] + list_items = item 0*((comma / whitespace) item) + set_arg = ((quoted_string) as single) / ("[" [whitespace] ((list_items) as list) [whitespace] "]") + + # rule(:alias_arg) — bare alias inside any(...) + alias_arg = ((!(zw_primitive) !(keyword) identifier)) as alias_ref + + # rule(:any_constructor) + any_ctor = "any" "(" [whitespace] ((range_arg / set_arg / alias_arg / (item as any_item)) as any) [whitespace] ")" + + # rule(:capture_constructor) / maybe / some + capture_ctor = "capture" "(" [whitespace] ((item) as capture_inner) [whitespace] ")" + maybe_ctor = "maybe" "(" [whitespace] ((item) as maybe_inner) [whitespace] ")" + some_ctor = "some" "(" [whitespace] ((item) as some_inner) [whitespace] ")" + + # rule(:item_atom) — ordered exactly as the concern + item_atom = quoted_string / none_lit / zw_primitive / any_char_lit / any_ctor / capture_ctor / maybe_ctor / some_ctor / fn_lit / capture_reference / alias_reference + + # rule(:item_continuation) — lookahead only: not a block-rule keyword, + # not `identifier =` (an alias declaration), and starts like an atom. + item_atom_start = %x22 / %x27 / ident_first + item_continuation = !("to" / "before" / "after" / "not_before" / "not_after" / "}") (!(identifier [whitespace] "=") item_atom_start) + + # rule(:concat_sep) — "+" with optional ws, or required whitespace + concat_sep = ([whitespace] "+" [whitespace]) / whitespace + + # rule(:item) — concatenation, one or more atoms + item = ((item_atom 0*((concat_sep &(item_continuation)) item_atom))) as concatenation + + # --------------------------------------------------------------------- + # Concerns::Items — constraints + # --------------------------------------------------------------------- + + # rule(:constraint) — order verbatim: before | after | not_before | not_after + constraint = ("before" whitespace ((item) as before)) / ("after" whitespace ((item) as after)) / ("not_before" whitespace ((item) as not_before)) / ("not_after" whitespace ((item) as not_after)) + + # rule(:constraints) — (whitespace >> constraint).repeat + constraints = (*(whitespace constraint)) as constraints + + # --------------------------------------------------------------------- + # Concerns::Stages — rules, directives, stage bodies + # --------------------------------------------------------------------- + + # rule(:compact_rule) — from/to are single atoms + compact_rule = "sub" whitespace ((item_atom) as from) whitespace ((item_atom) as to) constraints + + # rule(:block_rule) — from/to are concatenations + block_rule = "sub" [whitespace] "{" [whitespace] "from" whitespace ((item) as from) [whitespace] "to" whitespace ((item) as to) constraints [whitespace] "}" + + # rule(:rule) — block | compact, ordered + rule = block_rule / compact_rule + + # rule(:comment_item) — strict parse: comments only, no stray + # identifiers (WO87/A4 ruling). + comment_item = ("#" *(!%x0A chr)) as comment + + # rule(:bare_rule) — legacy .imp compatibility + bare_rule = ((rule)) as bare_rule + + # rule(:sequence_block) / rule(:parallel_block) + rule_line = [whitespace] (rule / comment_item) [whitespace] + rule_items = *(rule_line) + sequence_block = "sequence" [whitespace] "{" [whitespace] ((rule_items) as sequence) [whitespace] "}" + parallel_block = "parallel" [whitespace] "{" [whitespace] ((rule_items) as parallel) [whitespace] "}" + + # rule(:separate_directive) + separate_directive = (("separate")) as separate [whitespace "separator" whitespace ((item_atom) as separator)] + + # rule(:string_case_directive) — title_case takes optional kwargs + kwarg = ((identifier) as kwarg_name) [whitespace] ":" [whitespace] ((quoted_string) as kwarg_value) + kwarg_list = ((kwarg) as kwarg) 0*(("," [whitespace]) ((kwarg) as kwarg)) + string_case_directive = (("downcase" / "upcase")) as case / (("title_case") as case [whitespace ((kwarg_list) as case_kwargs)]) + + # rule(:compose_directive) + compose_directive = (("compose" / "decompose")) as compose + + # rule(:rababa_directive) + rababa_directive = (("rababa")) as funcall_name [whitespace ((kwarg_list) as funcall_kwargs)] + + # rule(:run_rule) — map..stage. | stage. + run_rule = "run" whitespace (("map." ((identifier) as dep) ".stage." ((identifier) as stage)) / (("stage." ((identifier) as stage))) as run_stage_only) + + # rule(:stage_item) — order verbatim + stage_item = [whitespace] (sequence_block / parallel_block / run_rule / separate_directive / string_case_directive / compose_directive / rababa_directive / bare_rule / comment_item) [whitespace] + stage_items = *(stage_item) + + # rule(:stage_block) — stage(name) | stage name + stage_block = "stage" (( "(" ((identifier) as stage_name) ")" ) / (whitespace ((identifier) as stage_name))) [whitespace] "{" [whitespace] ((stage_items) as stage) [whitespace] "}" + + # --------------------------------------------------------------------- + # Concerns::Tests + # --------------------------------------------------------------------- + + # rule(:test_line) + test_line = [whitespace] ((quoted_string) as input) arrow ((quoted_string) as expected) [(whitespace "note" whitespace ((quoted_string) as note))] [whitespace] + test_items = *(test_line) + tests_block = "tests" [whitespace] "{" [whitespace] ((test_items) as tests) [whitespace] "}" + + # --------------------------------------------------------------------- + # Concerns::Metadata + # --------------------------------------------------------------------- + + # rule(:raw_text) + raw_text = *( %x5C "{" / %x5C "}" / (!"}" chr) ) + + # rule(:description_field) + description_field = "description" [whitespace] "{" [whitespace] ((raw_text) as description) [whitespace] "}" + + # rule(:relations_field) / relation_block / relation_type + relation_type = "supersedes" / "superseded_by" / "based_on" / "basis_for" / "alias_of" / "adopted_from" / "related_to" + relation_block = [whitespace] ((relation_type)) as type whitespace ((quoted_string)) as system [(whitespace "note" whitespace ((quoted_string) as note))] [whitespace] + relations_block = "relations" [whitespace] "{" [whitespace] ((*(relation_block)) as relations) [whitespace] "}" + + # rule(:notes_field) / note_line + note_line = [whitespace] "note" [whitespace] ((quoted_string) as note) [whitespace] + notes_field = "notes" [whitespace] "{" [whitespace] ((*(note_line)) as notes) [whitespace] "}" + + # rule(:system_status_field) / code_status_field + system_status_field = "system_status" whitespace (("current" / "former" / "inactive")) as system_status + code_status_field = "code_status" whitespace (("preferred" / "proposed" / "deprecated")) as code_status + + # rule(:specification_field) + spec_value = (quoted_string) as specification + specification_field = "specification" whitespace ((spec_value 0*((comma) (spec_value))) as specification) + + # rule(:provenance_field) + provenance_value = (quoted_string) as provenance + provenance_field = "provenance" [whitespace] ((provenance_value 0*((comma) (provenance_value))) as provenance) + + # rule(:generic_field) / empty_field / field_value + empty_field = (&newline / &"}") + field_raw = 1*(!newline !"}" !"{" chr) + field_value = ((quoted_string / field_raw)) as field_value + field_block = "{" [whitespace] ((raw_text) as field_block) [whitespace] "}" + generic_field = ((identifier) as field_name) iwsq (empty_field / field_value / field_block) + + # rule(:metadata_field) — order verbatim. The concern's dead rules + # (authority_field, source/target_spelling_field, identifying_field, + # name_field) are unreachable from metadata_field and are subsumed by + # generic_field here; see grammar/ISC-PARG-GAPS.md. + metadata_field = [whitespace] (description_field / relations_block / system_status_field / code_status_field / specification_field / notes_field / provenance_field / generic_field) [whitespace] + metadata_items = *(metadata_field) + metadata_block = "metadata" [whitespace] "{" [whitespace] ((metadata_items) as metadata) [whitespace] "}" + + # --------------------------------------------------------------------- + # Concerns::Aliases + # --------------------------------------------------------------------- + + # rule(:alias_decl) + alias_decl = [whitespace] ((identifier) as name) [whitespace] "=" [whitespace] ((item) as value) [whitespace] + alias_items = *(alias_decl) + aliases_block = "aliases" [whitespace] "{" [whitespace] ((alias_items) as aliases) [whitespace] "}" + + # --------------------------------------------------------------------- + # Concerns::Dependencies + # --------------------------------------------------------------------- + + # rule(:dependency_decl) + dependency_decl = "dependency" whitespace ((quoted_string) as target) [(whitespace "as" whitespace ((identifier) as alias))] + + # --------------------------------------------------------------------- + # Concerns::System + # --------------------------------------------------------------------- + + # rule(:block_item) — order verbatim + block_item = [whitespace] (metadata_block / aliases_block / tests_block / stage_block / dependency_decl) [whitespace] + + # rule(:system_block) + system_block = "system" whitespace ((quoted_string) as system_code) [whitespace] "{" [whitespace] ((*(block_item)) as body) [whitespace] "}" [whitespace] + + # rule(:isc_source) — root; !utf pins full-input consumption + isc_source = [whitespace] ((system_block)) as system [whitespace] !utf + + entry isc: isc_source + + test isc { + accept "system \"T:a-b:c-d:1\" { stage main { parallel { sub \"a\" \"b\" } } }" + accept "system \"T:a-b:c-d:1\" { metadata { } tests { } }" + accept "system \"T:a-b:c-d:1\" { stage main { sub { from \"x\" to \"y\" before boundary } } }" + accept "system \"T:a-b:c-d:1\" { stage main { sub { from \"e\" to \"f\" after any(\"aeiou\") } } }" + reject "system \"T:a-b:c-d:1\" {" + reject "system \"T:a-b:c-d:1\" { stage main { sub \"a\" } }" + reject "system \"T:a-b:c-d:1\" { stage main { oops } }" + reject "system \"T:a-b:c-d:1\" }" + } +} From 56116222a583bfe8ee0cec92b934a8121a75cd65 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Tue, 3 Nov 2026 11:12:03 +0800 Subject: [PATCH 2/3] feat(WO13): parsanol-rs validation harness for grammar/isc.parg tools/isc-parg-check compiles the .parg through parsanol-rs (parse_grammar_with_mode + compile_entry_mode, edition v1) and parses corpus files with the portable engine, reporting failure wires (offset/line/col/expected) on misses. --expect-reject runs the negative corpus: truncated rule string, unbalanced brace, stray stage identifier (strict parse), trailing garbage. Verdicts are at parity with the Ruby DSL on all runs: 319/319 corpus files parse (318 maps/ + 1 maps-staging/), 4/4 negatives and 7/7 construct-coverage fixtures (stage(name) paren form, sequence, title_case kwargs, separate separator, kwarg funcall, run map.N.stage.M, relations, specification/provenance comma lists, aliases, capture+ref, CJK/emoji string content) match DSL accept/reject exactly. --- tools/isc-parg-check/.gitignore | 1 + tools/isc-parg-check/Cargo.lock | 456 ++++++++++++++++++ tools/isc-parg-check/Cargo.toml | 11 + tools/isc-parg-check/src/main.rs | 169 +++++++ .../tests/constructs/aliases-and-refs.isc | 11 + .../tests/constructs/metadata-extras.isc | 20 + .../tests/constructs/sequence-block.isc | 8 + .../tests/constructs/stage-directives.isc | 11 + .../tests/constructs/stage-paren-form.isc | 7 + .../tests/constructs/title-case-kwarg.isc | 5 + .../tests/constructs/unicode-strings.isc | 14 + .../tests/negative/stray-identifier.isc | 5 + .../tests/negative/trailing-garbage.isc | 3 + .../tests/negative/truncated-string.isc | 7 + .../tests/negative/unbalanced-brace.isc | 6 + 15 files changed, 734 insertions(+) create mode 100644 tools/isc-parg-check/.gitignore create mode 100644 tools/isc-parg-check/Cargo.lock create mode 100644 tools/isc-parg-check/Cargo.toml create mode 100644 tools/isc-parg-check/src/main.rs create mode 100644 tools/isc-parg-check/tests/constructs/aliases-and-refs.isc create mode 100644 tools/isc-parg-check/tests/constructs/metadata-extras.isc create mode 100644 tools/isc-parg-check/tests/constructs/sequence-block.isc create mode 100644 tools/isc-parg-check/tests/constructs/stage-directives.isc create mode 100644 tools/isc-parg-check/tests/constructs/stage-paren-form.isc create mode 100644 tools/isc-parg-check/tests/constructs/title-case-kwarg.isc create mode 100644 tools/isc-parg-check/tests/constructs/unicode-strings.isc create mode 100644 tools/isc-parg-check/tests/negative/stray-identifier.isc create mode 100644 tools/isc-parg-check/tests/negative/trailing-garbage.isc create mode 100644 tools/isc-parg-check/tests/negative/truncated-string.isc create mode 100644 tools/isc-parg-check/tests/negative/unbalanced-brace.isc diff --git a/tools/isc-parg-check/.gitignore b/tools/isc-parg-check/.gitignore new file mode 100644 index 00000000..ea8c4bf7 --- /dev/null +++ b/tools/isc-parg-check/.gitignore @@ -0,0 +1 @@ +/target diff --git a/tools/isc-parg-check/Cargo.lock b/tools/isc-parg-check/Cargo.lock new file mode 100644 index 00000000..9ba71291 --- /dev/null +++ b/tools/isc-parg-check/Cargo.lock @@ -0,0 +1,456 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "ahash" +version = "0.8.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" +dependencies = [ + "cfg-if", + "getrandom 0.3.4", + "once_cell", + "version_check", + "zerocopy", +] + +[[package]] +name = "aho-corasick" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba" +dependencies = [ + "memchr", +] + +[[package]] +name = "allocator-api2" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923" + +[[package]] +name = "block-buffer" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa" +dependencies = [ + "hybrid-array", +] + +[[package]] +name = "bumpalo" +version = "3.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" + +[[package]] +name = "cfg-if" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600" + +[[package]] +name = "const-oid" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" + +[[package]] +name = "cpufeatures" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566" +dependencies = [ + "libc", +] + +[[package]] +name = "crypto-common" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453" +dependencies = [ + "hybrid-array", +] + +[[package]] +name = "digest" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" +dependencies = [ + "block-buffer", + "const-oid", + "crypto-common", +] + +[[package]] +name = "equivalent" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "00d174d5400e5e8fd687ad1049e2f578285fa914201b1af7e8b112a4546bd826" + +[[package]] +name = "foldhash" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb" + +[[package]] +name = "getrandom" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" +dependencies = [ + "cfg-if", + "js-sys", + "libc", + "r-efi 5.3.0", + "wasip2", + "wasm-bindgen", +] + +[[package]] +name = "getrandom" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099" +dependencies = [ + "cfg-if", + "js-sys", + "libc", + "r-efi 6.0.0", + "wasm-bindgen", +] + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" +dependencies = [ + "allocator-api2", + "equivalent", + "foldhash", +] + +[[package]] +name = "hybrid-array" +version = "0.4.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "27f864f10dfb56725ce5ce5472bc52252c8f93a4ab86327122cebf62c5f59a17" +dependencies = [ + "typenum", +] + +[[package]] +name = "isc-parg-check" +version = "0.1.0" +dependencies = [ + "parsanol", +] + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "js-sys" +version = "0.3.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8dc09d2154098bbf4ef54eab6b117a3c24cbbffe94046b16ee6d6d4510584d3a" +dependencies = [ + "cfg-if", + "wasm-bindgen", +] + +[[package]] +name = "libc" +version = "0.2.191" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "65cd87c9a6d60536580b316ad985726104730a5151c99845f2490428ef91240a" + +[[package]] +name = "memchr" +version = "2.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "parsanol" +version = "0.19.8" +dependencies = [ + "ahash", + "getrandom 0.3.4", + "getrandom 0.4.3", + "hashbrown", + "parsanol-derive", + "regex", + "regex-syntax", + "serde", + "serde_json", + "sha2", +] + +[[package]] +name = "parsanol-derive" +version = "0.19.8" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.9", +] + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "5.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f" + +[[package]] +name = "r-efi" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" + +[[package]] +name = "regex" +version = "1.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4" + +[[package]] +name = "rustversion" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f" + +[[package]] +name = "serde" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.9", +] + +[[package]] +name = "serde_json" +version = "1.0.154" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7e9cc8b1b85264074fbcc02a88680c4096b1e47df8f739dceb03bf482f04bd6" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "sha2" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "3.0.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d78c8dee4c7bf0e14673097256fed6142ce9d3b85a408189d07482442145823b" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "typenum" +version = "1.20.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" + +[[package]] +name = "unicode-ident" +version = "1.0.27" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2c754d6c33795a1c324727428e5a7dedb5b06195f9890bdbcba760d3e246563" + +[[package]] +name = "version_check" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" + +[[package]] +name = "wasip2" +version = "1.0.4+wasi-0.2.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b67efb37e106e55ce722a510d6b5f9c17f083e5fc79afc2badeb12cc313d9487" +dependencies = [ + "wit-bindgen", +] + +[[package]] +name = "wasm-bindgen" +version = "0.2.130" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a0181c14ddd6c7b8b0b0132a5f1ea2f32be0d893cd3ea97b6e3b28886b8a3d1c" +dependencies = [ + "cfg-if", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.130" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e931a02a39abdbf66966fc35902b5ed58344750a70885bed4ebb3d8eaf28ed9" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.130" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ca4b54280e455bd831e33a6b8e734baf432fe63fbc35c42ac14ce67a043a53a" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn 3.0.9", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.130" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b1f9c3798725016c6ee5f2f57869ec0831e8bbc0ab0310a86e787e97f41389e" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "wit-bindgen" +version = "0.57.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e" + +[[package]] +name = "zerocopy" +version = "0.8.63" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5fe1f8f1b06191a00962174c61aa5005e0bb391a6d80d07e24d115c01a92ed8" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.63" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "863ad3ac83293fb4d740aedbfdc9240dd8d1a50c1099acd76ce80ce7c7230c7f" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "zmij" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" diff --git a/tools/isc-parg-check/Cargo.toml b/tools/isc-parg-check/Cargo.toml new file mode 100644 index 00000000..f9782581 --- /dev/null +++ b/tools/isc-parg-check/Cargo.toml @@ -0,0 +1,11 @@ +[package] +name = "isc-parg-check" +version = "0.1.0" +edition = "2021" +publish = false + +# WO13 validation harness: parse .isc corpus files with grammar/isc.parg +# through parsanol-rs. Local development tool — the parsanol path is a +# local checkout, not a crates.io release pin. +[dependencies] +parsanol = { path = "/Users/mulgogi/src/parsanol/parsanol-rs/parsanol" } diff --git a/tools/isc-parg-check/src/main.rs b/tools/isc-parg-check/src/main.rs new file mode 100644 index 00000000..622289f0 --- /dev/null +++ b/tools/isc-parg-check/src/main.rs @@ -0,0 +1,169 @@ +//! WO13 validation harness: compile grammar/isc.parg with parsanol-rs +//! and parse .isc corpus files with it. +//! +//! Usage: +//! isc-parg-check ... [--expect-reject] +//! +//! Each .isc file must parse to completion (the grammar pins EOF with +//! `!utf`). On failure the failure wire (offset, line, column, expected +//! set) is printed. Exit status: 0 all pass, 1 any failure. +//! +//! `--expect-reject` flips the verdict: every listed file must FAIL to +//! parse (negative corpus). + +use parsanol::parg::frontend::compile::{compile_entry_mode, Tables}; +use parsanol::parg::frontend::parser::parse_grammar_with_mode; +use parsanol::{AstArena, PortableParser}; +use std::path::{Path, PathBuf}; +use std::process::ExitCode; + +fn main() -> ExitCode { + let mut args: Vec = std::env::args().skip(1).collect(); + let expect_reject = match args.iter().position(|a| a == "--expect-reject") { + Some(i) => { + args.remove(i); + true + } + None => false, + }; + if args.len() < 2 { + eprintln!("usage: isc-parg-check ... [--expect-reject]"); + return ExitCode::from(2); + } + let parg_path = args.remove(0); + + let source = match std::fs::read_to_string(&parg_path) { + Ok(s) => s, + Err(err) => { + eprintln!("{}: unreadable: {err}", parg_path); + return ExitCode::from(2); + } + }; + let (parsed, mode) = match parse_grammar_with_mode(&source) { + Ok(pair) => pair, + Err(err) => { + eprintln!("{}: parg parse failed: {err}", parg_path); + return ExitCode::from(2); + } + }; + let grammar = match compile_entry_mode( + &parsed.document, + "isc", + &Tables::new(), + &parsed.atomic_rules, + mode.as_deref(), + ) { + Ok(g) => g, + Err(err) => { + eprintln!("{}: parg compile failed: {err}", parg_path); + return ExitCode::from(2); + } + }; + println!( + "compiled {} (edition {:?}) — {} rules", + parg_path, + mode.as_deref().unwrap_or("1"), + parsed.document.rules.len() + ); + + let mut files: Vec = Vec::new(); + for arg in &args { + let path = Path::new(arg); + if path.is_dir() { + let mut dir_files: Vec = walk_isc(path); + dir_files.sort(); + files.extend(dir_files); + } else { + files.push(path.to_path_buf()); + } + } + + let mut passed = 0usize; + let mut failed = 0usize; + for file in &files { + let input = match std::fs::read_to_string(file) { + Ok(s) => s, + Err(err) => { + eprintln!("FAIL {} (unreadable: {err})", file.display()); + failed += 1; + continue; + } + }; + let mut arena = AstArena::for_input(input.len().max(1 << 16)); + let mut parser = PortableParser::new(&grammar, &input, &mut arena); + let outcome = parser.parse(); + let ok = outcome.is_ok(); + let wanted = if expect_reject { !ok } else { ok }; + if wanted { + passed += 1; + if expect_reject { + println!("OK {} (rejected as expected)", file.display()); + } + } else if ok { + println!("FAIL {} (parsed, expected rejection)", file.display()); + failed += 1; + } else { + let err = outcome.unwrap_err(); + let msg = match (err.is_abort(), parser.failure_wire()) { + (false, Some((offset, expected))) => { + let (line, column) = line_column(&input, offset); + format!( + "offset {offset} line {line} col {column}, expected one of: {expected:?}" + ) + } + _ => format!("{err:?}"), + }; + println!("FAIL {} ({msg})", file.display()); + failed += 1; + } + } + + println!( + "{passed}/{}/{} files {}", + passed + failed, + files.len(), + if expect_reject { "rejected" } else { "parsed" } + ); + if failed == 0 { + ExitCode::SUCCESS + } else { + ExitCode::FAILURE + } +} + +fn walk_isc(dir: &Path) -> Vec { + let mut out = Vec::new(); + let mut stack = vec![dir.to_path_buf()]; + while let Some(dir) = stack.pop() { + let entries = match std::fs::read_dir(&dir) { + Ok(entries) => entries, + Err(_) => continue, + }; + for entry in entries.flatten() { + let path = entry.path(); + if path.is_dir() { + stack.push(path); + } else if path.extension().and_then(|e| e.to_str()) == Some("isc") { + out.push(path); + } + } + } + out +} + +fn line_column(input: &str, offset: usize) -> (usize, usize) { + let mut line = 1usize; + let mut column = 1usize; + for (index, ch) in input.char_indices() { + if index >= offset { + break; + } + if ch == '\n' { + line += 1; + column = 1; + } else { + column += 1; + } + } + (line, column) +} diff --git a/tools/isc-parg-check/tests/constructs/aliases-and-refs.isc b/tools/isc-parg-check/tests/constructs/aliases-and-refs.isc new file mode 100644 index 00000000..fc94c9ba --- /dev/null +++ b/tools/isc-parg-check/tests/constructs/aliases-and-refs.isc @@ -0,0 +1,11 @@ +system "T:aliases:x-Latn:1" { +aliases { +vowel = any(['a', 'e', 'i']) +cons = any("a".."z") +} +stage main { +parallel { +sub { from capture(vowel) + "-" to ref(1) + "-" after cons } +} +} +} diff --git a/tools/isc-parg-check/tests/constructs/metadata-extras.isc b/tools/isc-parg-check/tests/constructs/metadata-extras.isc new file mode 100644 index 00000000..93ef47e6 --- /dev/null +++ b/tools/isc-parg-check/tests/constructs/metadata-extras.isc @@ -0,0 +1,20 @@ +system "T:meta:x-Latn:1" { +metadata { +specification "ISO 9", "ISO 23951" +provenance "a", "b" +relations { +supersedes "OLD:x:y:1999" note "replaced" +basis_for "NEW:x:y:2020" +} +system_status current +code_status preferred +empty_field_example +} +dependency "other-sys" as other +stage main { +parallel { +sub "x" "y" +} +run map.other.stage.main +} +} diff --git a/tools/isc-parg-check/tests/constructs/sequence-block.isc b/tools/isc-parg-check/tests/constructs/sequence-block.isc new file mode 100644 index 00000000..19657053 --- /dev/null +++ b/tools/isc-parg-check/tests/constructs/sequence-block.isc @@ -0,0 +1,8 @@ +system "T:sequence:x-Latn:1" { +stage main { +sequence { +sub "a" "b" +sub { from "c" to "d" after "b" } +} +} +} diff --git a/tools/isc-parg-check/tests/constructs/stage-directives.isc b/tools/isc-parg-check/tests/constructs/stage-directives.isc new file mode 100644 index 00000000..59ef1da3 --- /dev/null +++ b/tools/isc-parg-check/tests/constructs/stage-directives.isc @@ -0,0 +1,11 @@ +system "T:directives:x-Latn:1" { +stage main { +parallel { +sub { from some("a") to ref(1) } +sub { from maybe("-") + "b" to "B" } +} +downcase +compose +separate separator "-" +} +} diff --git a/tools/isc-parg-check/tests/constructs/stage-paren-form.isc b/tools/isc-parg-check/tests/constructs/stage-paren-form.isc new file mode 100644 index 00000000..4bbf839d --- /dev/null +++ b/tools/isc-parg-check/tests/constructs/stage-paren-form.isc @@ -0,0 +1,7 @@ +system "T:stage-paren:x-Latn:1" { +stage(main) { +parallel { +sub "a" "b" +} +} +} diff --git a/tools/isc-parg-check/tests/constructs/title-case-kwarg.isc b/tools/isc-parg-check/tests/constructs/title-case-kwarg.isc new file mode 100644 index 00000000..8611ea43 --- /dev/null +++ b/tools/isc-parg-check/tests/constructs/title-case-kwarg.isc @@ -0,0 +1,5 @@ +system "T:titlecase:x-Latn:1" { +stage main { +title_case word_separator: "-" +} +} diff --git a/tools/isc-parg-check/tests/constructs/unicode-strings.isc b/tools/isc-parg-check/tests/constructs/unicode-strings.isc new file mode 100644 index 00000000..d9fc4f02 --- /dev/null +++ b/tools/isc-parg-check/tests/constructs/unicode-strings.isc @@ -0,0 +1,14 @@ +system "T:unicode:x-Hani:Latn:1" { +metadata { +notes { +note "CJK 中文 and emoji-adjacent ✅, quotes \" and \\ and 中" +note 'single quoted with # hash and "quotes"' +} +} +stage main { +parallel { +sub "中" "zhōng" +sub "\U0001F600" ":)" +} +} +} diff --git a/tools/isc-parg-check/tests/negative/stray-identifier.isc b/tools/isc-parg-check/tests/negative/stray-identifier.isc new file mode 100644 index 00000000..3b82188e --- /dev/null +++ b/tools/isc-parg-check/tests/negative/stray-identifier.isc @@ -0,0 +1,5 @@ +system "T:a-b:c-d:1" { +stage main { + oops +} +} diff --git a/tools/isc-parg-check/tests/negative/trailing-garbage.isc b/tools/isc-parg-check/tests/negative/trailing-garbage.isc new file mode 100644 index 00000000..919f9e01 --- /dev/null +++ b/tools/isc-parg-check/tests/negative/trailing-garbage.isc @@ -0,0 +1,3 @@ +system "T:a-b:c-d:1" { +} +trailing garbage diff --git a/tools/isc-parg-check/tests/negative/truncated-string.isc b/tools/isc-parg-check/tests/negative/truncated-string.isc new file mode 100644 index 00000000..500321d8 --- /dev/null +++ b/tools/isc-parg-check/tests/negative/truncated-string.isc @@ -0,0 +1,7 @@ +system "T:a-b:c-d:1" { +stage main { +parallel { +sub "a +} +} +} diff --git a/tools/isc-parg-check/tests/negative/unbalanced-brace.isc b/tools/isc-parg-check/tests/negative/unbalanced-brace.isc new file mode 100644 index 00000000..878ef711 --- /dev/null +++ b/tools/isc-parg-check/tests/negative/unbalanced-brace.isc @@ -0,0 +1,6 @@ +system "T:a-b:c-d:1" { +stage main { +parallel { +sub "a" "b" +} +} From e118fbd2975a67b1468b03ee52aad9c3d570e812 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Tue, 3 Nov 2026 11:12:15 +0800 Subject: [PATCH 3/3] =?UTF-8?q?docs(WO13):=20ISC-PARG-GAPS=20=E2=80=94=20e?= =?UTF-8?q?xpressiveness=20record=20for=20the=20.parg=20port?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit No concern construct proved inexpressible. The file records the one forced deviation (keyword alternation reordered longest-first for the parg shadow lint — semantics-preserving under negative lookahead), the omitted dead concern rules (authority/source_spelling/target_spelling/ identifying/name fields are unreachable from metadata_field), the field_value repeat-binding trap the corpus caught (din-hye), and the any-matches-newlines semantics multiline descriptions depend on. --- grammar/ISC-PARG-GAPS.md | 91 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 grammar/ISC-PARG-GAPS.md diff --git a/grammar/ISC-PARG-GAPS.md b/grammar/ISC-PARG-GAPS.md new file mode 100644 index 00000000..ce39515e --- /dev/null +++ b/grammar/ISC-PARG-GAPS.md @@ -0,0 +1,91 @@ +# ISC .parg port — expressiveness notes and gaps + +Status of the port: `grammar/isc.parg` expresses **every rule of the +canonical Ruby-DSL grammar** (`lib/interscript/isc/grammar/concerns/*.rb` ++ `core.rb`, origin/main lineage). No construct of the concerns proved +inexpressible in the parsanol .parg surface. What follows is the precise +record of the four places the port could not be a verbatim transcription +and why, plus the porting traps the corpus caught. Validation evidence: +tools/isc-parg-check parses 319/319 corpus files (318 `maps/` + 1 +`maps-staging/`) and rejects 4/4 negative fixtures, with verdict parity +against the Ruby DSL (`Interscript::Isc::Parser.parse_with_dsl`) on every +file, positive and negative. + +## 1. Keyword alternation: shadow-lint-forced reorder (deviation, safe) + +- Construct: `rule(:keyword)` in `concerns/items.rb` — the ordered + literal list of reserved words, used ONLY under `.absent?` (negative + lookahead) in `alias_reference` and `alias_arg`. +- Ruby spelling: `... | str("any") | ... | str("any_character")` — + `"any"` precedes `"any_character"`. +- Why the port deviates: the parsanol .parg compiler rejects + prefix-shadowed alternation branches at compile time ("branch 21 + ("any") shadows branch 34 ("any_character")"), because in a plain + choice the shorter literal would make the longer unreachable. +- Resolution: `"any_character"` moved before `"any"` (longest-first), + as the lint instructs. +- Why it is safe: `keyword` appears only under `!`. For a negative + lookahead, every branch ordering is equivalent — the predicate fails + whenever ANY branch matches at the position, and the two branch sets + are identical. No corpus input changes verdict under the reorder. + +## 2. Dead concern rules: omitted (not a .parg limitation) + +- Construct: `authority_field`, `source_spelling_field`, + `target_spelling_field`, `identifying_field`, `name_field` in + `concerns/metadata.rb`. +- Why omitted: none of them is reachable from `metadata_field` (they + are defined but never listed in its alternation). In the Ruby DSL + they are dead code; an `authority "X"` line parses via + `generic_field` (identifier + quoted value), same as here. +- Consequence: none for acceptance — a bare `authority "..."` / + `name "..."` metadata line parses identically through + `generic_field` on both engines (verified on the corpus, where + `authority_id`, `name`, `url`, ... all flow through the generic + branch). + +## 3. Unreachable lexical helpers: not carried over + +- `space?`, `newlines`, `newlines?` (`concerns/primitives.rb`) and + `list_item` (`concerns/items.rb`, an alias of `item`) are defined but + never referenced by a reachable rule. They are folded into their use + sites here (`wsq`, `list_items`). `braced(inner)` is a Ruby method, + not a rule; every use site spells the scaffold inline. + +## 4. The `field_value` repeat-binding trap (expressed; first port got it wrong) + +- Construct: `rule(:field_value)` in `concerns/metadata.rb`: + `(newline.absent? >> (str("}").absent? >> str("{").absent? >> any)).repeat(1)`. +- Trap: the `.repeat(1)` binds over the WHOLE parenthesized group, so + `newline.absent?` is re-checked per iteration — the raw value run + stops AT the newline (`din-hye`: `authority_id din` captures `raw: + "din"`, not the whole metadata block). +- Port: `field_raw = 1*(!newline !"}" !"{" chr)`. The first draft wrote + `!newline` outside the repetition; the run then swallowed fields up + to the next `{` and `din-hye-Armn-Latn-32706-2010.isc` failed. The + harness caught it; the per-iteration guard matches the Ruby + semantics. + +## 5. Related semantics fixed while porting (recorded for reviewers) + +- The DSL's `any` matches ANY character including raw newlines — + multiline `description { ... }` bodies (raw_text) and quoted strings + cross line breaks. `chr = %x00-10FFFF` here; newline stops exist + ONLY where the concerns put an explicit `newline.absent?` + (`line_comment`, `field_raw`, stages' `comment_item`). +- Escape rulings from the concerns are mirrored: `\uXXXX` exactly 4 + hex; `\U` 4..8 hex (`gki-bel "\U040E"`); a lone `\U` accepted + (un-mar converter corruption). Acceptance parity holds on all three + corpus shapes. +- The root rule pins full-input consumption with an explicit `!utf` + (`isc_source`), so the harness rejects trailing garbage without + relying on engine EOF behavior. + +## Not a gap, but out of harness scope + +The harness gates ACCEPTANCE (parse/reject verdicts). The capture-tree +SHAPE produced by a .parg artifact is parg-shaped (`cap` nodes); the +gem's `Isc::Normalizer` owns DSL-tree equivalence for the artifact +runtime. Verifying shape parity byte-for-byte against +`parse_with_dsl` trees is the gem's corpus gate's job, not this +harness's.