Skip to content

Add sequence length guardrail - #51

Merged
rayokota merged 3 commits into
masterfrom
sequence-guardrail
Nov 13, 2026
Merged

rayokota merged 3 commits into
masterfrom
sequence-guardrail

Conversation

@rayokota

Copy link
Copy Markdown
Owner

Port the excessive sequence length guardrail from the JS reference implementation: a new optional sequence option caps the length of any sequence created during evaluation, including intermediate sequences, raising D2015 when exceeded. The range operator and $append also check the size of their result up front. Also update the D2014 message to match the 1e7 limit that is already enforced.

Port the excessive sequence length guardrail from the JS reference
implementation: a new optional `sequence` option caps the length of any
sequence created during evaluation, including intermediate sequences,
raising D2015 when exceeded. The range operator and $append also check
the size of their result up front. Also update the D2014 message to
match the 1e7 limit that is already enforced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Sequence-limit enforcement remains incorrect: wildcard results can bypass the limit after a bounded accumulator becomes an unbounded list, while an off-by-one range length rejects valid exact-limit results.

2 open issues
What changed in this PR

Adds an optional sequence-length guardrail to the Python JSONata evaluator, extending its protections for untrusted expressions.

Changes:

  • Introduces the sequence option, bounded sequence construction, and limit restoration after evaluation.
  • Adds upfront range and $append checks, introduces D2015, and corrects D2014’s documented limit to 1e7.
  • Documents the option and adds tests for limits, intermediate sequences, $eval, and restoration.
File Description
tests/​string_test.py Adds sequence guardrail regression tests.
src/​jsonata/​utils.py Adds thread-local limits and bounded sequence lists.
src/​jsonata/​jsonata.py Exposes the option and applies it during evaluation.
src/​jsonata/​jexception.py Adds D2015 and corrects D2014’s message.
src/​jsonata/​functions.py Checks $append result size before copying.
README.md Documents the sequence-length guardrail.

Review effort: Balanced

Comment thread src/jsonata/functions.py Resolved
Comment thread src/jsonata/functions.py Resolved
Address review comments:
- $append returned an unbounded copy, so callers that keep growing the
  result, such as evaluate_wildcard, bypassed the max sequence length.
- RangeList reported one more item than it holds, so a range of exactly
  the max sequence length raised D2015.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Sequence-limit errors report the attempted length instead of the configured maximum, and returned lists retain bounds that incorrectly restrict caller mutations after evaluation.

1 open issue
1 previously missed issue

In code that hasn't changed since last review

Medium severity Report the configured maximum in D2015 errors

src/​jsonata/​utils.py:129

This passes the attempted length into D2015, but its message describes that value as the configured maximum. For example, [0..1001] with sequence=1000 reports “The maximum sequence length of 1002 was exceeded,” and $append has the same problem. Pass max_length instead so the message and JException.current agree with the configured limit and with BoundedJList._check.

2 resolved since last review

Review effort: Balanced

Comment thread src/jsonata/utils.py Resolved
Address review comment: a BoundedJList kept enforcing the max sequence
length it was created with, so a returned result raised D2015 when the
caller grew it after evaluate() finished. It now checks the max sequence
length of the evaluation in progress on the thread, if any.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The sequence limit is checked only after wildcard flattening copies the entire array, so intermediate allocations can still exhaust memory.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Medium severity Bound flattening accumulator to prevent oversized intermediate allocations

src/​jsonata/​functions.py:2055

This check runs after evaluate_wildcard has flattened each array into an ordinary list (src/jsonata/jsonata.py:709–710, 717–718). With sequence=2, a large input array is therefore copied in full before D2015 is raised, so the intermediate allocation can still exhaust memory. Make the accumulator in Jsonata.flatten bounded so it checks the limit as it grows.

1 resolved since last review

Review effort: Balanced

@rayokota
rayokota merged commit 9691a6b into master Nov 13, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants