Skip to content

fix(symbolic): stop accepting near-miss numbers (PIE-905) - #76

Merged
CarlaCostea merged 6 commits into
developfrom
fix/PIE-905
Nov 4, 2026
Merged

CarlaCostea merged 6 commits into
developfrom
fix/PIE-905

Conversation

@CarlaCostea

@CarlaCostea CarlaCostea commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

What

In symbolic mode, a numeric answer within 0.00005 of the correct answer (for example 800.00001 for 800) was scored correct. Now a numeric answer must match the correct one to about 13 significant digits, which only allows for floating-point noise.

Jira: https://illuminate.atlassian.net/browse/PIE-905

Why it happened

At the end of normalize() in src/symbolic/index.ts, every constant result was rounded with Math.round(v * 10000) / 10000, so both sides collapsed to the same 4-decimal value. The rounding was masking imprecise trig results. Most of that imprecision came from the degree and gradian conversion in latex-to-ast.ts, which used 355/113 for π. That put values like cos(90°) off by about 1.3e-7, which is far more than floating-point noise.

Change

  • latex-to-ast.ts: degrees and gradians now convert with Math.PI / 180 and Math.PI / 200 instead of 355/113.
  • symbolic/index.ts: removed the 4-decimal rounding. When both sides normalize to an expression with no variables, isMathEqual evaluates both and accepts them when |a − b| < 1e-14 + 1e-13 · max(|a|, |b|).

Why these tolerances: with exact π, the remaining error is floating-point noise, and it grows with the size of the angle. For example, sin(2π) = -2.4e-16, sin(200π) = 3.9e-15 and tan(36045°) − 1 = -5.4e-14. The relative part (1e-13) covers that up to about 100 full turns. The absolute floor (1e-14) is there for comparisons with 0, where a relative tolerance alone would require an exact match. It is kept small so that values below 1 need as many matching digits as larger ones: \frac{1}{3} vs 0.333333333333 (twelve 3s) and 0 vs 0.0000000000001 are rejected.

A tolerance wide enough to hide the old 355/113 error (about 1e-7) would still have accepted 800.00001, whose relative difference is 1.25e-8, so the conversion constant had to be fixed too. Equations without variables (for example 4=4) evaluate to booleans and are excluded from the numeric comparison. They keep their existing path.

The Infinity sentinel (±1.497258191621251e6, used for tan(90°)) is unchanged here. With exact π, tan(90°) is about 1.6e16, which is still above the threshold.

Scoring impact

Answers that were previously scored correct and are now scored incorrect:

  • 800 vs 800.00001, 799.99999, 800.0001
  • -800 vs -800.00001, -799.99999
  • 100 vs 100.000001
  • 10^2 vs 100.00001
  • 1000000 vs 1000000.0001
  • Decimal approximations of irrational values that happened to round to the same 4 decimals, for example \sqrt{2} vs 1.4142 and 1.41421356237, and \frac{\sqrt{3}}{2} or \sin(60°) vs 0.866. This was already inconsistent before: 0.8660 was scored incorrect while 0.866 was scored correct. If authors rely on rounded decimals being accepted for irrational answers, those items need explicit alternate answers. The same applies to -\sqrt{2} vs -1.4142.
  • Small values that the rounding collapsed to 0, for example 10^{-10} vs 0
  • 0 vs \sin(1000\pi). Mathematically this is a correct answer, but the floating-point residue of a 500-turn angle (3.2e-13) is above the tolerance. It's unlikely in practice.

Answers that were previously scored incorrect and are now scored correct (the 355/113 conversion was off by more than the 4-decimal rounding could hide):

  • 30° vs \arcsin(0.5)
  • \frac{\pi}{2} vs 90° and 100\grad
  • \pi vs 180°
  • 1000 vs 1000\tan(45°)
  • Longer decimal approximations of π, which develop rounded differently from the exact value, for example \pi vs 3.14159265358979, 2\pi vs 6.28318530717959 and \pi^2 vs 9.86960440108936
  • Angles in radians vs degrees, for example 2\pi vs 360° and \frac{\pi}{3} vs 60°
  • Exact values of large angles, for example 0 vs \sin(36000°) and 1 vs \tan(36045°)

Answers whose scoring is unchanged: exact trig values in degrees, gradians and radians (for example \sin(30°) vs \frac{1}{2}, \tan(60°) vs \sqrt{3}, \sin(2\pi) and \cos(270°) vs 0), 0.1+0.2 vs 0.3, \tan(90°) vs \infty, trailing zeros (800.00), and equations without variables. Literal mode is unaffected.

Tests

  • Before (develop): 1546 passed. After: 1597 passed (+47 new fixtures in src/fixtures/latex-equal/symbolic/PIE-905.ts, +4 re-enabled). tsc --noEmit is clean.
  • With src/symbolic and src/conversion from develop, exactly the 16 changed-answer cases above fail.
  • Re-enabled / changed fixtures:
    • src/fixtures/latex-equal/symbolic/math-evaluator.ts: re-enabled the commented-out ne for target 100 (100.000001, 44 + 57, 50 * 3, 25 * 2 + 51).
    • src/conversion/__tests__/latex-to-ast.spec.ts: the expected degree and gradian constants changed from 355/113-based to Math.PI-based, matching the implementation. This changes the names of 7 tests, because the names include the constant.
  • No existing eq fixture regressed.

Follow-up

  • Demo page (docs/) to be regenerated after merge.
  • Not fixed here: any constant at or above 1,497,258 normalizes to Infinity, so 2000000 vs 3000000 and -2000000 vs 2000000 are scored correct in symbolic mode. fix(symbolic): stop treating large numbers as infinity (PIE-1418) #78 (PIE-1418) removes that cutoff. Both PRs add helper functions in the same place in index.ts, so whichever merges second has a small textual conflict: keep both sides. With both merged, all 1681 tests pass (fix(symbolic): stop treating large numbers as infinity (PIE-1418) #78's fixtures no longer depend on the 4-decimal rounding this PR removes).
  • Not fixed here (pre-existing): values below about 1e-14 are compared only by the absolute floor. So two very small values count as equal, for example 6.626\times10^{-34} vs 1\times10^{-34} (develop also scores this correct, because both round to 0). Science items in scientific notation would need a purely relative comparison when neither side is 0. For comparisons with 0, the headroom for trig noise is the 1e-14 floor, so 0 vs \cos(36090°) (residue 5.8e-14) is rejected.
  • Not fixed here (pre-existing): the numeric comparison only runs when neither side has variables, so x\sin(30°) vs 0.5x is still scored incorrect, and y=x\sin(30°) vs y=\frac{x}{2} still throws a Fraction conversion error.

The 355/113 approximation of pi put trig results off by about 1e-7,
which the symbolic comparison hid by rounding to 4 decimals.
Constant results were rounded to 4 decimals, so any number within
0.00005 of the answer was scored correct. Compare constants with a
1e-12 relative tolerance instead, which only absorbs floating-point
noise from trig functions and pi.
Re-enable the commented-out ne assertions in math-evaluator.
@CarlaCostea
CarlaCostea changed the base branch from master to develop November 3, 2026 13:38
@CarlaCostea
CarlaCostea merged commit ac236ce into develop Nov 4, 2026
4 checks passed
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 1.4.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant