Repository navigation
fix(symbolic): stop accepting near-miss numbers (PIE-905) - #76
Merged
Merged
Conversation
The 355/113 approximation of pi put trig results off by about 1e-7, which the symbolic comparison hid by rounding to 4 decimals.
Constant results were rounded to 4 decimals, so any number within 0.00005 of the answer was scored correct. Compare constants with a 1e-12 relative tolerance instead, which only absorbs floating-point noise from trig functions and pi.
Re-enable the commented-out ne assertions in math-evaluator.
|
🎉 This PR is included in version 1.4.3 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
In symbolic mode, a numeric answer within 0.00005 of the correct answer (for example
800.00001for800) was scored correct. Now a numeric answer must match the correct one to about 13 significant digits, which only allows for floating-point noise.Jira: https://illuminate.atlassian.net/browse/PIE-905
Why it happened
At the end of
normalize()insrc/symbolic/index.ts, every constant result was rounded withMath.round(v * 10000) / 10000, so both sides collapsed to the same 4-decimal value. The rounding was masking imprecise trig results. Most of that imprecision came from the degree and gradian conversion inlatex-to-ast.ts, which used355/113for π. That put values likecos(90°)off by about 1.3e-7, which is far more than floating-point noise.Change
latex-to-ast.ts: degrees and gradians now convert withMath.PI / 180andMath.PI / 200instead of355/113.symbolic/index.ts: removed the 4-decimal rounding. When both sides normalize to an expression with no variables,isMathEqualevaluates both and accepts them when|a − b| < 1e-14 + 1e-13 · max(|a|, |b|).Why these tolerances: with exact π, the remaining error is floating-point noise, and it grows with the size of the angle. For example,
sin(2π) = -2.4e-16,sin(200π) = 3.9e-15andtan(36045°) − 1 = -5.4e-14. The relative part (1e-13) covers that up to about 100 full turns. The absolute floor (1e-14) is there for comparisons with 0, where a relative tolerance alone would require an exact match. It is kept small so that values below 1 need as many matching digits as larger ones:\frac{1}{3}vs0.333333333333(twelve 3s) and0vs0.0000000000001are rejected.A tolerance wide enough to hide the old
355/113error (about 1e-7) would still have accepted800.00001, whose relative difference is 1.25e-8, so the conversion constant had to be fixed too. Equations without variables (for example4=4) evaluate to booleans and are excluded from the numeric comparison. They keep their existing path.The
Infinitysentinel (±1.497258191621251e6, used fortan(90°)) is unchanged here. With exact π,tan(90°)is about 1.6e16, which is still above the threshold.Scoring impact
Answers that were previously scored correct and are now scored incorrect:
800vs800.00001,799.99999,800.0001-800vs-800.00001,-799.99999100vs100.00000110^2vs100.000011000000vs1000000.0001\sqrt{2}vs1.4142and1.41421356237, and\frac{\sqrt{3}}{2}or\sin(60°)vs0.866. This was already inconsistent before:0.8660was scored incorrect while0.866was scored correct. If authors rely on rounded decimals being accepted for irrational answers, those items need explicit alternate answers. The same applies to-\sqrt{2}vs-1.4142.10^{-10}vs00vs\sin(1000\pi). Mathematically this is a correct answer, but the floating-point residue of a 500-turn angle (3.2e-13) is above the tolerance. It's unlikely in practice.Answers that were previously scored incorrect and are now scored correct (the
355/113conversion was off by more than the 4-decimal rounding could hide):30°vs\arcsin(0.5)\frac{\pi}{2}vs90°and100\grad\pivs180°1000vs1000\tan(45°)developrounded differently from the exact value, for example\pivs3.14159265358979,2\pivs6.28318530717959and\pi^2vs9.869604401089362\pivs360°and\frac{\pi}{3}vs60°0vs\sin(36000°)and1vs\tan(36045°)Answers whose scoring is unchanged: exact trig values in degrees, gradians and radians (for example
\sin(30°)vs\frac{1}{2},\tan(60°)vs\sqrt{3},\sin(2\pi)and\cos(270°)vs0),0.1+0.2vs0.3,\tan(90°)vs\infty, trailing zeros (800.00), and equations without variables. Literal mode is unaffected.Tests
develop): 1546 passed. After: 1597 passed (+47 new fixtures insrc/fixtures/latex-equal/symbolic/PIE-905.ts, +4 re-enabled).tsc --noEmitis clean.src/symbolicandsrc/conversionfromdevelop, exactly the 16 changed-answer cases above fail.src/fixtures/latex-equal/symbolic/math-evaluator.ts: re-enabled the commented-outnefor target100(100.000001,44 + 57,50 * 3,25 * 2 + 51).src/conversion/__tests__/latex-to-ast.spec.ts: the expected degree and gradian constants changed from355/113-based toMath.PI-based, matching the implementation. This changes the names of 7 tests, because the names include the constant.eqfixture regressed.Follow-up
docs/) to be regenerated after merge.Infinity, so2000000vs3000000and-2000000vs2000000are scored correct in symbolic mode. fix(symbolic): stop treating large numbers as infinity (PIE-1418) #78 (PIE-1418) removes that cutoff. Both PRs add helper functions in the same place inindex.ts, so whichever merges second has a small textual conflict: keep both sides. With both merged, all 1681 tests pass (fix(symbolic): stop treating large numbers as infinity (PIE-1418) #78's fixtures no longer depend on the 4-decimal rounding this PR removes).6.626\times10^{-34}vs1\times10^{-34}(developalso scores this correct, because both round to 0). Science items in scientific notation would need a purely relative comparison when neither side is 0. For comparisons with 0, the headroom for trig noise is the 1e-14 floor, so0vs\cos(36090°)(residue 5.8e-14) is rejected.x\sin(30°)vs0.5xis still scored incorrect, andy=x\sin(30°)vsy=\frac{x}{2}still throws a Fraction conversion error.