Skip to content

Characterize how usage reporting reads a mis-shaped settings block - #71

Merged
dmccoystephenson merged 1 commit into
mainfrom
feature/characterize-settings-value-shapes
Nov 16, 2026
Merged

dmccoystephenson merged 1 commit into
mainfrom
feature/characterize-settings-value-shapes

Conversation

@dmccoystephenson

Copy link
Copy Markdown
Member

Summary

Stage B (unit-test expansion) cycle for usage_reporting.py. It covers two ways settings.json can hold a usage_reporting block of the wrong shape. Neither was tested or documented before. Production code is not changed.

  • test_a_block_that_is_not_an_object_is_replaced_by_the_defaults asserts what happens when usage_reporting is false, a string, a list or null. The value is overwritten with the default block, with reporting on. The first-run notice is shown again, and other entries in the file are kept.
  • test_enabled_is_read_by_truthiness asserts that enabled goes through bool(). "false", "off" and 1 leave reporting on, while 0, "" and null turn it off with disabled_reason == "config". A missing enabled leaves reporting on.
  • The README's Usage reporting section gains one paragraph that describes both behaviours, so a reader who wants to opt out writes the JSON value false.

Both behaviours turn an attempted opt-out into reporting, so they were filed as #70 for a decision rather than changed in this test cycle. This PR does not close #70.

Test plan

  • python3 -m py_compile *.py
  • python3 -m unittest discover -s tests: 216 tests ran, OK (local, Python 3.8)
  • Mutation check: isinstance(section, dict) was changed to section is not None, and bool(enabled) to enabled is True. Six subtests failed across the two new tests. With the source restored they pass, and usage_reporting.py has no diff.
  • Tests workflow is green on this head

Manual validation of the render path is not applicable, since neither main.py nor any rendering or Viron-service code is touched.

Issues not picked this cycle

This PR description was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

🤖 Generated with Claude Code

A usage_reporting value that is not an object is replaced by the
default block with reporting on, and enabled is read by truthiness,
so the string "false" leaves reporting on. Both are now asserted by
tests and described in the README; the behaviour itself is tracked
in #70.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dmccoystephenson

Copy link
Copy Markdown
Member Author

Self-review rubric (head d3c5722):

  • Scope: PASS. Only README.md (+6) and tests/test_usage_reporting.py (+38) change. No production code is touched, as Stage B requires.
  • Tests-new: PASS. No new public functions. The two new tests exercise existing loadSettings and buildClient paths that had no test.
  • Tests-fix: not applicable, since no bug is fixed here. As a substitute, a mutation check was run. isinstance(section, dict) was changed to section is not None, and bool(enabled) to enabled is True. Six subtests failed, and all passed again after the restore.
  • Sibling structure: PASS. The new tests follow the TestUsageReportingSettings conventions: self.settingsFile, self.log, readSettingsFile(), subTest per case, and client.close() with the existing "nothing was reported" note.
  • Sibling renames: not applicable, since nothing is renamed.
  • Docs: PASS. The README Usage reporting section describes both behaviours. The Running the tests section already lists usage_reporting.py among the modules tested without Pygame, so it needs no change. The batch scripts are unaffected.
  • Issue resolution: PASS. No Closes line is given. The behaviours are tracked in An enabled value of "false" or a usage_reporting value of false leaves usage reporting on #70, which stays open for a decision.
  • Manual validation: PASS. The Tests workflow is green on d3c5722, with 216 tests run locally. The render and Viron-service path is untouched, so the main.py check is not applicable.

Findings to note, not fix:

  • README.md:360 says a non-empty string such as "false" "leaves reporting on". That holds only when neither TRACE_USAGE_REPORTING nor DO_NOT_TRACK opts out. The paragraph after it already says the environment variables win over settings.json, so no change was made.
  • tests/test_usage_reporting.py: the missing case in test_enabled_is_read_by_truthiness overlaps with test_missing_endpoint_and_key_fall_back_to_the_shipped_defaults. It is kept so that the truthiness table is complete in one place.

This review was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

@dmccoystephenson
dmccoystephenson merged commit bb5f7f4 into main Nov 16, 2026
1 check passed
@dmccoystephenson
dmccoystephenson deleted the feature/characterize-settings-value-shapes branch November 16, 2026 06:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

An enabled value of "false" or a usage_reporting value of false leaves usage reporting on

1 participant