Skip to content

Docs/0.2.0 revision - #14

Merged
Adam Corrao (AdamCorrao) merged 18 commits into
NSLS2:mainfrom
AdamCorrao:docs/0.2.0-revision
Nov 6, 2026
Merged

Adam Corrao (AdamCorrao) merged 18 commits into
NSLS2:mainfrom
AdamCorrao:docs/0.2.0-revision

Conversation

@AdamCorrao

Copy link
Copy Markdown
Collaborator

docs: rebuild the documentation around its readers (0.2.0)

This PR rewrites the PowderLine documentation for the 0.2.0 release. It
changes no behaviour. Outside docs/, it changes only:

  • message, comment and docstring wording;
  • test assertions that follow that wording;
  • docs tooling (theme, generators, checks, CI steps).

Why

The 0.2.0 docs were accurate and thorough, but hard to use:

  • there was no front door;
  • the same facts were written out on several pages, each slightly
    differently;
  • user pages mixed in maintainer process and 0.1.x history;
  • engine-behaviour IDs (EB-NN) could not be looked up.

The new docs give a mental model up front, one page per task, and one home
per fact. They describe the current release only.

What changed

Structure. The docs are reorganised by reader and page type, in MyST
Markdown:

Section Holds
Get started Install, Your first refinement (LaB6 end to end), Refine your own data
Concepts How PowderLine works, Recipes, Validation, Results, Good refinement practice, Comparing engines
How-to guides Calibrate the instrument, Refine a structure, Model disorder near special positions, Fit single peaks, Simulate, Run a series, Python and pipelines, Run at a beamline, Run TOPAS recipes, Troubleshoot
Examples A gallery with a fit figure per example; ill-posed examples labelled as teaching examples
Engines Overview (features, scattering types and special positions by engine), GSAS-II, TOPAS, easydiffraction, with every engine behaviour anchored
Reference Recipe format, Fields, Outputs, Messages, Command line, Python API, Versions, Scope, Glossary, ID index, Known issues
Development Architecture, Contributing, Adding an engine, Policies, Internals API, Migrating from 0.1
About Citing, Changelog

Front door. The README and the landing page now say:

  • what PowderLine is and why it exists;
  • that a recipe is structured data, written for one engine in its own terms;
  • that results come back uniform.

Both show a diagram of that path.

Look-up-able IDs. Every EB-NN and KI-NN has its own anchored entry
and appears in an ID index. The entries are generated from a public catalogue
(docs/catalog/*.yaml) by scripts/gen_catalog.py. All 88 behaviours of the
old page are kept, and EB-04 is added. Three new known issues cover the
GSAS-II server:

  • KI-19: the server keeps running until it is stopped;
  • KI-20: its state directory must be on the computer's own disk;
  • KI-21: a narrow race in gsas-server stop.

Facts corrected while checking every page against the code:

  • The special-position rule is stated as the distance between an atom and
    its symmetry image, which is what validation measures. The validation
    error message now says so too.
  • GSAS-II's own default background is fixed. The warning text said it would
    be refined.
  • TOPAS's TCHZ Z is in degree², like U, V and W.
  • Result keys differ slightly per engine: easydiffraction has no spf_*
    keys, and peak_fit is GSAS-II only.
  • Two cases of what an output directory keeps between runs are documented.
  • The leftover-server cleanup lists processes and ends them by PID. It no
    longer suggests pkill -f, whose pattern also matches an editor open on
    server.py.
  • The teaching-example descriptions now match their committed outputs.

Tooling.

  • The theme moves from sphinx_rtd_theme to pydata-sphinx-theme, with
    sphinx-design (tabs, cards), sphinxcontrib-mermaid (diagrams) and
    sphinx-copybutton.
  • pixi.lock changes only by these packages and their dependencies, plus
    the removal of the old theme.
  • New pixi tasks run in CI beside fields-check:
    • catalog-check: the catalogue fragments are current, every
      EB-/KI- ID cited anywhere in the repository has an entry, and every
      "what you see" item is a real warning code or failure reason;
    • figures-check: the example figures match the committed fit profiles.
  • The cross-engine comparison is regenerated under 0.2.0. Every number
    reproduces.

Scope checks

  • Frozen GSAS-II modules. setters.py, constraints.py and
    extractors.py changed in comments and docstrings only.
    _code_hash.json is regenerated, and test_code_hash passes.

  • Runtime text. Only message strings changed:

    • CLI hints name pixi run -e gsasii powderline …;
    • the install command carries --frozen;
    • server hints name gsas-server stop and -e gsasii;
    • pixi reinstall powderline --all;
    • the special-position and background-default wording.

    Tests that assert the wording follow it.

  • No logic, schema, default or reference (examples/**/output/) changed.

Where each old page went

Old page New home
index.rst, quickstart.md index.md, getting-started/first-refinement.md
getting-started.rst getting-started/install.md, concepts/recipes.md, reference/outputs.md
schemas.md engines/index.md, engines/*.md, reference/outputs.md, reference/messages.md
fields.md reference/fields.md (generated tables), conventions in reference/recipe-format.md
input-guidance.md concepts/validation.md, how-to/disorder.md, engines/index.md, getting-started/your-own-data.md, reference/scope.md
engine-behavior.md engines/*.md (generated entries), reference/ids.md, warnings table in reference/messages.md
cross-engine-comparison.md concepts/comparing-engines.md
TROUBLESHOOTING.md how-to/troubleshooting.md; contributor items in development/contributing.md
integration-guide.md how-to/python-pipelines.md, how-to/sequential-refinement.md
known_issues.md reference/known-issues.md (generated entries)
policies.md, engine-versions.md development/policies.md
DEVELOPMENT.md development/architecture.md, development/contributing.md, development/adding-an-engine.md
SCHEMA_HISTORY.md, support-matrix.md reference/versions.md
migration-0.1-to-0.2.md development/migrating-from-0.1.md
api.rst reference/api.md, development/internals-api.md, reference/cli.md
citing.md, changelog.md about/
regression-tolerance.md, cross-platform-guide.md Removed: internal studies. The rules contributors need are in development/contributing.md

There are no redirects. Links into the old page names will break; the site
root URL is unchanged.

How it was checked

  • pixi run docs (sphinx-build -W --keep-going) builds with no warnings.
  • fields-check, catalog-check and figures-check pass.
  • Engine-free suite (pytest -m "core or topas_gen"): 1848 passed and 23
    skipped. That is identical to v0.2.0 on the same machine.
  • A real GSAS-II LaB6 run (--no-server) gives Rwp 6.534 %, as before.
  • Every command, flag, key, warning code, failure reason, unit and quoted
    number was checked against the code and the committed outputs. Snippets
    were run where possible. TOPAS execution and Windows were checked by
    reading the code only.
  • The built HTML was scanned for internal tracking material.

For reviewers

  • 4d4e69e is the large rebuild commit. Git keeps file history only for
    changelog.md and citing.md; the table above maps every old page to its
    new home.
  • The CHANGELOG records the docs rebuild and the message wording in the
    0.2.0 section: the v0.2.0 tag moves to the merge commit of this PR.

Add sphinx-design (cards, tabs, badges), sphinxcontrib-mermaid (diagrams
written as text) and sphinx-copybutton to the dev feature; drop
sphinx_rtd_theme. conf.py: the new theme, MyST-only sources, mermaid
fences rendered as diagrams, and comments without internal tracking IDs.

Assisted-by: claude-code:claude-opus-5-5
docs/catalog/*.yaml holds the public facts of every engine behaviour
(EB-NN) and known issue (KI-NN); scripts/gen_catalog.py renders the
per-engine entries, the ID index and the known-issues page, each ID with
its own anchor. scripts/gen_example_figures.py draws deterministic SVG
fit plots from the committed fit profiles for the examples gallery.
New pixi tasks catalog-check and figures-check run in CI beside
fields-check.

Assisted-by: claude-code:claude-opus-5-5
New structure: Get started (tutorials), Concepts, How-to guides,
Examples (gallery with labelled teaching examples), Engines (per-engine
reference with every EB entry anchored), Reference, Development and
About. Pages are MyST Markdown, describe the current release only, and
keep one home per fact. Material that was internal process or history
(the regression-tolerance study, the cross-platform analysis, the
unified-schema archive, known-issue disposition rounds) leaves the public
docs. README and landing page lead with what PowderLine is and why;
CONTRIBUTING and AGENTS point at the new pages; CHANGELOG links follow
the moves. The two tests that read docs pages read the new ones.

Assisted-by: claude-code:claude-opus-5-5
Comment and docstring edits only: paths to moved docs pages, and
internal tracking IDs removed from the published run() and
recipes.block() docstrings. The two comments in the frozen GSAS-II
modules change only their docs path (no behaviour change);
_code_hash.json is regenerated for them.

Assisted-by: claude-code:claude-opus-5-5
- every command that runs a committed example writes to --output, so
  the regression references under examples/**/output stay untouched
- gsas-server start/restart run in the foreground: stale-server fixes use
  stop; execution modes have one home (Run at a beamline)
- negative Uiso, simulation_mode, generate-only files, console_tail,
  CLI exit status and default-warning counts match the code
- restored: data and fit-window rules, tie-bounds mapping, regression
  tolerances, GSASClient use, server security facts, leftover pre-0.1.2
  servers, simulation builder notes
- one rule for moving a structure between engines (recipe format);
  known issues move to Reference; engine pages drop upstream status
- comparing engines rewritten as tables for scientists; scope line on
  README and the landing page

Assisted-by: claude-code:claude-opus-5-5
Comment-only edits in gateways/gsasii/{constraints,setters}.py (no
behaviour change); _code_hash.json regenerated.

Assisted-by: claude-code:claude-opus-5-5
- migrating-from-0.1: list only the user's own Python server processes
  and end them by PID (the former pkill pattern also matched editors and,
  on Windows, other users' processes)
- outputs: converged is None for every simulation
- beamline-operation: the in-process fallback shows in result['method']
  and the CLI line; early refusal applies to malformed signing headers
- the simulation examples' run commands write to --output

Assisted-by: claude-code:claude-opus-5-5
Message text and docstrings only, no logic: the CLI's hints use
'pixi run -e gsasii powderline ... --output' and 'add --verbose';
EngineNotAvailableError's install command carries --frozen; stale and
unauthenticated server messages recommend 'gsas-server stop' (start and
restart hold the terminal) instead of restart; the registry and doctor
say 'pixi reinstall powderline --all'; server.py's usage notes give the
-e gsasii environment and say start runs in the foreground; the
simulation builder's docstring names its 'sites' key. Tests asserting
the old wording follow it.

Assisted-by: claude-code:claude-opus-5-5
Install page, README and engine pages say PowderLine is tested on Linux
and Windows and expected to work on macOS. The simulation examples'
DESCRIPTIONs name the beamline 28-ID-1, matching their recipes.
CHANGELOG [Unreleased] records the documentation rebuild and the
message changes.

Assisted-by: claude-code:claude-opus-5-5
Message and comment text only, no logic:
- a special-position error reports the distance to a symmetry image
  as such, not as a distance from the site;
- the GSAS-II background-default warning says GSAS-II's own default
  background is fixed;
- server and client hints name `pixi run -e gsasii gsas-server start`;
  the template hint copies into a folder of your own;
- test_code_hash states the frozen-code rule (comment edits need no
  consent) instead of a private record;
- docstrings published by the API page drop history wording, and the
  modules whose source the docs site shows, the two scripts' --help
  text and an example script drop internal tracking ids.
The two frozen modules change in comments only; _code_hash.json is
regenerated. The symmetry test follows the new message.

Assisted-by: claude-code:claude-opus-5-5
- Safety: the leftover-server cleanup lists processes and ends them by
  PID instead of `pkill -f`; TMPDIR stays on the local disk; the
  server's lifetime and shared logins are stated; new KI-19 to KI-21.
- Facts: the special-position measure (distance to a symmetry image),
  the TOPAS TCHZ Z unit, GSAS-II's fixed default background, the
  CrysPy Rwp comparison, result keys per engine, what an output
  directory keeps, the method values, console_tail for TOPAS.
- Catalogue: EB-84/EB-80, EB-46/EB-50 and EB-47/EB-74 hold their own
  facts; editorial notes removed; EB-33/EB-77 list what you see.
- Pages: one recipe, one engine on the README and landing page; the
  preprint is cited without a journal claim; the migrating page lists
  the import breaks and typical converter stops; CI and review text
  match the workflow; the calibration page explains GSAS-II's width
  floor on the LaB6 example (EB-93).
- Examples: teaching descriptions match their outputs; contributor
  guidance reports Rwp with Rexp and GoF and links refinement
  practice; the strainonly command no longer overwrites its reference;
  SRM 660c for the textured example.

Assisted-by: claude-code:claude-opus-5-5
catalog-check scans examples/, scripts/, tests/ and the top-level
Markdown files as well as src/ and docs/, scans the generated files it
does not write, fails on an eb_*.md fragment it does not write, and
checks every you_see item against the warning codes and failure
reasons. Both generators name the runnable regenerate command and
print repository-relative paths.

Assisted-by: claude-code:claude-opus-5-5
Every number reproduces. The DRX_33 GSAS-II row gains the
output_recipe_not_valid warning (KI-15), which the page now explains,
and the footer names PowderLine 0.2.0.

Assisted-by: claude-code:claude-opus-5-5
The v0.2.0 tag moves to the merge commit of this branch, so its
documentation and message-wording entries belong to 0.2.0. The
Unreleased section stays, empty, for the next change.

Assisted-by: claude-code:claude-opus-5-5
Copilot AI balanced review requested due to automatic review settings November 6, 2026 00:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The provenance documentation incorrectly implies a traversable run history and needs to clarify that each run replaces the block, so full pipeline tracing requires archived inputs, results, and upstream run associations.

1 open issue
What changed in this PR

Reorganizes PowderLine’s 0.2.0 documentation around readers and tasks, explaining how engine-specific recipes produce standardized results. Runtime changes are limited to message wording, with corresponding test updates.

Changes:

  • Adds task-focused tutorials, concepts, how-to guides, and reference pages.
  • Updates examples, engine-behaviour references, and contributor guidance.
  • Refreshes documentation dependencies and adds catalogue and figure checks to CI.
File Description
tests/​test_warning_codes.py Points warning checks to the new reference page.
tests/​test_symmetry_core.py Updates the expected symmetry error wording.
tests/​test_gsas_client_visibility.py Expects revised stale-server guidance.
tests/​test_gateways.py Expects frozen-install command hints.
tests/​test_examples_validate.py Extracts the minimal recipe from its new Markdown page.
tests/​test_example_temperature_series.py Updates a tolerance documentation comment.
tests/​test_example_output_recipe_regression.py Updates a tolerance documentation reference.
tests/​test_easydiffraction_native.py Updates the tolerance reference in comments.
tests/​test_code_hash.py Revises frozen-code guidance and failure messages.
tests/​subprocess_utils.py Updates the tolerance documentation comment.
src/​powderline/​symmetry.py Clarifies the symmetry-image distance error.
src/​powderline/​schema_core.py Updates documentation references in comments and docstrings.
src/​powderline/​results.py Simplifies envelope comments and docstrings.
src/​powderline/​registry.py Updates reinstall guidance and explanatory text.
src/​powderline/​recipes.py Removes internal references from documentation.
src/​powderline/​kicker.py Refreshes CLI hints and comments.
src/​powderline/​gateways/​topas/​gateway.py Cleans up gateway documentation and comments.
src/​powderline/​gateways/​gsasii/​simulation_builder.py Corrects the documented structure input key.
src/​powderline/​gateways/​gsasii/​setters.py Updates comments and docstrings only.
src/​powderline/​gateways/​gsasii/​server.py Clarifies foreground operation and server commands.
src/​powderline/​gateways/​gsasii/​schema.py Corrects the default-background warning text.
src/​powderline/​gateways/​gsasii/​gateway.py Updates install hints and gateway documentation.
src/​powderline/​gateways/​gsasii/​extractors.py Cleans up comments and docstrings only.
src/​powderline/​gateways/​gsasii/​constraints.py Updates a known-issue reference in the docstring.
src/​powderline/​gateways/​gsasii/​client.py Revises server recovery messages.
src/​powderline/​gateways/​easydiffraction/​gateway.py Updates install hints and documentation.
src/​powderline/​exceptions.py Refreshes exception and warning-registry documentation.
src/​powderline/​doctor.py Updates the stale-install recovery hint.
src/​powderline/​compat.py Simplifies compatibility documentation.
src/​powderline/​api.py Clarifies result keys and public API documentation.
src/​powderline/​_code_hash.json Refreshes hashes after comment and docstring edits.
scripts/​gen_schema_field_tables.py Documents the generated tables’ new homes.
scripts/​cross_engine_comparison.py Updates usage notes and documentation references.
scripts/​check_engine_version_candidate.py Points guidance to the relocated policies page.
pyproject.toml Updates reinstall advice in comments only.
pixi.toml Adds documentation checks and new theme dependencies.
examples/​example_template/​DESCRIPTION.md Improves example-authoring and fit-reporting guidance.
examples/​example_LaB6/​topas/​README.md Removes outdated recipe-authoring restrictions.
examples/​example_LaB6/​easydiffraction/​README.md Removes obsolete migration commentary.
examples/​example_LaB6/​DESCRIPTION.md Clarifies parameters, warnings, and reference outputs.
examples/​example_LaB6_textured/​make_textured.py Removes internal references from the docstring.
examples/​example_LaB6_textured/​DESCRIPTION.md Corrects the standard designation and trims history.
examples/​example_LaB6_temperature_series/​run_series.py Updates explanatory text and the guide reference.
examples/​example_LaB6_temperature_series/​DESCRIPTION.md Clarifies data provenance and current commands.
examples/​example_LaB6_singlepeakfit/​topas/​README.md Removes outdated recipe-authoring restrictions.
examples/​example_LaB6_singlepeakfit/​DESCRIPTION.md Labels the teaching case and explains a valid variant.
examples/​example_LaB6_simulation/​DESCRIPTION.md Refreshes commands, links, and beamline wording.
examples/​example_LaB6_ionic/​DESCRIPTION.md Documents the easydiffraction teaching-case limitation.
examples/​example_DRX_33/​topas/​README.md Removes outdated recipe-authoring restrictions.
examples/​example_DRX_33/​easydiffraction/​README.md Removes obsolete migration commentary.
examples/​example_DRX_33/​DESCRIPTION.md Explains ill-conditioning and committed outputs.
examples/​example_DRX_33_strainonly/​DESCRIPTION.md Corrects reference descriptions and uses a separate output directory.
examples/​example_DRX_33_simulation/​DESCRIPTION.md Updates commands and beamline wording.
examples/​example_DRX_33_anisoADP/​DESCRIPTION.md Documents unphysical results and corrects constraint guidance.
docs/​support-matrix.md Removes the page replaced by the versions reference.
docs/​requirements.txt Replaces the theme and adds documentation extensions.
docs/​reference/​versions.md Consolidates versions, compatibility, and schema history.
docs/​reference/​scope.md Lists supported boundaries and limitations.
docs/​reference/​known-issues.md Introduces the generated known-issue register.
docs/​reference/​index.md Organizes reference navigation.
docs/​reference/​ids.md Includes the generated EB/KI index.
docs/​reference/​glossary.md Defines recipe, engine, and result terminology.
docs/​reference/​fields.md Hosts generated schema field tables.
docs/​reference/​cli.md Documents CLI options, configuration, and exit statuses.
docs/​reference/​api.md Separates public API and helper documentation.
docs/​quickstart.md Removes the superseded quickstart.
docs/​index.rst Removes the old landing page.
docs/​index.md Adds the reader-oriented landing page and diagrams.
docs/​how-to/​simulate.md Explains simulation setup for each engine.
docs/​how-to/​sequential-refinement.md Documents chaining runs through output recipes.
docs/​how-to/​run-topas.md Explains TOPAS execution and generate-only operation.
docs/​how-to/​index.md Organizes task-focused guides.
docs/​getting-started/​your-own-data.md Walks through preparing and running a custom recipe.
docs/​getting-started/​install.md Documents environments, installation, and platforms.
docs/​getting-started/​index.md Defines the introductory tutorial sequence.
docs/​getting-started/​first-refinement.md Provides an end-to-end LaB6 tutorial.
docs/​fields.md Removes the superseded fields page.
docs/​engine-versions.md Removes the procedure relocated to policies.
docs/​development/​internals-api.md Separates contributor-facing internal API documentation.
docs/​development/​index.md Organizes development documentation.
docs/​concepts/​results.md Explains statistics, output recipes, provenance, and failures.
docs/​concepts/​recipes.md Explains recipe structure and engine-specific conventions.
docs/​concepts/​index.md Organizes conceptual documentation.
docs/​api.rst Removes the combined API page replaced by focused references.
docs/​about/​index.md Groups citation, licensing, and project information.
docs/​about/​citing.md Updates the versions-page link.
docs/​about/​changelog.md Adjusts the changelog include for its new location.
docs/​_generated/​eb_all.md Adds shared engine-behaviour entries with stable anchors.
docs/​_generated/​cross_engine_comparison.md Refreshes comparison provenance and reported warnings.
docs/​_generated/​cross_engine_comparison.json Refreshes comparison records from the 0.2.0 run.
CONTRIBUTING.md Condenses setup and contribution guidance.
CHANGELOG.md Records the documentation rebuild and message updates.
.github/​workflows/​ci.yaml Adds catalogue and example-figure checks.

Review effort: Balanced

Comment thread docs/concepts/results.md Resolved
Clarify the relationship between run_id and parent_run_id for tracing analyses.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 6, 2026 01:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Documentation still misstates the symmetry-image distance cutoff and describes Rexp as a lower bound rather than an expected noise-level Rwp.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

The hint said to move an atom until its symmetry images are at least
0.002 away. An image within 5e-06 is allowed (the atom sits on that
element); the error is an image between 5e-06 and 0.002. Message text
only.

Assisted-by: claude-code:claude-opus-5-5
- Validation checks each symmetry image on its own: below 5e-6 the atom
  sits on that element, 5e-6 to 2e-3 is an error. The concept page, the
  disorder how-to, troubleshooting, KI-17, KI-18 and EB-01 said
  otherwise or measured from the site.
- Rexp is the Rwp expected from the noise the weights describe, not a
  lower bound: Rwp falls below it when the weights overstate the noise.
- An output recipe names only the run that produced it; two pages still
  said the provenance links lead back through a whole chain.

Assisted-by: claude-code:claude-opus-5-5
Copilot AI balanced review requested due to automatic review settings November 6, 2026 01:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The sequential-refinement loop accepts generate-only TOPAS results as completed refinements, allowing the entire series to run with missing statistics unless it first checks that the status is ok.

No open issues 🎉

Review effort: Balanced

A TOPAS run without tc.exe returns status "generated" with no
statistics and the input as its output recipe, which validates, so the
how-to's loop ran the whole series unrefined. The loop now checks
status first, and the page says why.

Assisted-by: claude-code:claude-opus-5-5
Copilot AI balanced review requested due to automatic review settings November 6, 2026 02:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because the documentation may mislead readers about simulation output alignment and engine limits when bounds are left unstated.

No open issues 🎉

Review effort: Balanced

@AdamCorrao
Adam Corrao (AdamCorrao) merged commit d8677c8 into NSLS2:main Nov 6, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants