diff --git a/.claude/skills/bhom-branch-naming/CHANGELOG.md b/.claude/skills/bhom-branch-naming/CHANGELOG.md new file mode 100644 index 0000000..0e35005 --- /dev/null +++ b/.claude/skills/bhom-branch-naming/CHANGELOG.md @@ -0,0 +1,37 @@ +--- +name: BHoM Branch Naming Skill Changelog +description: Version history for all files in the bhom-branch-naming skill folder +version: 0.1.1 +author: BHoM +tags: [BHoM, git, changelog] +--- + +# Changelog - BHoM Branch Naming Skill + +## Change History + +### 2026-11-19 (Latest) +**Files modified**: +- SKILL.md (v0.1.0 → v0.1.1) +- CHANGELOG.md (v0.1.0 → v0.1.1) + +**Changes:** +- Frontmatter `author` set to `BHoM`. + +**Rationale:** +PR #265 review feedback. + +--- + +### 2026-11-18 +**Files created**: +- SKILL.md (v0.1.0) +- CHANGELOG.md (v0.1.0) + +**Changes:** +- Initial draft: `-#-` format, project-of-the-issue prefix, paired-change naming, no-issue handling, rename steps, checklist. + +**Rationale:** +Claude created `Python_Toolkit-bhom-python-claude-skill` with no issue number. The convention is taken from branch names on the Python_Toolkit, BHoM, BHoM_Engine and LadybugTools_Toolkit remotes. + +--- diff --git a/.claude/skills/bhom-branch-naming/SKILL.md b/.claude/skills/bhom-branch-naming/SKILL.md new file mode 100644 index 0000000..2d3ec4a --- /dev/null +++ b/.claude/skills/bhom-branch-naming/SKILL.md @@ -0,0 +1,128 @@ +--- +name: bhom-branch-naming +description: Name git branches for BHoM repositories as -#-. Use whenever creating, renaming or reviewing a branch in any BHoM or BHoM toolkit repo - including before committing on develop/main - and when a task has no issue number yet. +version: 0.1.1 +author: BHoM +tags: [BHoM, git, branches, workflow] +--- + +# BHoM Branch Naming Skill + +## Role + +You create and check git branch names in BHoM repositories so every branch traces back to the issue it addresses. + +## Usage Guide + +1. Never commit directly to `develop` or `main` - create a branch first. +2. Find the **issue** the work addresses and the **project** that issue belongs to. +3. Build the name with the [format](#format) below. +4. If there is no issue, [stop and ask](#no-issue-yet) - do not invent one or leave it out. +5. Check the name against the [checklist](#checklist) before creating the branch. + +--- + +## Format + +``` +-#- +``` + +| Part | Rule | Example | +|------|------|---------| +| `` | Name of the repository (or project) **where the issue lives**, exactly as GitHub spells it | `Python_Toolkit`, `Structure_Engine`, `Grasshopper_UI` | +| `-#` | Literal hyphen then hash | `-#` | +| `` | The GitHub issue number in that project | `257` | +| `-` | Short PascalCase summary, a few words, no spaces | `ReviewOldTests` | + +Examples from BHoM remotes: + +``` +Python_Toolkit-#253-UV_environment_management +Python_Toolkit-#228-CaptureToolkitName +Structure_Engine-#3642-Extend-IsValid-for-ICase (in the BHoM_Engine repo) +Classification_oM-#1762-AddRICSCategory (same name in BHoM and BHoM_Engine for a paired change) +Grasshopper_UI-#740-bhom-sample-scripts (in a toolkit repo, for an issue raised on Grasshopper_UI) +``` + +### The prefix is the issue's project, not always the repo + +- In multi-project repos (BHoM, BHoM_Engine), use the project: `Structure_oM-#...`, `Serialiser_Engine-#...`. +- For an issue in another repo, use that repo's name and issue number, even on a branch in this repo (`Grasshopper_UI-#740-...` on a toolkit). +- **Paired changes** across repos (e.g. oM + Engine, or C# + Python) use the **same branch name in every repo**, so CI and reviewers can pair them. + +### Description + +- PascalCase, describing the change: `AddSpaceCalcMethods`, `FixSunriseSunset`. +- Hyphens or underscores inside the description appear in existing branches and are tolerated; **for new branches, prefer PascalCase**. +- Keep it short (about 2-5 words). + +--- + +## No Issue Yet + +A branch without an issue number breaks the convention (e.g. `Python_Toolkit-bhom-python-claude-skill`). When there is no issue: + +1. **Ask the user** for the issue number, or whether to open one. +2. Opening an issue publishes content to GitHub: draft the title and body and **get explicit approval** before creating it. +3. Do not use `#XX`, `#0`, or a guessed number as a placeholder, and do not leave the number out. + +--- + +## Not Allowed + +| Wrong | Why | Right | +|-------|-----|-------| +| `Python_Toolkit-bhom-python-claude-skill` | No issue number | `Python_Toolkit-#270-AddBHoMPythonClaudeSkill` | +| `Python_Toolkit_#207-tkinter-ui-patches` | `_` before `#` | `Python_Toolkit-#207-TkinterUIPatches` | +| `Python_Toolkit-#XX-EnsureDirectoriesExist` | Placeholder number | Real issue number | +| `BHoM=#1570-DocumentationUpdates` | `=` instead of `-` | `BHoM-#1570-DocumentationUpdates` | +| `feature/add-thing`, `fix-bug`, `combined-branch` | Not the BHoM format | `-#-AddThing` | +| Committing on `develop` / `main` | Protected integration branches | Branch first | + +Automated branches (`dependabot/...`, `copilot/...`) are created by tools and are exempt. + +(`#270` above is illustrative only.) + +--- + +## Renaming a Branch + +Local only (not pushed yet): + +```bash +git branch -m -#- +``` + +If already pushed, rename locally, push the new name, then delete the old remote branch **only with the user's approval** (it may have an open PR or other users): + +```bash +git push -u origin +git push origin --delete +``` + +An open PR does not follow a renamed branch - it must be reopened from the new branch. + +--- + +## Checklist + +- [ ] Not on `develop` / `main` +- [ ] Format `-#-` +- [ ] `` is where the issue lives, spelled exactly +- [ ] Real issue number (no placeholder) +- [ ] PascalCase description, a few words +- [ ] Same name in every repo of a paired change + +--- + +## Related Skills + +- [bhom-develop](../bhom-develop/SKILL.md) - overall development workflow; branch creation happens at the start of implementation. +- [bhom-python](../bhom-python/SKILL.md) - Python work in Python_Toolkit. + +--- + +## Version History + +**See [CHANGELOG.md](./CHANGELOG.md) for full version history.** diff --git a/.claude/skills/bhom-python/CHANGELOG.md b/.claude/skills/bhom-python/CHANGELOG.md new file mode 100644 index 0000000..ca9d300 --- /dev/null +++ b/.claude/skills/bhom-python/CHANGELOG.md @@ -0,0 +1,204 @@ +--- +name: BHoM Python Skill Changelog +description: Version history for all files in the bhom-python skill folder +version: 0.2.10 +author: BHoM +tags: [BHoM, Python, changelog] +--- + +# Changelog - BHoM Python Skill + +## Change History + +### 2026-11-19 (Latest) +**Files modified**: +- SKILL.md (v0.2.3 → v0.2.4) +- SKILL-FRAMEWORK.md (v0.2.2 → v0.2.3) +- SKILL-IMPLEMENTATION.md (v0.2.7 → v0.2.8) +- SKILL-REVIEW.md (v0.2.5 → v0.2.6) +- SKILL-TESTING.md (v0.2.4 → v0.2.5) +- CHANGELOG.md (v0.2.9 → v0.2.10) + +**Files removed**: +- QA-NOTES.md + +**Changes:** +- Removed the reviewer-only QA notes; remaining open decisions are tracked in issue #264. +- Frontmatter `author` set to `BHoM` in every file. +- Removed "for QA" wording from the skill text. + +**Rationale:** +PR #265 review feedback. + +--- + +### 2026-11-18 +**Files modified**: +- SKILL-TESTING.md (v0.2.3 → v0.2.4) +- CHANGELOG.md (v0.2.8 → v0.2.9) + +**Changes:** +- "Golden JSON string" renamed to "BHoM JSON string" throughout. + +**Rationale:** +PR #265 review feedback. + +--- + +### 2026-11-18 +**Files modified**: +- SKILL.md (v0.2.2 → v0.2.3) +- SKILL-IMPLEMENTATION.md (v0.2.6 → v0.2.7) +- SKILL-REVIEW.md (v0.2.4 → v0.2.5) +- SKILL-TESTING.md (v0.2.2 → v0.2.3) +- CHANGELOG.md (v0.2.7 → v0.2.8) +- QA-NOTES.md + +**Changes:** +- The BHoM copyright header is now required on every `.py` file, source and tests: new review rule PY-F06 with an automated check, style rule S21, checklist items, and the "Copyright for Python Files" section rewritten from an open decision into the rule. + +**Rationale:** +PR #265 review feedback (decision D1). + +--- + +### 2026-11-18 +**Files modified**: +- SKILL.md (v0.2.1 → v0.2.2) +- SKILL-REVIEW.md (v0.2.3 → v0.2.4) +- SKILL-TESTING.md (v0.2.1 → v0.2.2) +- CHANGELOG.md (v0.2.6 → v0.2.7) + +**Changes:** +- "Happy path" renamed to "success path" throughout. + +**Rationale:** +PR #265 review feedback. + +--- + +### 2026-11-18 +**Files modified**: +- SKILL.md (v0.2.0 → v0.2.1) +- SKILL-FRAMEWORK.md (v0.2.1 → v0.2.2) +- SKILL-IMPLEMENTATION.md (v0.2.5 → v0.2.6) +- SKILL-REVIEW.md (v0.2.2 → v0.2.3) +- CHANGELOG.md (v0.2.5 → v0.2.6) + +**Changes:** +- Methods in `bhom/wrapped/` may modify their inputs (they receive copies deserialised from C# JSON): exception added to PY-C04, the input-mutation guidance, golden rule 7 and the wrapped-method guidance. +- `"test"` wrapped-method defect no longer lists input mutation. + +**Rationale:** +PR #265 review feedback. + +--- + +### 2026-11-18 +**Files modified**: +- SKILL-IMPLEMENTATION.md (v0.2.4 → v0.2.5) +- SKILL-REVIEW.md (v0.2.1 → v0.2.2) +- CHANGELOG.md (v0.2.4 → v0.2.5) + +**Changes:** +- S5 reworded: give sensible defaults where a user would not know what to enter; omit them only for inputs the user must supply; document every default in the docstring. +- PY-D02 now requires default values to be documented. + +**Rationale:** +PR #265 review feedback. + +--- + +### 2026-11-18 +**Files modified**: +- SKILL-IMPLEMENTATION.md (v0.2.3 → v0.2.4) +- SKILL-REVIEW.md (v0.2.0 → v0.2.1) +- SKILL-TESTING.md (v0.2.0 → v0.2.1) +- CHANGELOG.md (v0.2.3 → v0.2.4) + +**Changes:** +- Notebooks are never committed (`*.ipynb` is in `.gitignore`): replaced "clear outputs before commit" guidance, added notebooks to PY-S04 committed artefacts, removed "notebook outputs" from privacy rules. + +**Rationale:** +PR #265 review feedback. + +--- + +### 2026-11-18 +**Files modified**: +- SKILL-IMPLEMENTATION.md (v0.2.2 → v0.2.3) +- CHANGELOG.md (v0.2.2 → v0.2.3) + +**Changes:** +- `jupyterlab` stays a runtime dependency (notebooks must work in non-dev environments); only the pytest packages are proposed for the dev group. + +**Rationale:** +PR #265 review feedback. + +--- + +### 2026-11-18 +**Files modified**: +- SKILL-IMPLEMENTATION.md (v0.2.1 → v0.2.2) +- CHANGELOG.md (v0.2.1 → v0.2.2) + +**Changes:** +- `__init__.py` guidance is no longer a hard "constants and re-exports only" rule: keep them light, with nothing heavy or surprising on import. + +**Rationale:** +PR #265 review feedback. + +--- + +### 2026-11-18 +**Files modified**: +- SKILL-FRAMEWORK.md (v0.2.0 → v0.2.1) +- SKILL-IMPLEMENTATION.md (v0.2.0 → v0.2.1) +- CHANGELOG.md (v0.2.0 → v0.2.1) +- QA-NOTES.md + +**Changes:** +- Console-logger-on-stdout defect: points to the UV environment management update (#253) as the fix, instead of an open question about moving logging to stderr. +- Removed references to defects that are already fixed. + +**Rationale:** +PR #265 review feedback. + +--- + +### 2026-11-18 +**Files modified**: +- SKILL.md (v0.1.0 → v0.2.0) +- SKILL-IMPLEMENTATION.md (v0.1.0 → v0.2.0) +- SKILL-FRAMEWORK.md (v0.1.0 → v0.2.0) +- SKILL-TESTING.md (v0.1.0 → v0.2.0) +- SKILL-REVIEW.md (v0.1.0 → v0.2.0) +- CHANGELOG.md (v0.1.0 → v0.2.0) +- QA-NOTES.md (rewritten) + +**Changes:** +- Scoped to Python_Toolkit only: removed all references to other toolkits; toolkit conventions move to their own skills. +- Made `bhom_wrapper.bhom_callable` + `run_wrapped` the only route from C# into Python: decorator required, decorated methods live in `python_toolkit/bhom/wrapped/`, wrappers stay thin. +- Documented that `bhom_callable` encodes return values to BHoM JSON when called from C#, so methods return objects (strings pass through unencoded). +- Documented the input/output contract as verified in the `Python_Toolkit/v3_10` environment, and the established C# call shape (`run_wrapped.py -command -in `, whole stdout deserialised, `Traceback` text = failure). +- Added review rules PY-W01 to PY-W08 and test patterns for wrapped methods. +- Updated "Known Framework Defects" to `develop` at `d9d1edd` (#263 fixed the mutable/frozen defaults); added newly verified wrapper defects. +- Removed references to colour scales and palettes, which are not on `develop`. +- Objects with no C# type use `_t = "BH.oM.Base.CustomObject"` (review feedback). + +**Rationale:** +Review feedback: C# should always call Python through `run_wrapped`; the draft missed the decorator requirement, the `bhom/wrapped/` location rule, and the automatic BHoM JSON output conversion. Content had to be shareable from the Python_Toolkit repo, which is open source. + +--- + +### 2026-11-18 +**Files created**: +- SKILL.md, SKILL-IMPLEMENTATION.md, SKILL-FRAMEWORK.md, SKILL-TESTING.md, SKILL-REVIEW.md, CHANGELOG.md (v0.1.0) + +**Changes:** +- Initial cross-toolkit draft for QA. + +**Rationale:** +No existing BHoM skill covered Python. + +--- diff --git a/.claude/skills/bhom-python/SKILL-FRAMEWORK.md b/.claude/skills/bhom-python/SKILL-FRAMEWORK.md new file mode 100644 index 0000000..8fb9938 --- /dev/null +++ b/.claude/skills/bhom-python/SKILL-FRAMEWORK.md @@ -0,0 +1,342 @@ +--- +name: BHoM Python Framework +description: How to use the python_toolkit BHoM framework - C#-callable methods (bhom_callable + run_wrapped), BHoMObject/IObject and BHoM JSON, bhom_analytics, logging and pythonnet. +version: 0.2.3 +parent_skill: bhom-python +author: BHoM +tags: [BHoM, Python, framework, serialisation, logging, pythonnet] +--- + +# BHoM Python Framework + +> **Use when:** writing Python that C# calls, that exchanges BHoM objects, that logs or records analytics, or that calls BHoM .NET code. +> **Source of truth:** `Python_Engine/Python/src/python_toolkit/bhom/`. If this file and that code disagree, the code wins - flag the drift. + +--- + +## Quick Navigation + +- [Import Cheat Sheet](#import-cheat-sheet) +- [C#-Callable Methods: bhom_callable + run_wrapped](#c-callable-methods-bhom_callable--run_wrapped) +- [BHoM Objects and Serialisation](#bhom-objects-and-serialisation) +- [Logging](#logging) +- [Analytics Decorator](#analytics-decorator) +- [Calling .NET from Python (pythonnet)](#calling-net-from-python-pythonnet) +- [The C# Side](#the-c-side) +- [Known Framework Defects](#known-framework-defects) + +--- + +## Import Cheat Sheet + +| Need | Import | +|------|--------| +| Make a method callable from C# | `from python_toolkit.bhom.decorators import bhom_wrapper` | +| Entry point C# runs | `from python_toolkit.bhom.run_wrapped import run_wrapped` | +| BHoM base classes | `from python_toolkit.bhom.bhom_object import BHoMObject, IObject` | +| BHoM JSON | `from python_toolkit.bhom.bhom_object import BHoMJSONEncoder, BHoMJSONDecoder` | +| Console logging | `from python_toolkit.bhom.logging import CONSOLE_LOGGER` | +| Usage analytics | `from python_toolkit.bhom.analytics import bhom_analytics` | +| Project number for analytics | `from python_toolkit.bhom.analytics import set_project_number, get_project_number` | +| Install constants | `from python_toolkit.bhom import BHOM_VERSION, BHOM_LOG_FOLDER, TOOLKIT_NAME` | +| BHoM plot style | `plt.style.use("python_toolkit.bhom")` / `"python_toolkit.bhom_dark"` | +| Desktop UI | `python_toolkit.bhom_tkinter` (windows, widgets, theming) | +| Time-series helpers | `python_toolkit.helpers` | +| Unit conversions | `python_toolkit.units` | + +--- + +## C#-Callable Methods: bhom_callable + run_wrapped + +**This is the only route from C# into python_toolkit.** C# never imports modules, runs ad hoc scripts, or parses bespoke CLI arguments - it calls `run_wrapped(identifier, json)`, which dispatches to a method registered with `@bhom_wrapper.bhom_callable(...)`. + +### The rules + +1. **C# calls Python methods only through `run_wrapped`** (see [The C# Side](#the-c-side) for the call shape). +2. **Every C#-callable method has `@bhom_wrapper.bhom_callable("", ...)`.** Without it the method is not registered and `run_wrapped` raises `NotImplementedError`. +3. **Every method with that decorator lives under `python_toolkit/bhom/wrapped/`.** `run_wrapped.py` imports `python_toolkit.bhom.wrapped`, whose `__init__.py` imports every `.py` file in that folder (recursively) - that import is what registers the method. A decorated method anywhere else is never imported by `run_wrapped`, so it is never registered. +4. **Return Python objects, not JSON.** When called from C# (i.e. with `__input_json__`), the decorator encodes the return value with `BHoMJSONEncoder` for you - `BHoMObject`s, `IObject`s, dicts, lists, numpy arrays, pandas Series/Timestamps, `datetime`, `UUID` and `Path` all become BHoM JSON. Do **not** `json.dumps` the result yourself. +5. **The same method stays a normal Python function.** Called from Python without `__input_json__`, the decorator returns the raw result untouched, so Python users and tests call it directly. + +### Pattern + +`python_toolkit/bhom/wrapped/plot/heatmap_png.py`: + +```python +"""C# entry point: render a heatmap of a time series to a PNG file.""" + +from pathlib import Path + +import matplotlib + +matplotlib.use("Agg") # headless: C# runs this in a hidden process + +import matplotlib.pyplot as plt # noqa: E402 +import pandas as pd # noqa: E402 + +from python_toolkit.bhom.analytics import bhom_analytics # noqa: E402 +from python_toolkit.bhom.decorators import bhom_wrapper # noqa: E402 +from python_toolkit.plot.heatmap import heatmap # noqa: E402 + + +@bhom_wrapper.bhom_callable("python_toolkit.plot.heatmap_png") +@bhom_analytics() +def heatmap_png(values: list[float], start: str, output_path: str) -> Path: + """Render an hourly heatmap of values starting at start and save it as a PNG. + + Args: + values (list[float]): + Hourly values. + start (str): + ISO 8601 timestamp of the first value. + output_path (str): + Where to write the PNG. + + Returns: + Path: + The written file (returned to C# as a BH.oM.Adapter.FileSettings). + """ + series = pd.Series(values, index=pd.date_range(start, periods=len(values), freq="h")) + fig, ax = plt.subplots() + heatmap(series, ax=ax) + fig.savefig(output_path, dpi=150) + plt.close(fig) + return Path(output_path) +``` + +Points to copy: + +- **Thin wrapper.** The wrapped method converts C#-friendly inputs (lists, strings, BHoM objects) into Python types, calls the real function in a domain module (`python_toolkit.plot.heatmap`), and returns plain objects. Logic does not live in `bhom/wrapped/`. +- **Decorator order:** `bhom_callable` outermost (it registers what it wraps), `bhom_analytics()` inside it, so analytics records the real function name. +- **Light imports.** Every module in `bhom/wrapped/` is imported on *every* `run_wrapped` call, whichever method is requested. Keep module-level imports cheap and side-effect free; import heavy optional libraries inside the function. +- **Input types C# can produce.** Use JSON-native types (`float`, `str`, `list`, `dict`) or BHoM objects. A `pd.Series` or `datetime` cannot arrive from C# directly. +- **Inputs are the method's to modify.** From C#, a wrapped method receives copies deserialised from JSON and returns a copy as JSON, so it may change its inputs in place - the usual "don't mutate inputs" rule does not apply here. A Python caller calling a wrapped method directly should not rely on its arguments being left unchanged. + +### Identifiers + +- Globally unique across everything registered: **(proposed)** `"python_toolkit.."`, mirroring the module path. +- Never change a published identifier without a C# deprecation path - it is the API. + +### What `run_wrapped` receives (verified) + +`run_wrapped(identifier, json)` where `json` is either a JSON string beginning with `{` or a **path to a file** containing it (anything not starting with `{` is treated as a path). + +The JSON must be a **plain object whose keys are the method's argument names**. Values can be BHoM-serialised objects: + +```json +{ "values": [1.0, 2.0, 3.0], "start": "2026-01-01T00:00", "output_path": "C:/temp/out.png" } +``` + +```json +{ "geometry": { "_t": "BH.oM.Base.CustomObject", "BHoM_Guid": "f428e614-...", "X": 1 }, "arg2": "hello" } +``` + +**Do not** pass a BHoM-serialised `CustomObject` or `Dictionary` as the *root*. The BHoM serialiser adds `BHoM_Guid` / `_t` / `_bhomVersion` keys at the root, and these arrive as unexpected keyword arguments (`TypeError: ... unexpected keyword argument 'BHoM_Guid'` / `'_t'`). See [Known Framework Defects](#known-framework-defects). + +### `argument_types` - getting your subclass back + +The decoder turns any JSON object with `_t` and `BHoM_Guid` into a plain `BHoMObject`. To receive your subclass instead, map the argument name to it: + +```python +@bhom_wrapper.bhom_callable("python_toolkit.example.move", argument_types={"point": MyPoint}) +def move(point: MyPoint, dx: float) -> MyPoint: + ... +``` + +Limits: it upgrades **top-level arguments only**, only when the decoded value is exactly `BHoMObject` (not `IObject`), and not items inside lists or nested properties - convert those yourself with `MyPoint._from_bhom_object(obj)`. + +### What C# gets back (verified) + +- `run_wrapped` `print`s the encoded result: **one line of JSON** on stdout. +- **A `str` return value is not encoded** - it is printed as-is, so C# receives bare text, not JSON. Return a `Path`, dict or object instead of a string when C# deserialises the result. +- **Other stdout output comes first.** `CONSOLE_LOGGER` writes to stdout, and so does any `print`. A real run of the shipped `"test"` method prints: + + ``` + BHoMObject of type BH.oM.Base.CustomObject, name: '', version: '9.3', id: 'f428e614-...' + hello + {"Name": "", "BHoM_Guid": "f428e614-...", "_t": "BH.oM.Base.CustomObject", "_bhomVersion": "9.3", "X": 20} + ``` + + The established C# callers deserialise the **whole** of stdout, so anything printed before the result breaks the call. **No `print` in wrapped code**, and keep `CONSOLE_LOGGER` to warnings/errors on wrapped code paths (see [Known Framework Defects](#known-framework-defects)). +- Exceptions propagate: the process exits non-zero with the traceback on stderr, and `Compute.RunCommandStdout` returns **stderr instead of stdout** on a non-zero exit. The established C# callers treat output containing `Traceback (most recent call last):` as a failure, so raising is the correct way to fail - no need (and no reason) to return `traceback.format_exc()` yourself. + +--- + +## BHoM Objects and Serialisation + +### The two base classes + +| Class | Mirrors | Use for | +|-------|---------|---------| +| `IObject` | C# `BH.oM.Base.IObject` | BHoM-serialised objects that are **not** `BHoMObject`s (e.g. `BH.oM.Geometry.Point`) | +| `BHoMObject(IObject)` | C# `BH.oM.Base.BHoMObject` | Objects with `Name`, `BHoM_Guid`, `Tags`, `Fragments`, `CustomData` | + +Both carry `_t` (the full C# type name) and `_bhom_version` (short version, e.g. `"9.3"`). + +### Naming across the boundary + +The encoder/decoder convert property names automatically: + +| C# (JSON) | Python attribute | +|-----------|------------------| +| `FileName` | `file_name` | +| `BHoM_Guid` | `bhom_guid` | +| `CustomData` | `custom_data` | +| `_bhomVersion` | `_bhom_version` | +| `{"$date": }` | `datetime` | + +Write Python attributes in the snake_case the converter produces from the C# property name. Acronyms round-trip but give awkward names: `HVACSystem` <-> `h_v_a_c_system` (verified). A hand-written `hvac_system` would serialise as `HvacSystem`, which C# ignores. Test the round trip for any property with consecutive capitals. + +### Subclassing (the supported pattern) + +```python +from python_toolkit.bhom.bhom_object import BHoMObject + + +class FileSettings(BHoMObject): + """Python mirror of BH.oM.Adapter.FileSettings.""" + + def __init__(self, file_name: str, directory: str, **kwargs) -> None: + """ + Args: + file_name: The name of the file, with extension. + directory: The folder containing the file. + **kwargs: BHoMObject properties (name, bhom_guid, tags, ...) passed through from deserialisation. + """ + kwargs.setdefault("_t", "BH.oM.Adapter.FileSettings") + super().__init__(**kwargs) + self.file_name = file_name + self.directory = directory +``` + +- `_t` must be the exact C# full type name. **If there is no C# type for the object, use `"BH.oM.Base.CustomObject"`** - never an invented `_t`. C# then reads the extra properties into the `CustomObject`'s `CustomData`. +- Accept `**kwargs` and pass them to `super().__init__`, so `from_json` (via `_from_bhom_object`) can rebuild your subclass. +- Deserialise with `FileSettings.from_json(j)` (single) or `FileSettings.from_json_array(j)` (list). `BHoMObject.from_json` on a JSON array logs a warning and returns only the first item. +- Equality (`==`) compares `_t` and all properties except `bhom_guid` and `_bhom_version`. + +### Types the encoder already handles + +`uuid.UUID`, `datetime` / `pd.Timestamp` (-> `{"$date": ...}`), `np.ndarray` (-> list), `pd.Series` (-> dict, index as str), `pd.DatetimeIndex` (-> ISO strings), `pathlib.Path` (-> `BH.oM.Adapter.FileSettings`). + +For anything else, subclass `BHoMJSONEncoder` and override `serialise_unknown`, calling `super().serialise_unknown(obj)` for types you do not handle; mirror with `deserialise_unknown` on a `BHoMJSONDecoder` subclass. Pass the subclasses to `bhom_callable(encoder_cls=..., decoder_cls=...)`. + +### Do not hand-roll BHoM JSON + +Building `{"_t": ..., "BHoM_Guid": ..., "Name": ...}` dicts by hand, converting key case yourself, or stripping `_t` keys ad hoc duplicates the encoder/decoder and drifts from them. Extend the classes above instead. + +--- + +## Logging + +Two loggers, both configured at import time in `python_toolkit/bhom/logging/`: + +| Logger | Name | Handler | Purpose | +|--------|------|---------|---------| +| `CONSOLE_LOGGER` | `"Python_Toolkit[console]"` | `StreamHandler(sys.stdout)`, INFO | User-facing messages | +| `ANALYTICS_LOGGER` | `"Python_Toolkit"` | `RotatingFileHandler` in `%PROGRAMDATA%/BHoM/Logs` (25 MB, 1 backup) | **Analytics only** - written by `bhom_analytics`. Never log to it directly. | + +Both set `propagate = False`, so they do not leak into a host's root logger. + +### Rules + +1. **No `print()`.** Use `CONSOLE_LOGGER`. +2. Pick the level by consequence, mirroring C# `RecordError / RecordWarning / RecordNote`: + + | C# | Python | When | + |----|--------|------| + | `RecordError` | `raise` a specific exception | Operation cannot continue | + | `RecordWarning` | `CONSOLE_LOGGER.warning(...)` | Continues, but the result has caveats | + | `RecordNote` | `CONSOLE_LOGGER.info(...)` | Informational | + | - | `CONSOLE_LOGGER.debug(...)` | Developer diagnostics (dropped at the default INFO level) | + +3. Inside wrapped methods, remember the console logger shares stdout with the result (see above). +4. Never log secrets, tokens, or project/client names. +5. Never call `logging.basicConfig()` or attach handlers to the root logger - it hijacks the host (Grasshopper, Excel, notebooks). + +Code that builds on python_toolkit and needs its own logger makes it a **child** of `CONSOLE_LOGGER` (`logger.parent = CONSOLE_LOGGER; logger.propagate = True`) rather than adding a second handler, so messages are not printed twice. + +--- + +## Analytics Decorator + +`@bhom_analytics()` records one usage entry (`BH.oM.Base.UsageLogEntry` shape) per call to `%PROGRAMDATA%/BHoM/Logs/Usage__.log`, including any exception raised, then re-raises. + +- **Apply it to public, user-facing functions and methods** - including wrapped methods. Not to private helpers (`_name`), dunders, properties, classes, or tight inner loops (each call writes a log line). +- **Always call it: `@bhom_analytics()`.** Written as `@bhom_analytics` (no brackets), the decorated function silently returns a wrapper *function* instead of its result - no error (verified). +- Order: `@staticmethod` / `@classmethod` / `@property` outermost; `@bhom_wrapper.bhom_callable` outside `@bhom_analytics()`. +- Set the project number once per session with `set_project_number("")`; never hard-code one. +- Tests and CI run with `DISABLE_BHOM_ANALYTICS=1` (read once at import - set it before importing). +- Type hints matter: the decorator records parameter annotations as `_t` in the log. + +--- + +## Calling .NET from Python (pythonnet) + +Direction: **C# never embeds Python** (no `Py.GIL` / `PythonEngine`); it runs Python as a subprocess via `run_wrapped`. pythonnet is used the other way - Python calling BHoM .NET - as in `python_toolkit/bhom/installer_info.py`: + +```python +import os +import sys + +import clr + +sys.path.append(os.path.expandvars("%ProgramData%\\BHoM\\Assemblies")) +clr.AddReference("Serialiser_Engine") + +from BH.Engine.Serialiser import Convert # pylint: disable=import-error,wrong-import-position +``` + +- Load assemblies from `%ProgramData%\BHoM\Assemblies` only. +- Keep `clr` imports in the module that needs them, guarded for non-Windows (as `python_toolkit.bhom.__init__` does with `os.name == "nt"`), so the package still imports without .NET. +- Cross the boundary with JSON (`Convert.ToJson` / `Convert.FromJson` + `BHoMJSONDecoder`) rather than passing live .NET objects through Python code. +- Non-Windows (Docker) uses `PYTHONNET_RUNTIME=coreclr` - see `Python_Engine/Python/Dockerfile`. + +--- + +## The C# Side + +`BH.Engine.Python` (`Python_Engine/`): + +| Method | Purpose | +|--------|---------| +| `Compute.BasePythonEnvironment(version, reload, run)` | Install/retrieve base Python (default `v3_10`) with `virtualenv`, `jupyterlab`, `black`, `pylint` and `python_toolkit` | +| `Compute.VirtualEnvironment(version, name, reload)` | Create a venv in `%ProgramData%\BHoM\Extensions\PythonEnvironments\`, add `ipykernel`, `pytest`, `black`, register a Jupyter kernel | +| `Compute.InstallPackageLocal` / `InstallRequirements` / `InstallPackages` | Install local packages and dependencies | +| `Compute.UpdateBHoMPackages(env)` | Reinstall BHoM Python packages into an existing env | +| `Compute.RunCommandStdout(command, ...)` | Run via `cmd.exe`, return stdout - or **stderr** if the exit code is non-zero | +| `Compute.RunPythonString(env, script)` | Write a script to a temp `.py`, run it, return stdout | +| `Query.DirectoryCode()` / `DirectoryEnvironments()` / `VirtualEnvironmentExecutable(name)` | Standard paths - never hard-code them | + +`BH.oM.Python.PythonEnvironment` has `Name` and `Executable`. + +**Calling a wrapped method - the established shape.** BHoM toolkits that build on python_toolkit each ship a small `run_wrapped.py` **script** in their own `bhom/` folder. It imports `python_toolkit.bhom.wrapped` (and the toolkit's own `wrapped` package) so every method is registered, parses `-command -in `, calls `method(__input_json__=)` and prints the result. The C# side: + +1. Builds a plain JSON object of argument name -> value, BHoM-serialising object values with `BH.Engine.Serialiser.Convert.ToJson`. +2. Writes it to `Path.GetTempFileName()` (avoids `cmd.exe` quoting and the ~8191 character command-line limit). +3. Runs `"" "<...>/bhom/run_wrapped.py" -command -in ""` through `Compute.RunCommandStdout`, with the script path built from `Query.DirectoryCode()`, then deletes the temp file. +4. Treats output containing `Traceback (most recent call last):` as failure; otherwise `Convert.FromJson` on the whole output. + +python_toolkit's own `run_wrapped(identifier, json)` is the in-process function form of the same thing. From a shell the equivalent is `"" -c "from python_toolkit.bhom.run_wrapped import run_wrapped; run_wrapped('', r'')"` (verified against the installed `Python_Toolkit/v3_10` env). + +> **(proposed)** Give python_toolkit's `run_wrapped.py` the `-command` / `-in` `__main__` entry point, and add a `Compute.RunWrapped(env, identifier, inputs)` to `BH.Engine.Python` that does steps 1-4, so each toolkit stops re-implementing both halves. + +Python code ships via the `Python_Engine.csproj` build step: `robocopy` of `Python_Engine/Python` to `%ProgramData%\BHoM\Extensions\PythonCode\Python_Toolkit` (excluding `*.pyc`, `*.ipynb`, dunder and dot folders) plus a `VERSION.txt` that `setup.py` reads. **Rebuild to deploy Python changes**; then `UpdateBHoMPackages` (or recreate the env) to install them. + +--- + +## Known Framework Defects + +On `develop` at `d9d1edd`. Until fixed, write code that does not depend on the broken behaviour, and do not copy these patterns: + +| Location | Defect | Workaround | +|----------|--------|-----------| +| `_bhom_callable_decorator.py` | Root input that is a BHoM-serialised `CustomObject` or `Dictionary` fails: `BHoM_Guid` / `_t` / `_bhomVersion` become unexpected kwargs (verified) | C# sends a plain JSON object (see [The C# Side](#the-c-side)); fix: drop reserved keys before calling | +| `_bhom_callable_decorator.py` | Bare `except:`; on invalid JSON it logs, then fails with `UnboundLocalError: json_kwargs` (verified) | Validate JSON on the C# side; fix: raise a `ValueError` with the parse error | +| `_bhom_callable_decorator.py` | Input not starting with `{` is assumed to be a path - leading whitespace/BOM or a JSON array gives `FileNotFoundError` | Send compact JSON or a file path; fix: `lstrip()` and check `Path.exists()` | +| `_bhom_callable_decorator.py` | `argument_types={}` mutable default (read-only today, but violates PY-C01) | - | +| `wrapped/_test_wrapped.py` | Ships a `"test"` method that `print`s to stdout - and it is registered in every environment that imports `python_toolkit.bhom.wrapped` | Move to `tests/` | +| `logging/console.py` | Console logger writes to **stdout**, the same stream C# deserialises | Keep wrapped paths quiet until the UV environment management update (#253), where the C# run method returns stdout and stderr separately | +| `analytics.py` wrapper | `Errors.extend()` adds the dict's *keys*, not the error event | - (fix: `append`) | +| `bhom_object.py` `IObject.from_dict` | Catches `ctypes.ArgumentError`, but bad kwargs raise `TypeError` | - | +| `helpers/timeseries.py` `validate_timeseries` | `is_annual` check `(n != 365) or (n != 366)` is always true (verified) | Do not use `is_annual=True` | +| `python_toolkit/__init__.py` | On Windows, overwrites `os.environ["HOME"]` with `C:/Users/` at import | Do not rely on `HOME` after import | diff --git a/.claude/skills/bhom-python/SKILL-IMPLEMENTATION.md b/.claude/skills/bhom-python/SKILL-IMPLEMENTATION.md new file mode 100644 index 0000000..7c84f6b --- /dev/null +++ b/.claude/skills/bhom-python/SKILL-IMPLEMENTATION.md @@ -0,0 +1,247 @@ +--- +name: BHoM Python Implementation +description: Package layout, packaging, environments and code style for writing Python in Python_Toolkit. +version: 0.2.8 +parent_skill: bhom-python +author: BHoM +tags: [BHoM, Python, structure, style, packaging] +--- + +# BHoM Python Implementation + +> **Use when:** adding or changing Python in Python_Toolkit - a new module, function or class, or packaging/environment changes. +> **Authority:** `Python_Engine/Python/README.md` ("BHoM/Python specific style guidance") is the published standard. Where it is silent, PEP 8 applies. Rules stricter than the README are marked **(proposed)**. + +--- + +## Quick Navigation + +- [Most Common Task: Add a Function](#most-common-task-add-a-function) +- [Package Layout](#package-layout) +- [Packaging and Environments](#packaging-and-environments) +- [Style Rules](#style-rules) +- [Errors and Validation](#errors-and-validation) +- [Performance and Data](#performance-and-data) +- [Security and Privacy](#security-and-privacy) +- [Checklist](#checklist) + +--- + +## Most Common Task: Add a Function + +`src/python_toolkit/helpers/degree_days.py`: + +```python +"""Heating and cooling degree-day calculations.""" + +import pandas as pd + +from python_toolkit.bhom.analytics import bhom_analytics +from python_toolkit.bhom.logging import CONSOLE_LOGGER +from python_toolkit.helpers.timeseries import validate_timeseries + + +@bhom_analytics() +def heating_degree_days(air_temperature: pd.Series, base_temperature: float = 15.5) -> float: + """Calculate heating degree days from an hourly air temperature series. + + Args: + air_temperature (pd.Series): + Hourly dry-bulb air temperature in degC, with a DatetimeIndex. + base_temperature (float, optional): + Base temperature in degC below which heating is required. Defaults to 15.5 (CIBSE TM41). + + Returns: + float: + Heating degree days in K.day. + + Raises: + TypeError: If air_temperature is not a pandas Series with a DatetimeIndex. + ValueError: If air_temperature is empty. + """ + validate_timeseries(air_temperature, is_hourly=True) + if air_temperature.empty: + raise ValueError("air_temperature is empty - provide at least one day of hourly data.") + + if air_temperature.isna().any(): + CONSOLE_LOGGER.warning("air_temperature contains NaN values; these hours are ignored.") + + deficit = (base_temperature - air_temperature).clip(lower=0) + return float(deficit.sum() / 24) +``` + +The BHoM copyright header goes above the module docstring (S21); it is left out of the examples in this skill for length. + +What this shows: one public function per file, file named after it, module docstring, absolute imports grouped stdlib / third-party / python_toolkit, full type hints, Google docstring with units and `Raises`, validation at entry, a specific exception with a fix-it message, a warning via `CONSOLE_LOGGER`, analytics on the public entry point, vectorised pandas, a plain `float` return (not `numpy.float64`). + +If C# needs to call it, add a thin wrapper under `bhom/wrapped/` - see [SKILL-FRAMEWORK.md](./SKILL-FRAMEWORK.md#c-callable-methods-bhom_callable--run_wrapped). Then add tests - see [SKILL-TESTING.md](./SKILL-TESTING.md). + +--- + +## Package Layout + +``` +Python_Toolkit/ +├── Python_Engine/ # C#: BH.Engine.Python (environments, RunCommandStdout, ...) +│ └── Python/ +│ ├── README.md # published BHoM Python guidance +│ ├── pyproject.toml +│ ├── setup.py # reads VERSION.txt written by the build +│ ├── Dockerfile +│ ├── run_tests.bat +│ ├── src/python_toolkit/ +│ │ ├── __init__.py # package constants +│ │ ├── bhom/ # BHoM integration +│ │ │ ├── analytics.py, util.py, bhom_object.py, installer_info.py +│ │ │ ├── logging/ # CONSOLE_LOGGER, ANALYTICS_LOGGER +│ │ │ ├── decorators/ # bhom_wrapper.bhom_callable +│ │ │ ├── wrapped/ # ALL C#-callable methods, and nothing else +│ │ │ └── run_wrapped.py # the single C# entry point +│ │ ├── bhom_tkinter/ # BHoM-styled tkinter windows and widgets +│ │ ├── helpers/ # general helpers (time series, decay, cardinality) +│ │ ├── plot/ # matplotlib plots +│ │ └── units/ # unit conversion enums +│ └── tests/ +└── Python_oM/ # C#: BH.oM.Python (PythonEnvironment, PythonVersion) +``` + +Rules: + +- **One public function or class per file**, file named in snake_case after it (`AThing` -> `a_thing.py`, `do_stuff` -> `do_stuff.py`). Private helpers used only by that function may sit in the same file with a leading `_` (README). +- Organise by domain into sub-packages; general helpers go in `helpers/` (README). +- **C#-callable methods live in `bhom/wrapped/` and nowhere else; `bhom/wrapped/` holds only C#-callable methods** - thin wrappers calling into the domain sub-packages. +- Keep `__init__.py` files light: constants, re-exports and small setup are fine, but nothing heavy or surprising happens on import (see S20). +- New sub-packages need an `__init__.py` and, if they ship non-`.py` files, an entry in `[tool.setuptools.package-data]`. + +--- + +## Packaging and Environments + +| Item | Rule | +|------|------| +| Python version | Base environment default is `PythonVersion.v3_10` (`Compute.BasePythonEnvironment`). Code must run on 3.10 - no 3.11+ syntax or stdlib (`tomllib`, `ExceptionGroup`, `typing.Self`). | +| `requires-python` | **(proposed)** Add `requires-python = ">=3.10"` to `pyproject.toml` (currently absent). | +| Version number | Never hard-code. `setup.py` reads `VERSION.txt`, written by the `Python_Engine.csproj` build step from the assembly `FileVersion`. | +| Dependencies | `[project] dependencies` in `pyproject.toml`. Pin a package only with a stated reason (comment) - every BHoM toolkit environment installs python_toolkit first, so tight pins here constrain all of them. | +| Test/dev tools | `[dependency-groups] dev`. **(proposed)** `pytest`, `pytest-cov` and `pytest-order` are currently runtime dependencies - move them. `jupyterlab` stays a runtime dependency: notebooks are expected to work in non-dev environments. | +| Package data | Non-`.py` files (`.mplstyle`, `.tcl`, `.png`) must be listed in `[tool.setuptools.package-data]` or they are missing after install. Currently only `*.mplstyle` is listed - check `.tcl` theme files and `bhom/assets/*.png` are covered. | +| Paths | C# side: `BH.Engine.Python.Query.*`. Python side: `BHOM_LOG_FOLDER`, `os.environ["PROGRAMDATA"]`, `Path(__file__)` - never literal `C:\...` paths. | +| Deploying | Build `Python_Engine` (post-build copy to `%ProgramData%\BHoM\Extensions\PythonCode\Python_Toolkit`), then `UpdateBHoMPackages`. `run_tests.bat` tests that deployed copy. | +| Notebooks | Fine for local exploration (`%load_ext autoreload` / `%autoreload 2`), but **never committed** - `*.ipynb` is in `.gitignore`. Anything worth keeping moves into the package or a pytest test. | + +--- + +## Style Rules + +Formatting is mechanical: **black** (installed in every BHoM environment). Do not hand-argue formatting in review. + +### Required (README) + +| # | Rule | +|---|------| +| S1 | PEP 8, formatted with `black`. | +| S2 | **Absolute imports** - `from python_toolkit.plot.heatmap import heatmap`, not `from ..plot import heatmap`. (Much of `bhom/` still uses relative imports - fix when touched.) | +| S3 | **Type hints on every function and method** - all parameters and the return (`-> None` included). | +| S4 | **Google-style docstrings** on every module, class and function/method (dunders excepted). Methods of a documented class need at least a one-line description. | +| S5 | Give a sensible default wherever a typical user would not know what to enter (settings, tolerances, styling). Leave the default off only for inputs the user must supply (e.g. the data being processed). **Document every default in the docstring** ("Defaults to 15.5."). The README words this more strictly ("If in doubt do not include a default"); this is the agreed reading. | +| S6 | Real defaults in the signature, not `x=None` then `if x is None: x = ...` - **except mutable or call-evaluated defaults**, which must use `None` (S10). | +| S7 | One public function/class per file; snake_case file names. | + +### Required (proposed) + +| # | Rule | +|---|------| +| S8 | **British English** in names, docstrings, messages and comments (`colour`, `normalise`, `analyse`) - as [bhom-coding-standards](../bhom-coding-standards/SKILL.md). Third-party API names keep their spelling (`cmap`, `color=`). Existing exports such as `lighten_color` stay for compatibility. | +| S9 | `snake_case` functions/variables/modules, `PascalCase` classes, `UPPER_SNAKE` module constants, `_leading` for private. No single-letter names outside comprehensions and maths. | +| S10 | **No mutable or call-evaluated defaults** - not `[]`, `{}`, `uuid.uuid4()`, `datetime.now()`, `pd.Series()` in a signature or dataclass field. Use `None` or `field(default_factory=...)`. | +| S11 | `pathlib.Path` for paths; accept `str | Path` and convert once at entry. Text files opened with explicit `encoding="utf-8"`. | +| S12 | f-strings for message text. | +| S13 | No `print()` - use `CONSOLE_LOGGER` ([SKILL-FRAMEWORK.md](./SKILL-FRAMEWORK.md#logging)). | +| S14 | No `import *`, unused imports, commented-out code blocks, or `TODO` without an issue link. | +| S15 | Units in names or docstrings for every physical quantity (`air_temperature` "in degC"); SI by default, as BHoM. | +| S16 | Magic numbers named as module constants with their source (`TM52_CRITERION_1_K = 1.0 # CIBSE TM52 criterion 1`). | +| S17 | Typing compatible with 3.10: `list[str]`, `X | None` are fine; one style per file (don't mix with `Optional`/`List`). | +| S18 | `# pylint: disable=...` only with the specific code and a reason; no blanket disables around import blocks. | +| S19 | UTF-8 **without BOM** (24 files in `src` currently start with a BOM). | +| S20 | No work at import time in package modules: no network, `atexit`, environment-variable changes, or `sys.path` edits. Modules in `bhom/wrapped/` are imported on every `run_wrapped` call - keep them especially light. | +| S21 | **BHoM copyright header** at the very top of every `.py` file, source and tests, as `#` comments above the module docstring; end year = current year, updated whenever the file is modified. Template: [SKILL-REVIEW.md](./SKILL-REVIEW.md#copyright-for-python-files). | + +### Docstring template (Google) + +```python +def function_name(param_a: int, param_b: str | None = None) -> list[str]: + """One-line summary in the imperative mood. + + Optional longer description: method, assumptions, references (standard + clause). + + Args: + param_a (int): + What it is, units, valid range. + param_b (str | None, optional): + What it is. Defaults to None, meaning ... + + Returns: + list[str]: + What comes back. + + Raises: + ValueError: When and why. + """ +``` + +`bhom/analytics.py` uses numpy-style sections (`Returns\n-------`) - legacy; new code uses Google style. Placeholder docstrings (`"""_"""`) fail review. + +--- + +## Errors and Validation + +README: errors are handled gracefully; the user is told **what went wrong and how to fix it**; specific exceptions are preferred; errors are captured at the lowest possible level. + +- Validate inputs at public entry points (types, ranges, lengths, index alignment), as C# BHoM does. +- Raise specific built-ins (`ValueError`, `TypeError`, `FileNotFoundError`, `KeyError`), or a package exception that subclasses one (`class TimeseriesError(ValueError)`), so callers can catch either. +- Message = what + how to fix: `f"EPW file not found: {path}. Check the path exists."`. +- **No bare `except:`.** `except Exception` only at a boundary (UI callback, analytics wrapper), logged with `exc_info=True`, then re-raised or turned into an explicit error - never `pass`, never a silent `None`. +- Chain causes: `raise ValueError("...") from exc`. +- No `assert` for runtime validation (stripped under `-O`). +- Caveats on a result -> `CONSOLE_LOGGER.warning`. `warnings.warn` only for genuine deprecations. +- **Wrapped methods:** let exceptions propagate - the process exits non-zero and C# receives the traceback as stderr. Do not return a traceback string as the "result". + +--- + +## Performance and Data + +BHoM priority order: **performance, security, maintainability, readability** ([bhom-coding-standards](../bhom-coding-standards/SKILL.md#priority-order)). + +- Vectorise with numpy/pandas; no Python loops over 8760-row series where an array operation exists. +- Do not mutate inputs (`df = df.copy()` first) unless the function is documented as in-place. Wrapped methods in `bhom/wrapped/` are the exception (see [SKILL-FRAMEWORK.md](./SKILL-FRAMEWORK.md#c-callable-methods-bhom_callable--run_wrapped)). +- Return plain Python types at public and C# boundaries where practical (`float(x)`, not `numpy.float64`). +- `with` for files; `plt.close(fig)` for figures created in non-interactive code. +- `functools.lru_cache` only for pure functions with hashable, immutable inputs. + +--- + +## Security and Privacy + +- No credentials, tokens, connection strings or API keys in code, tests or notebooks. +- No project names, client names, job numbers or personal data in code, test assets or log messages (PR template warning). +- `subprocess` with an argument list - never `shell=True` with interpolated input. +- No `eval`/`exec` on external input; `pickle` only for trusted local files. +- Validate paths received from C# or the UI before deleting or overwriting. + +--- + +## Checklist + +- [ ] Right sub-package; one public function/class per file; snake_case file name +- [ ] BHoM copyright header with the current end year +- [ ] Module docstring; Google docstrings with units and `Raises` +- [ ] Type hints on all parameters and returns; 3.10-compatible +- [ ] Absolute imports, grouped stdlib / third-party / python_toolkit +- [ ] No mutable or call-evaluated defaults +- [ ] Inputs validated; specific exceptions with fix-it messages; no bare `except` +- [ ] `CONSOLE_LOGGER`, not `print`; `@bhom_analytics()` on public entry points +- [ ] C#-callable? Thin `@bhom_wrapper.bhom_callable` wrapper in `bhom/wrapped/`, returning objects +- [ ] `pathlib`; no hard-coded machine paths +- [ ] Dependencies and package data declared +- [ ] British English; no secrets or project data +- [ ] `black` clean; tests added ([SKILL-TESTING.md](./SKILL-TESTING.md)) diff --git a/.claude/skills/bhom-python/SKILL-REVIEW.md b/.claude/skills/bhom-python/SKILL-REVIEW.md new file mode 100644 index 0000000..8de689d --- /dev/null +++ b/.claude/skills/bhom-python/SKILL-REVIEW.md @@ -0,0 +1,253 @@ +--- +name: BHoM Python Compliance Review +description: Rule catalogue, severities, automated checks and report format for reviewing Python in Python_Toolkit. +version: 0.2.6 +parent_skill: bhom-python +author: BHoM +tags: [BHoM, Python, review, compliance, CI] +--- + +# BHoM Python Compliance Review + +> **Use when:** asked to review, audit or compliance-check Python - a PR diff, a folder, or the whole package. +> **Scope:** BHoM CI (`.github/workflows/ci-beta.yml`, via `BHoM/CI_Toolkit`) runs build and compliance checks aimed at C#; **no Python linting, tests or copyright checks run in CI.** This review is currently the only gate for Python. + +--- + +## Quick Navigation + +- [Review Workflow](#review-workflow) +- [Automated Pass](#automated-pass) +- [Rule Catalogue](#rule-catalogue) +- [Report Format](#report-format) +- [Copyright for Python Files](#copyright-for-python-files) +- [Proposed Tool Configuration](#proposed-tool-configuration) + +--- + +## Review Workflow + +1. **Scope** - review the diff (`git diff develop...HEAD -- "*.py"`) unless asked for more. Issues on untouched lines are listed as "existing", never as blockers for the PR. +2. **Automated pass** - run the tools below in the Python_Toolkit environment; report counts, not raw dumps. +3. **Manual pass** - walk the [Rule Catalogue](#rule-catalogue) by severity. Read the code; grep hits are leads, not findings. +4. **Verify each finding** - `file:line`, the concrete failure (input -> wrong result/crash), and the fix. Drop anything you cannot substantiate. +5. **Tests** - check against [SKILL-TESTING.md](./SKILL-TESTING.md#checklist); missing tests for new public behaviour is **Major**. +6. **Report** in the [format below](#report-format). Do not edit code during a review unless asked. + +Reviewer behaviour follows [bhom-coding-standards](../bhom-coding-standards/SKILL.md#ai-behavior-guidelines): concise, British English, no whitespace-only findings (black owns formatting), no suggestions where nothing needs to change. + +--- + +## Automated Pass + +From `Python_Engine/Python`, with the env's `python.exe`: + +```bash +python -m black --check --diff src tests +python -m pylint src --disable=C0114 --output-format=parseable +python -m pytest tests -q --cov=python_toolkit --cov-report=term-missing +``` + +Greps (each hit needs a human look): + +```bash +grep -rnE --include=*.py 'except\s*:' src # PY-E01 +grep -rnE --include=*.py '^\s*print\(' src # PY-B05 +grep -rnE --include=*.py 'def .*=\s*(\[\]|\{\}|[A-Za-z_.]+\(\))' src # PY-C01 (single-line signatures only) +grep -rnE --include=*.py '(^|[^A-Za-z])[A-Z]:[/\\][^/\\]' src # PY-S03 drive paths +grep -rnE --include=*.py 'shell=True|os\.system|eval\(|exec\(' src # PY-S02 +grep -rn --include=*.py 'bhom_callable' src | grep -v 'bhom/wrapped/\|bhom/decorators/' # PY-W02 +python -c "import pathlib,sys; [print(p) for p in pathlib.Path(sys.argv[1]).rglob('*.py') if p.read_bytes()[:3]==b'\xef\xbb\xbf']" src # PY-F04 BOM +grep -rL --include=*.py 'Buildings and Habitats object Model (BHoM)' src tests # PY-F06 files missing the header +``` + +pylint `W0102 dangerous-default-value` catches `[]`/`{}` defaults but **not** call defaults (`uuid.uuid4()`, `datetime.now()`) - check those by eye. + +--- + +## Rule Catalogue + +Severity: **Blocker** (fix before merge) / **Major** (fix in this PR unless the CI/CD lead agrees otherwise) / **Minor** (fix when touched) / **Info**. + +### C#-callable methods + +| ID | Sev | Rule | +|----|-----|------| +| PY-W01 | Blocker | C# calls Python **only** through `run_wrapped` (`-command -in `). No C# code runs ad hoc scripts, `RunPythonString` snippets, or bespoke per-method CLI arguments. | +| PY-W02 | Blocker | Every C#-callable method has `@bhom_wrapper.bhom_callable("", ...)` **and** lives under `python_toolkit/bhom/wrapped/`. A decorated method elsewhere is never registered. | +| PY-W03 | Major | `bhom/wrapped/` contains only C#-callable wrappers: thin, delegating to domain modules; module-level imports light and side-effect free (all are imported on every `run_wrapped` call). | +| PY-W04 | Major | Wrapped methods return Python objects and let `bhom_callable` encode them - no `json.dumps` in the method, and no `str` return when C# deserialises the result (strings are passed through unencoded). | +| PY-W05 | Blocker | No `print` (or other stdout output, including `CONSOLE_LOGGER.info`) in wrapped code paths - C# deserialises the whole of stdout. | +| PY-W06 | Major | Identifiers unique and stable; renaming one needs a C# deprecation path. | +| PY-W07 | Major | `argument_types` supplied for every argument that should arrive as a `BHoMObject` subclass; nested/list items converted explicitly. | +| PY-W08 | Major | Failures raise (non-zero exit, traceback on stderr, which C# detects) - no `try/except` that returns `traceback.format_exc()` or error text as the result. | + +### Correctness and errors + +| ID | Sev | Rule | +|----|-----|------| +| PY-C01 | Blocker | No mutable or call-evaluated defaults (`[]`, `{}`, `uuid.uuid4()`, `datetime.now()`) in signatures or dataclass fields | +| PY-C02 | Blocker | Logic does what its name/docstring says - check boolean conditions for always-true/false (e.g. `validate_timeseries` `is_annual`) | +| PY-C03 | Major | `dataclasses.field(default_factory=...)` is given a callable | +| PY-C04 | Major | Inputs not mutated unless documented in-place. **Exception:** methods in `bhom/wrapped/` may modify their inputs, which are copies deserialised from C# JSON | +| PY-C05 | Major | Floats compared with tolerance | +| PY-C06 | Major | Annotations match what is accepted/returned (`-> str` returning a dict; `x: str = None`) | +| PY-C07 | Minor | `is None` / `is not None`, never `== None` | +| PY-E01 | Blocker | No bare `except:` (`_bhom_callable_decorator.py`) | +| PY-E02 | Blocker | No swallowed errors: an `except` re-raises, or logs at `error` with `exc_info=True` *and* surfaces the failure | +| PY-E03 | Major | Specific exceptions, what-and-how-to-fix messages, chained with `from exc` | +| PY-E04 | Major | No `raise Exception(...)` | +| PY-E05 | Major | Inputs validated at public entry points | +| PY-E06 | Minor | No `assert` for runtime validation | + +### BHoM integration + +| ID | Sev | Rule | +|----|-----|------| +| PY-B02 | Major | BHoM objects use `BHoMObject`/`IObject` and `BHoMJSONEncoder`/`Decoder`; no hand-built `_t`/`BHoM_Guid` dicts or ad hoc key-case conversion | +| PY-B03 | Major | `_t` is the exact C# full type name, or `"BH.oM.Base.CustomObject"` when no C# type exists - never an invented name; attribute names round-trip to C# property names (test acronyms) | +| PY-B04 | Major | Public entry points (including wrapped methods) have `@bhom_analytics()` - with brackets, on functions/methods, not classes, private helpers or hot loops | +| PY-B05 | Major | `CONSOLE_LOGGER`, not `print`; no root-logger configuration; never log to `ANALYTICS_LOGGER` directly | +| PY-B06 | Major | No hard-coded BHoM paths where a constant or `Query` method exists; plot style via `plt.style.use("python_toolkit.bhom")` | +| PY-B07 | Major | `clr` only where needed, assemblies from `%ProgramData%\BHoM\Assemblies`, guarded for non-Windows | +| PY-B08 | Major | No import-time side effects (network, `atexit`, environment variables such as `HOME`, `sys.path`) | +| PY-B09 | Minor | Dependencies declared; pins justified; dev tools not in runtime dependencies | +| PY-B10 | Minor | Runs on the base environment's Python (3.10) | + +### Security and privacy + +| ID | Sev | Rule | +|----|-----|------| +| PY-S01 | Blocker | No secrets, tokens, connection strings, personal data, or project/client names in code, tests, assets or logs | +| PY-S02 | Blocker | No `eval`/`exec` on external input; no `shell=True`/`os.system` with interpolated values | +| PY-S03 | Major | No machine-specific absolute paths (`C:/Users/...`, network shares) | +| PY-S04 | Major | No committed artefacts: notebooks (`*.ipynb`), `.venv`, `__pycache__`, `.pytest_cache`, `.coverage`, `cov_html/`, data dumps | + +### Documentation and style + +| ID | Sev | Rule | +|----|-----|------| +| PY-D01 | Major | Type hints on every parameter and return of public functions/methods | +| PY-D02 | Major | Google-style docstrings on public modules, classes, functions; units for physical quantities; every default value documented ("Defaults to ..."); `Raises:` when it raises | +| PY-D03 | Minor | British English (third-party API names exempt) | +| PY-D04 | Minor | Docstrings match the signature; no placeholders or copy-paste | +| PY-F01 | Minor | `black`-clean (one finding, not per line) | +| PY-F02 | Minor | Absolute imports; no `import *`; no unused imports | +| PY-F03 | Minor | One public function/class per file; snake_case file names | +| PY-F04 | Minor | UTF-8 without BOM | +| PY-F05 | Info | No commented-out code; `TODO` has an issue link | +| PY-F06 | Major | BHoM copyright header at the top of every new or changed `.py` file (source and tests), with the current end year ([template](#copyright-for-python-files)) | + +### Tests + +| ID | Sev | Rule | +|----|-----|------| +| PY-T01 | Major | New/changed public behaviour has success-path and failure-path tests | +| PY-T02 | Major | Bug fixes ship with a regression test | +| PY-T03 | Major | Wrapped methods tested on both faces (Python call and `__input_json__`) and registration | +| PY-T04 | Major | Tests independent, deterministic, no absolute paths or network; external tools skipped cleanly | +| PY-T05 | Minor | Meaningful assertions with clear messages | + +--- + +## Report Format + +```markdown +## Python review - Python_Toolkit + +**Scope:** **Automated:** black , pylint , pytest + +### Blockers +1. **PY-W02** `src/python_toolkit/plot/heatmap_png.py:12` - `@bhom_callable` method outside `bhom/wrapped/`, so `run_wrapped("python_toolkit.plot.heatmap_png", ...)` raises `NotImplementedError`. + Fix: move the wrapper to `bhom/wrapped/plot/heatmap_png.py`; keep the logic in `plot/`. + +### Major +... + +### Minor / Info +... + +### Existing issues (not introduced by this PR) +... + +### Verdict +Request changes | Approve with comments | Approve +``` + +- Order by severity, then file. One finding per root cause; group repeats ("PY-B05 x12 in `plot/`"). +- If the tools could not be run, say so under **Automated** - never imply they passed. + +--- + +## Copyright for Python Files + +**Every `.py` file - source and tests - carries the BHoM copyright header**, as [bhom-copyright](../bhom-copyright/SKILL.md) requires for all BHoM files (PY-F06). + +- **Form:** the C# header text verbatim, as `#` comments, at the very top of the file, above the module docstring (and so above any `from __future__` import). +- **Year:** `2015 - `. When you modify a file, update the end year to the current year, exactly as bhom-copyright describes. +- **Existing files:** no `.py` file in Python_Toolkit has the header yet. Add it to every new file and to every existing file a PR changes. Missing headers on untouched files go under "Existing issues", not as blockers. +- **CI:** the BHoM copyright check does not cover Python, so this review is what enforces it. + +Template (use the current year): + +```python +# +# This file is part of the Buildings and Habitats object Model (BHoM) +# Copyright (c) 2015 - 2026, the respective contributors. All rights reserved. +# +# Each contributor holds copyright over their respective contributions. +# The project versioning (Git) records all such contribution source information. +# +# +# The BHoM is free software: you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License as published by +# the Free Software Foundation, either version 3.0 of the License, or +# (at your option) any later version. +# +# The BHoM is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this code. If not, see . +# + +"""Module docstring.""" +``` + +--- + +## Proposed Tool Configuration + +Python_Toolkit has no lint or format config. **Proposal** - add to `pyproject.toml` so people, Claude and a future CI job run the same checks: + +```toml +[tool.black] +line-length = 120 +target-version = ["py310"] + +[tool.pylint.main] +py-version = "3.10" +load-plugins = ["pylint.extensions.docparams"] # checks Args/Returns/Raises against signatures +disable = [ + "C0114", # module docstring - matches the README's VS Code setting + "R0902", "R0903", "R0904", "R0912", "R0913", "R0914", "R0915", # "too many ..." design checks +] + +[tool.pylint.basic] +good-names = ["i", "j", "k", "x", "y", "z", "df", "ax", "fig", "_"] + +[tool.pylint.format] +max-line-length = 120 + +[tool.pytest.ini_options] +pythonpath = "src" +testpaths = ["tests"] +markers = [ + "slow: long-running tests excluded from the default run", + "dotnet: needs pythonnet and BHoM assemblies", +] +``` + +Plus a CI step (Windows runner) running `black --check`, `pylint --fail-under=` and `pytest` on `Python_Engine/Python` when `*.py` changes - to raise with the CI/CD lead as an addition to `BHoM/CI_Toolkit`. diff --git a/.claude/skills/bhom-python/SKILL-TESTING.md b/.claude/skills/bhom-python/SKILL-TESTING.md new file mode 100644 index 0000000..22e5cf5 --- /dev/null +++ b/.claude/skills/bhom-python/SKILL-TESTING.md @@ -0,0 +1,234 @@ +--- +name: BHoM Python Testing +description: How to write, organise and run pytest tests for Python_Toolkit, including serialisation round trips, wrapped (C#-callable) methods, plots, UI and pythonnet. +version: 0.2.5 +parent_skill: bhom-python +author: BHoM +tags: [BHoM, Python, testing, pytest] +--- + +# BHoM Python Testing + +> **Use when:** writing tests for new or changed Python, or judging whether a PR's tests are sufficient. +> **Policy (`Python_Engine/Python/README.md`):** everything that can reasonably be tested should be; a passing unit test is required for PR approval; the code author writes the tests and the reviewer judges their sufficiency. Dispensation only from the CI/CD lead with justification. + +--- + +## Quick Navigation + +- [Quick Start](#quick-start) +- [Where Tests Live](#where-tests-live) +- [Running Tests](#running-tests) +- [What to Test](#what-to-test) +- [Patterns by Code Type](#patterns-by-code-type) +- [Test Quality Rules](#test-quality-rules) +- [Checklist](#checklist) + +--- + +## Quick Start + +```python +"""Tests for python_toolkit.helpers.degree_days.""" + +import pandas as pd +import pytest + +from python_toolkit.helpers.degree_days import heating_degree_days + + +def test_heating_degree_days_returns_expected_for_constant_temperature(): + # Arrange + index = pd.date_range("2026-01-01", periods=48, freq="h") + air_temperature = pd.Series(10.0, index=index) + + # Act + result = heating_degree_days(air_temperature, base_temperature=15.5) + + # Assert + assert result == pytest.approx(11.0) # 5.5 K x 2 days + + +def test_heating_degree_days_raises_for_non_series(): + with pytest.raises(TypeError, match="pandas Series"): + heating_degree_days([10.0, 11.0]) +``` + +- Framework: **pytest** only. +- Name: `test___` - same intent as the C# `Method_Scenario_Expected` convention in [bhom-coding-standards](../bhom-coding-standards/SKILL.md#testing-standards). +- Arrange / Act / Assert, one behaviour per test. +- Every `raise` in the code gets a `pytest.raises(..., match=...)` test. + +--- + +## Where Tests Live + +`Python_Engine/Python/tests/`, mirroring the source tree: + +``` +tests/ +├── __init__.py # shared constants (current practice, e.g. TIMESERIES_COLLECTION) +├── conftest.py # (proposed) fixtures, Agg backend, analytics off +├── assets/ # small, licence-safe sample data +├── test_bhom_serialiser.py +├── test_plot.py +└── ... # test_.py or test_/test_.py +``` + +- Sample data in `tests/assets/`, loaded via `Path(__file__).parent / "assets" / ...` - never absolute, user or network paths. +- No project or client data in assets - public or synthetic only. +- **Open question:** the README says tests go in `.ci/unit_tests/python/`, which holds only an empty, misspelt `__init_.py`. Practice is `Python_Engine/Python/tests/`; this skill follows practice and the README should be updated. + +--- + +## Running Tests + +From the repo, against source (`pyproject.toml` sets `pythonpath = "src"`): + +```bash +"C:/ProgramData/BHoM/Extensions/PythonEnvironments/Python_Toolkit/v3_10/python.exe" -m pytest Python_Engine/Python/tests --cov python_toolkit --cov-report term-missing -v +``` + +- `run_tests.bat` runs against the **deployed copy** in `%ProgramData%\BHoM\Extensions\PythonCode\Python_Toolkit` - rebuild `Python_Engine` first or you test stale code. +- Run with `DISABLE_BHOM_ANALYTICS=1` so tests do not write usage logs (set before any python_toolkit import). +- Coverage output (`.coverage`, `cov_html/`) and `.pytest_cache/` must not be committed; **(proposed)** add `cov_html/` to `.gitignore` (it ignores `htmlcov/` but `run_tests.bat` writes `cov_html`). + +`conftest.py` starter **(proposed)**: + +```python +"""Shared pytest configuration.""" + +import os + +os.environ.setdefault("DISABLE_BHOM_ANALYTICS", "1") # before python_toolkit imports + +import matplotlib # noqa: E402 +import matplotlib.pyplot as plt # noqa: E402 +import pytest # noqa: E402 + +matplotlib.use("Agg") # headless; avoids tkinter backend errors under pytest + + +@pytest.fixture(autouse=True) +def _close_figures(): + yield + plt.close("all") +``` + +--- + +## What to Test + +Minimum per change (README policy, made explicit): + +1. **Success path** returns the predefined expected result. +2. **Each raised error** is raised with the expected type and message fragment. +3. **Lowest-level functions first** in multi-step processes, so errors do not cascade untested. +4. **External programs** - assert availability first and `pytest.skip` with a reason if absent. +5. **Floating point / stochastic** - `pytest.approx` with deliberate `rel`/`abs`; never `==` on floats. +6. **Regression** - a bug fix ships with the test that would have caught it. + +--- + +## Patterns by Code Type + +### Wrapped (C#-callable) methods + +Test both faces of the method: + +```python +import json + +import python_toolkit.bhom.wrapped # noqa: F401 - registers every wrapped method, as run_wrapped does +from python_toolkit.bhom.decorators import bhom_wrapper +from python_toolkit.bhom.wrapped.plot.heatmap_png import heatmap_png + + +def test_heatmap_png_is_registered(): + assert bhom_wrapper.get_registered_method("python_toolkit.plot.heatmap_png") is not None + + +def test_heatmap_png_python_call_returns_path(tmp_path): + out = heatmap_png(values=[1.0] * 48, start="2026-01-01T00:00", output_path=str(tmp_path / "h.png")) + assert out.exists() + + +def test_heatmap_png_csharp_call_returns_bhom_json(tmp_path): + payload = json.dumps({"values": [1.0] * 48, "start": "2026-01-01T00:00", "output_path": str(tmp_path / "h.png")}) + + result = heatmap_png(__input_json__=payload) + + obj = json.loads(result) + assert obj["_t"] == "BH.oM.Adapter.FileSettings" +``` + +- **Registered:** the identifier resolves through `bhom_wrapper.get_registered_method` after `import python_toolkit.bhom.wrapped` - this catches a method placed outside `bhom/wrapped/` or missing the decorator. +- **Python face:** called normally, returns Python objects. +- **C# face:** called with `__input_json__` (a JSON string, and a file path via `tmp_path`), returns valid BHoM JSON with the expected `_t`. +- **Through `run_wrapped`:** with `capsys`, stdout is **exactly** one valid JSON document - C# deserialises the whole of stdout, so any extra output is a failure. +- **Inputs from C#:** include at least one argument given as a BHoM JSON string, exactly as the BHoM serialiser produces it, including any `argument_types` upgrade to your subclass. +- **Failure:** invalid input raises a clear exception (the process must exit non-zero, not print a traceback as the result). + +### BHoM serialisation + +```python +def test_file_settings_round_trips_through_bhom_json(): + obj = FileSettings(file_name="a.epw", directory="C:/data") + + restored = FileSettings.from_json(obj.to_json()) + + assert restored == obj # __eq__ ignores bhom_guid and _bhom_version +``` + +- **Python -> JSON -> Python**, and **C# JSON -> Python** from a BHoM JSON string produced by the BHoM serialiser - the second catches PascalCase/snake_case drift (acronyms). +- Assert `_t` is the exact C# type name (or `BH.oM.Base.CustomObject` for objects with no C# type). +- `test_bhom_serialiser.py` is the reference; its TODOs (use real BHoM-serialiser output) are worth closing. + +### Plots + +- `Agg` backend; assert return type (`plt.Axes` / `Figure`) and key properties (title, number of artists, colourbar present) - not pixels. +- Close figures after each test (fixture above). +- Saving: write to `tmp_path`; assert the file exists and is non-empty. + +### tkinter UI + +- `tk.Tk()`, `withdraw()`, drive widgets programmatically, `destroy()` in teardown. +- Skip when no display is available (catch `tk.TclError` -> `pytest.skip`), so headless CI does not fail. +- Test widget **state and callbacks**, not layout. + +### pythonnet / .NET + +- Mark `@pytest.mark.dotnet`; skip when `clr` or `%ProgramData%\BHoM\Assemblies` is unavailable. +- Test the JSON contract rather than live .NET objects. + +### Files and paths + +- `tmp_path` only; never write into the repo or `%ProgramData%` from a test. +- Where an API accepts `str | Path`, test both. + +--- + +## Test Quality Rules + +- **Independent:** no ordering dependencies. `pytest-order` is installed but should not be needed; shared setup is a fixture. +- **No shared mutable state:** module-level objects (`TIMESERIES_COLLECTION`) are read-only - copy before mutating. +- **Fast:** unit tests under ~1 s; mark slow ones `@pytest.mark.slow`. +- **Deterministic:** seed randomness (`np.random.default_rng(0)`); pass times explicitly. +- **Meaningful assertions:** "it didn't crash" is not sufficient except for labelled smoke tests. No placeholder docstrings (`"""_"""`). +- **Clear failures:** assertion messages where the cause would otherwise be unclear, as in `test_bhom_serialiser.py`. +- **No notebooks:** notebooks are never committed and never count as tests; turn checks into pytest tests. + +--- + +## Checklist + +- [ ] Every new/changed public function has a success-path and a failure-path test +- [ ] Bug fixes include a regression test +- [ ] Floats compared with `pytest.approx` +- [ ] Wrapped methods: registered, Python face, C# face (`__input_json__`), stdout clean, BHoM JSON input +- [ ] BHoM objects: round trip and a BHoM JSON string from the C# serialiser +- [ ] Plots use `Agg` and close figures; UI tests skip without a display +- [ ] No absolute paths, network paths, project data or secrets in tests or assets +- [ ] Test files carry the BHoM copyright header +- [ ] Analytics disabled for the test run +- [ ] `python -m pytest` passes in the Python_Toolkit environment diff --git a/.claude/skills/bhom-python/SKILL.md b/.claude/skills/bhom-python/SKILL.md new file mode 100644 index 0000000..36dabc1 --- /dev/null +++ b/.claude/skills/bhom-python/SKILL.md @@ -0,0 +1,119 @@ +--- +name: bhom-python +description: Write, test and compliance-review Python in Python_Toolkit, the BHoM Python base package. Covers package layout and packaging, C#-callable methods (bhom_wrapper.bhom_callable + run_wrapped, BHoM JSON outputs), BHoMObject/IObject serialisation, bhom_analytics, CONSOLE_LOGGER, pythonnet, pytest standards and a review rule catalogue. Use for any .py work in Python_Toolkit or when asked to review/audit its Python. Toolkit-specific Python skills build on this one. +version: 0.2.4 +author: BHoM +tags: [BHoM, Python, standards, testing, review] +--- + +# BHoM Python Skill + +## Role + +You are a senior Python developer working on Python_Toolkit, the base package every BHoM Python environment installs. You write Python that C# can call through the BHoM framework, follow the published BHoM Python guidance, and hold it to the same quality bar as BHoM C#: typed, documented, tested and reviewable. + +## Usage Guide + +1. Identify the task: **write**, **test**, or **review**. +2. Read the surrounding code first and follow its patterns unless the task is to change them. +3. Load only the sub-skill(s) for the phase (see [Quick Navigation](#quick-navigation)). +4. Apply the [Golden Rules](#golden-rules) every time. +5. Finish with the relevant checklist (Implementation, Testing or Review). +6. Communicate per [bhom-coding-standards](../bhom-coding-standards/SKILL.md#ai-behavior-guidelines): concise, British English, only requested changes, file-by-file. + +--- + +## Quick Navigation + +| I need to... | Load | +|--------------|------| +| Add or change code, packaging, style | [SKILL-IMPLEMENTATION.md](./SKILL-IMPLEMENTATION.md) | +| Make a method callable from C#; BHoM objects/JSON; logging; analytics; pythonnet | [SKILL-FRAMEWORK.md](./SKILL-FRAMEWORK.md) | +| Write or judge tests | [SKILL-TESTING.md](./SKILL-TESTING.md) | +| Review / compliance-check code or a PR | [SKILL-REVIEW.md](./SKILL-REVIEW.md) | +| C# side (oM, Engine) | [bhom-om](../bhom-om/SKILL.md), [bhom-engine](../bhom-engine/SKILL.md) | +| Debug | [bhom-debug](../bhom-debug/SKILL.md) | + +**Quick answers** + +- *Where does code go?* `Python_Engine/Python/src/python_toolkit//`; tests in `Python_Engine/Python/tests/`. +- *C# needs to call it?* A thin `@bhom_wrapper.bhom_callable("")` wrapper in `python_toolkit/bhom/wrapped/`, returning Python objects; C# calls it through `run_wrapped`. +- *Logging?* `CONSOLE_LOGGER`. Never `print`. +- *Docstrings?* Google. *Formatter?* black. *Tests?* pytest. *Python?* 3.10. + +--- + +## How Python Fits in BHoM + +``` + C# (UI / Engine / Adapter) + │ BH.Engine.Python.Compute.BasePythonEnvironment / VirtualEnvironment ─► %ProgramData%\BHoM\Extensions\PythonEnvironments\... + │ writes inputs as JSON to a temp file + │ RunCommandStdout("\python.exe .../bhom/run_wrapped.py -command -in ") + ▼ + run_wrapped ─► bhom_wrapper.get_registered_method() (registered by importing python_toolkit.bhom.wrapped) + ▼ + @bhom_callable wrapper: JSON ─(BHoMJSONDecoder)─► kwargs ─► method ─► return value ─(BHoMJSONEncoder)─► stdout + ▼ + C#: BH.Engine.Serialiser.Convert.FromJson(stdout) + + Python ──(pythonnet / clr)──► %ProgramData%\BHoM\Assemblies\*.dll (Python calling BHoM .NET) +``` + +- The build copies `Python_Engine/Python` to `%ProgramData%\BHoM\Extensions\PythonCode\Python_Toolkit`, and the environment installs it from there. **Rebuild to deploy Python changes.** +- Every BHoM Python environment installs python_toolkit first, so its APIs (`bhom_callable`, `BHoMObject`, loggers, analytics, plots, UI) are shared infrastructure. Breaking changes ripple to every toolkit. + +--- + +## Golden Rules + +Each maps to rule IDs in [SKILL-REVIEW.md](./SKILL-REVIEW.md#rule-catalogue). + +1. **C# calls Python only through `run_wrapped`.** C#-callable methods carry `@bhom_wrapper.bhom_callable(...)`, live under `python_toolkit/bhom/wrapped/`, and stay thin. (PY-W01-W03) +2. **Return objects, not JSON.** `bhom_callable` converts the output to BHoM JSON when called from C#; the same method returns plain objects when called from Python. (PY-W04) +3. **Stdout is the API.** In wrapped code paths, nothing but the result reaches stdout; failures raise. (PY-W05, PY-W08) +4. **Use the framework.** BHoM JSON, logging, analytics, plot styles and Tk UI come from `python_toolkit.bhom*` - do not re-implement them. (PY-B02, PY-B05) +5. **Errors are shown, never swallowed.** No bare `except`; specific exceptions with what-and-how-to-fix messages. (PY-E01-E04) +6. **Typed and documented.** Type hints on every signature; Google docstrings with units and `Raises`. (PY-D01, PY-D02) +7. **No hidden state.** No mutable or call-evaluated defaults, no import-time side effects, no input mutation outside `bhom/wrapped/`. (PY-C01, PY-B08, PY-C04) +8. **Tested.** Success-path and failure-path pytest tests; wrapped methods tested from both the Python and the C# side. (PY-T01-T03) +9. **No machine or project specifics.** No hard-coded user/drive paths, secrets, or project/client data. (PY-S01, PY-S03) + +--- + +## Decision Tree + +``` +Will C# call it? +├── Yes ─► domain function in its sub-package (SKILL-IMPLEMENTATION) +│ + thin @bhom_callable wrapper in bhom/wrapped/ (SKILL-FRAMEWORK) +│ └── BHoM objects in or out? ─► BHoMObject/IObject subclass + argument_types +└── No ─► domain function in its sub-package (SKILL-IMPLEMENTATION) +Then ─► tests (SKILL-TESTING) ─► self-review with the SKILL-REVIEW checklist +``` + +--- + +## Related Skills + +- [bhom-coding-standards](../bhom-coding-standards/SKILL.md) - AI behaviour, priority order (performance > security > maintainability > readability), error-recording semantics. This skill maps those to Python and does not repeat the C# formatting rules. +- [bhom-copyright](../bhom-copyright/SKILL.md) - the header text and year rules. Python files carry the same header as `#` comments ([SKILL-REVIEW.md](./SKILL-REVIEW.md#copyright-for-python-files)). +- [bhom-develop](../bhom-develop/SKILL.md) - overall assess / plan / implement / review workflow; use this skill for the Python steps. +- **Toolkit-specific Python skills** (one per toolkit) build on this skill and add only that toolkit's conventions. Where they are stricter, they win. + +--- + +## References + +- Published BHoM Python guidance: `Python_Engine/Python/README.md` +- BHoM code compliance: https://github.com/BHoM/documentation/wiki/Code-Compliance +- BHoM coding style: https://github.com/BHoM/documentation/wiki/Coding-Style +- PEP 8: https://peps.python.org/pep-0008/ +- Google docstrings: https://google.github.io/styleguide/pyguide.html#383-functions-and-methods +- pytest: https://docs.pytest.org/ + +--- + +## Version History + +**See [CHANGELOG.md](./CHANGELOG.md) for full version history.**