Skip to content

Keep existing L0.5 files when replacing them fails - #150

Open
namurphy wants to merge 1 commit into
PlasmaPy:mainfrom
namurphy:l05-replace-atomically
Open

namurphy wants to merge 1 commit into
PlasmaPy:mainfrom
namurphy:l05-replace-atomically

Conversation

@namurphy

Copy link
Copy Markdown
Member

This is the decommutator counterpart of #149, which makes the same change for l05_to_l1.

Problem

With overwrite=True (or --overwrite), l0_to_l05 deleted an existing L0.5 CDF file before creating the new one. If filling the new file then failed, the partial file was deleted too, so no L0.5 file was left for that APID. If the run was interrupted (for example, by Ctrl-C) or killed while writing, an incomplete file was left at the final path, where it could be mistaken for a complete one.

Change

  • Each L0.5 CDF is now written at a hidden temporary path in the same directory, like .0523462910_4_EA_APID352_L1.<random hex>.tmp.cdf. It's moved over the final path with Path.replace only once it has been filled and closed. That means:
    • an existing file is only ever replaced by a complete one;
    • nothing with the final name is ever incomplete.
  • If creating, filling, closing, or renaming the file fails, including with KeyboardInterrupt, the temporary file is closed and deleted, and the exception is still recorded or raised as before. _discard_partial_cdf logs any error from closing or deleting it rather than letting that error replace the original one.
  • temporary_path is added to pyfaradaycup.utils._cdf, next to close_cdf.
  • The failure message for a CDF that couldn't be filled no longer says (deleted). The partial file is now only ever the temporary one, and any existing file is kept.
  • In the decommutator README, the item "Write each CDF to a temporary file, and rename it when it is complete" is now marked as done.

The CDF contents are unchanged. The tests that compare the output with the IDL L0.5 files pass.

Testing

  • nox -s tests-3.9, nox -s tests-3.14, nox -s dagster_with_coverage, nox -s typecheck, nox -s lint, and nox -s docs pass.
  • New tests:
    • With overwrite, a fill that fails, and a fill interrupted by KeyboardInterrupt, both leave the existing CDF unchanged byte for byte, with no temporary files left behind.
    • A successful overwrite leaves only the final file.
    • Without an existing file, a failed fill or a failed close leaves the APID directory empty.
    • temporary_path returns a hidden path in the same directory that ends in .cdf and is different each time.

Notes

🤖 Generated with Claude Code

With overwrite, an existing L0.5 CDF file was deleted before the new
one was created, so if filling the new one failed or was interrupted,
no file was left for that APID. A run that was interrupted while
writing also left an incomplete file with the final name. Each L0.5
CDF file is now written at a temporary path in the same directory and
renamed over the final path only once it is complete.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 68.57143% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.97%. Comparing base (c66036e) to head (f712087).

Files with missing lines Patch % Lines
src/pyfaradaycup/decommutator/_l0_to_l05.py 63.33% 10 Missing and 1 partial ⚠️

❌ Your patch check has failed because the patch coverage (68.57%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #150      +/-   ##
==========================================
- Coverage   76.19%   75.97%   -0.23%     
==========================================
  Files          15       15              
  Lines        1420     1440      +20     
  Branches      200      201       +1     
==========================================
+ Hits         1082     1094      +12     
- Misses        298      305       +7     
- Partials       40       41       +1     
Files with missing lines Coverage Δ
src/pyfaradaycup/utils/_cdf.py 100.00% <100.00%> (ø)
src/pyfaradaycup/decommutator/_l0_to_l05.py 72.12% <63.33%> (-0.82%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment on lines +571 to +574
except BaseException:
# Such as KeyboardInterrupt
_discard_partial_cdf(cdf, tmp_path)
raise

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should start doing this in all of my other projects. 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant