Repository navigation
Conversation
With overwrite, an existing L1 file was deleted before the new one was created, and if writing the new one then failed, the partial file was deleted too, leaving no L1 file for that day. Each L1 file is now written at a temporary path in the same directory and renamed over the existing file only once it is complete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #149 +/- ##
==========================================
+ Coverage 76.19% 76.45% +0.26%
==========================================
Files 15 15
Lines 1420 1423 +3
Branches 200 199 -1
==========================================
+ Hits 1082 1088 +6
+ Misses 298 295 -3
Partials 40 40
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #136, from its ultrareview.
Problem
With
-o/--overwrite,_read_master_cdfdeleted the existing L1 file before the new one was created. If writing the new file then failed (for example, a CDF library error, a full disk, or Ctrl-C), the cleanup incdf_merge_to_dailydeleted the partial file too. That left no L1 file at all for that day.Change
.<name>.<random hex>.tmp.cdf, and moved over the final path withPath.replaceonly once it is complete. The new file therefore replaces the old one in a single step, and if anything fails, the old file is left unchanged and the temporary file is deleted..so that it is hidden, and ends with.cdfbecause the CDF library adds.cdfto names that don't end with it.Logical_file_idis still set to the final file name, which is now passed to_write_data_to_cdfexplicitly.--overwritenow happens incdf_merge_to_daily, before anything is written.read_master_cdfcopies the skeleton over the existing file.The output files themselves are unchanged.
Testing
nox -s tests-3.9,nox -s tests-3.14,nox -s typecheck,nox -s lint, andnox -s docspass. That includes the comparison with the IDL L1 files for 2026-08-03, which checksLogical_file_id.overwrite, a failure after the data has been written (aCDFErroror aKeyboardInterrupt) leaves the existing file unchanged and no temporary files behind.Limitations
--overwrite, the check for an existing file and the rename aren't one atomic step. A file created by another process in between would be replaced.🤖 Generated with Claude Code