Skip to content

read_tables: tell a data-lake empty result from a really empty table - #7

Merged
nikolaos-paximadakis-eea merged 2 commits into
mainfrom
fix/read-tables-lake-detection
Nov 20, 2026
Merged

nikolaos-paximadakis-eea merged 2 commits into
mainfrom
fix/read-tables-lake-detection

Conversation

@nikolaos-paximadakis-eea

Copy link
Copy Markdown
Collaborator

When the ETL export has no rows at all, read_tables (and read_table, read_records, read_all_records) now looks up the dataflow's bigData flag. That's one extra call, made only in this case.

  • Data-lake dataflow: a definite RuntimeWarning pointing to export_to_file(). The plain route never returns rows there.
  • Not a data lake: no warning, because the tables really are empty. Before, this case showed a misleading "may be a data lake" warning.
  • Lookup fails (API or transport error): the previous warning is kept, and the rows already read are returned.

The tests cover all three cases, plus a check that nothing is looked up when rows do arrive. Two existing tests that used the empty-result warning as a trigger now mock a data-lake answer. The reference docs and the CHANGELOG ("Unreleased") are updated.

🤖 Generated with Claude Code

An empty ETL export was ambiguous: a data-lake dataflow answers 200 with 0 rows
even when data exists, so every empty result warned "may be a data lake", also
for tables that really are empty. Now an all-empty result looks up the
dataflow's bigData flag (one extra call, only in that case): a lake gets a
definite warning pointing to export_to_file(), a non-lake dataflow gets none,
and a failed lookup keeps the old warning without failing the read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only an explicit boolean bigData decides; an empty 200 (the route swallows
its own errors), a body without bigData or a non-JSON body keeps the vague
warning instead of silencing it. The lookup passes the caller's provider_id
(reporters need it) and never raises (ValueError from a non-JSON body is
caught too).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nikolaos-paximadakis-eea
nikolaos-paximadakis-eea merged commit 2f9104a into main Nov 20, 2026
6 checks passed
@nikolaos-paximadakis-eea
nikolaos-paximadakis-eea deleted the fix/read-tables-lake-detection branch November 20, 2026 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant