fix: Export silently yields zero GDR/DTR rows when result fits in initial QSqlQueryModel batch (closes #177)#179
Merged
Conversation
…tial batch (closes noonchen#177) `getDatalogForReport()` is a generator that the Excel exporter uses to stream GDR/DTR rows out of the `tmodel_datalog` Qt model. The previous shape was while model.canFetchMore(): model.fetchMore() while row < model.rowCount(): ... yield datalogRow `QSqlQueryModel` lazily fetches rows in batches (default 256). After `setQuery()` returns, the model is already populated with up to 256 rows, and `canFetchMore()` only returns `True` if there are *more* rows beyond that batch. So whenever the entire query result fits in the initial batch — which is the common case for GDR/DTR records, typically a few per device times a few devices — the outer loop never enters and the generator yields zero rows. The exporter then writes only the three column headers and the GDR & DTR Summary sheet looks blank, even though the option is checked and the data is in the database. The converter path (`DatabaseFetcher.getDTR_GDRs`) executes `DATALOG_QUERY` directly against the SQLite cursor and is unaffected, which is why "Convert STDF to XLSX" from the Utility menu produces the expected output for the same file. The fix reorders the loop to "yield current rows first, then fetch more, repeat until exhausted": while True: while row < model.rowCount(): ... yield datalogRow if not model.canFetchMore(): break model.fetchMore() Streaming behaviour is preserved for large result sets, and the small- result-set case now yields the rows that are already in the model before checking for more. Reproduction & verification (with a `FakeModel` mimicking `QSqlQueryModel`'s 256-row prefetch): total= 0 | old yielded= 0 [ok ] | new yielded= 0 [ok] total= 1 | old yielded= 0 [BUG] | new yielded= 1 [ok] total= 5 | old yielded= 0 [BUG] | new yielded= 5 [ok] total= 100 | old yielded= 0 [BUG] | new yielded= 100 [ok] total= 256 | old yielded= 0 [BUG] | new yielded= 256 [ok] total= 257 | old yielded= 257 [ok ] | new yielded= 257 [ok] total= 500 | old yielded= 500 [ok ] | new yielded= 500 [ok] total=1000 | old yielded=1000 [ok ] | new yielded=1000 [ok] `getDatalogForReport` is only called from `stdfExporter.getTableDataFromParent` in `deps/uic_stdExporter.py`, so the scope is limited to the Export path; the in-GUI GDR & DTR table already iterates the model directly and is unaffected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
noonchen
requested changes
Jun 21, 2026
noonchen
left a comment
Owner
There was a problem hiding this comment.
Code changes seem good, only need to update the comment :)
Collaborator
|
@seam0814 could you please update the comments as suggested? thanks! |
Per PR noonchen#179 feedback: keep only the first sentence inline since the 256-row prefetch root cause is already described in the PR body, and restore the `# method 1` commented-out block to minimize diff. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
Author
|
Updated and pushed (db25269). Inline comment trimmed and the |
noonchen
approved these changes
Jun 25, 2026
Owner
|
@seam0814 thanks for your contribution! merged. The windows build error might be related to msvc version or other reasons, ignored for now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
getDatalogForReport()is a generator the Excel exporter uses tostream GDR/DTR rows out of the
tmodel_datalogQt model. Theprevious shape silently yielded zero rows whenever the entire
query result fit in
QSqlQueryModel's initial prefetch batch(default 256 rows). DTR/GDR records in a typical STDF file are well
under 256, so the GDR & DTR Summary sheet always came out blank even
though the data was in the DB — exactly the symptom in #177. The
"Convert STDF to XLSX" path goes through
DatabaseFetcher.getDTR_GDRswhich executes the SQL directly, and so was unaffected.
Closes #177.
Root cause
QSqlQueryModelpre-populates up to 256 rows onsetQuery().canFetchMore()returnsTrueonly if there are more rows beyondthat initial batch. So for small result sets the outer loop never
runs and the generator yields nothing.
Change
Yield current rows first, then fetch more, repeat until exhausted.
Streaming behaviour is preserved for large result sets; the small-
result-set case now drains the rows that are already in the model.
Verification
Reproduced and verified deterministically using a
FakeModelthatmimics
QSqlQueryModel's 256-row prefetch:(The repo has no automated test harness, so this was static
reproduction; if you have a sample STDF with DTR records that
previously reproduced #177, the same file should now produce a
populated
GDR & DTR Summarysheet on Export.)Scope
getDatalogForReportis only called fromstdfExporter.getTableDataFromParentindeps/uic_stdExporter.py, so the change is scoped to the Exportpath. The in-GUI GDR & DTR table iterates the model directly via
tmodel_datalogand was not affected by the bug.