Skip to content

[FIX]: Preserve reports with duplicate timestamps - #179

Open
Rio Yu (rioyu123) wants to merge 2 commits into
microsoft:mainfrom
rioyu123:codex/fix-report-filename-collisions
Open

[FIX]: Preserve reports with duplicate timestamps#179
Rio Yu (rioyu123) wants to merge 2 commits into
microsoft:mainfrom
rioyu123:codex/fix-report-filename-collisions

Conversation

@rioyu123

@rioyu123 Rio Yu (rioyu123) commented Aug 27, 2026

Copy link
Copy Markdown

Description

JsonFileReportSink used a seconds-only timestamp and write_text(), so a second report emitted to the same directory within that second replaced the first.

Following the review, filenames now combine a UTC millisecond timestamp with a UUID. Files are opened exclusively ("x"), so even a UUID collision cannot overwrite an existing report. The suffix loop, retry limit, and helper are removed. Serialization still completes before the output file is opened.

Updated the filename documentation and tests for repeated timestamps, legacy reports, forced UUID collisions, and serialization failures.

Validation:

  • uv run pytest tests/unit/reporting -q — 46 passed.
  • uv run pytest tests/unit -q — 1033 passed, 2 skipped.
  • uv run pre-commit run --all-files — passed.
  • uv run mkdocs build --strict — passed on Linux.

Breaking changes

Generated filenames now use run_report_<UTC timestamp with milliseconds>_<uuid>.json. Consumers relying on the exact old filename format should use run_report_*.json; names do not imply ordering within the same millisecond. The report JSON and ReportSink API are unchanged.

Checklist

  • pre-commit run --all-files passes
  • Tests added or updated for changes
  • Documentation updated

@rioyu123
Rio Yu (rioyu123) requested a review from a team August 27, 2026 09:15
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@rioyu123

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

@rioyu123
Rio Yu (rioyu123) force-pushed the codex/fix-report-filename-collisions branch from 7a7f44e to 489675a Compare August 27, 2026 12:59

@nina-msft Nina Chikanov (nina-msft) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for flagging this Rio Yu (@rioyu123)! Let me know what you think of the following comments :-)

Comment thread rampart/reporting/json_file.py Outdated
"""
self._output_dir.mkdir(parents=True, exist_ok=True)

timestamp = datetime.now(UTC).strftime("%Y-%m-%dT%H-%M-%S")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to also include milliseconds in the timestamp, to make it more concise and reduce the likelihood of collisions:

Suggested change
timestamp = datetime.now(UTC).strftime("%Y-%m-%dT%H-%M-%S")
timestamp = datetime.now(UTC).strftime("%Y-%m-%dT%H-%M-%S-%f")[:-3]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you update this, please also update docs that reference this filename :-)

  • rampart/reporting/json_file.py:48-50 — class docstring example.

  • docs/usage/results-and-reporting.md:90 — documented output example.

  • tests/unit/reporting/test_json_file.py:371-426 — exact filename expectations.

Comment thread rampart/reporting/json_file.py Outdated
Created automatically if it does not exist.
"""

_MAX_FILENAME_ATTEMPTS: int = 1000

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1000 as a max value seems excessive for this change, lets reduce to 10? Unless you are using highly concurrent multi-process writers, in which case 100 would be safer.

Comment thread rampart/reporting/json_file.py Outdated
content=json.dumps(data, indent=2, default=str),
)

def _write_report_file(self, *, stem: str, content: str) -> None:

@nina-msft Nina Chikanov (nina-msft) Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it important to you for the file names to be ordered by which was generated first within the timestamp?

If not, maybe consider this approach which provides a more concise solution that still avoids overwrites (a readable millisecond timestamp plus a UUID):

from uuid import uuid4

timestamp = datetime.now(UTC).strftime("%Y-%m-%dT%H-%M-%S-%f")[:-3]
filepath = self._output_dir / f"run_report_{timestamp}_{uuid4().hex}.json"
with filepath.open("x", encoding="utf-8") as report_file:
    report_file.write(json.dumps(data, indent=2, default=str))

This removes the suffix loop, attempt limit, and helper method. open("x") still guarantees no existing file is overwritten. File names look like:

run_report_2026-08-27T13-33-17-840_a3f18c92654d4b75ad15687d383d951b.json
run_report_2026-08-27T13-33-17-840_647e96193ec34b19a54eaad43fd82151.json

Alternatively: Microseconds or time.time_ns() alone are shorter, but neither guarantees uniqueness across concurrent processes. Timestamp plus UUID is the clearest robust option.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense; there's no need to preserve ordering within a millisecond here. I've switched to a UTC millisecond timestamp plus uuid4().hex and removed the suffix loop, retry limit, and helper.

The file is still opened with "x", so even a forced UUID collision can't replace an existing report. Serialization happens before opening the file. Tests cover repeated timestamps, the UTC argument, legacy files, forced collisions, and serialization failures; the filename docs are updated too.

Validation: all 1,033 unit tests passed (2 skipped), pre-commit passed, and the strict docs build passed on Linux.

@nina-msft
Nina Chikanov (nina-msft) dismissed their stale review August 28, 2026 00:49

Whoops - meant to comment only instead of approval :-)

Signed-off-by: Rio Yu <52408936+rioyu123@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants