Skip to content

fix: harden AI Studio JSONL upload checks - #16

Open
YOIMIYA66 wants to merge 1 commit into
liuyunlin:mainfrom
YOIMIYA66:codex/medrecord-run-hardening
Open

fix: harden AI Studio JSONL upload checks#16
YOIMIYA66 wants to merge 1 commit into
liuyunlin:mainfrom
YOIMIYA66:codex/medrecord-run-hardening

Conversation

@YOIMIYA66

Copy link
Copy Markdown
Contributor

Summary

This PR turns the MedRecord-CN ERNIE SFT run findings into reusable safeguards for the cli-trainer skill.

  • add a --upload-plain-file fallback that writes JSON/JSONL training files through the AI Studio Git contents API
  • optionally remove .gitattributes JSON/JSONL LFS rules before uploading plain training files
  • make --verify-upload --strict-lfs detect Git LFS pointer text, not only the is_lfs field
  • document the root-filename workaround for trainDataFiles 指定的文件不存在
  • parse PaddleFormers learning_rate: log lines so artifact export can produce LR charts

Why

During the MedRecord-CN full SFT run, the SDK/CLI upload path could leave a JSONL file as an LFS pointer or make the training mount fail even when the Git API could see the file. The reliable path was to remove JSON/JSONL LFS rules, upload the actual JSONL content as a plain file, verify pointer content and size, then submit with a short root filename such as train.jsonl.

Validation

  • python -m py_compile scripts/train.py
  • python scripts/train.py --help
  • helper smoke check for LFS pointer detection and learning_rate metric parsing
  • git diff --check HEAD~1 HEAD
  • token scan for exposed AI Studio credentials

@YOIMIYA66
YOIMIYA66 marked this pull request as ready for review May 20, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant