Description
Currently, logs only output to stdout. We should add the ability to log to a file
and automatically save that log file to MLflow for better debugging and analysis.
Context
When runs encounter errors or need debugging, having a complete log file saved in
MLflow would be valuable for post-mortem analysis and tracking training history.
Requirements
-
Configure Python's logging library to write to both stdout and a file
- Research the logging library's configuration for multiple handlers
- Add a file handler in addition to the existing stream handler
-
Add log file path configuration
- Option 1: Add a
log_file_path parameter to the run configuration
- Option 2: Use a default path convention (e.g.,
logs/{experiment_name}/{run_name}.log)
-
Integrate with simplexity run manager
- Ensure log file is saved to MLflow in the cleanup/teardown phase
- Include log file upload even when runs fail/error out
- Log file should be available as an MLflow artifact
Acceptance Criteria
Notes
The log file path needs to be known by the run manager's cleanup function to ensure
upload happens reliably.
Description
Currently, logs only output to stdout. We should add the ability to log to a file
and automatically save that log file to MLflow for better debugging and analysis.
Context
When runs encounter errors or need debugging, having a complete log file saved in
MLflow would be valuable for post-mortem analysis and tracking training history.
Requirements
Configure Python's logging library to write to both stdout and a file
Add log file path configuration
log_file_pathparameter to the run configurationlogs/{experiment_name}/{run_name}.log)Integrate with simplexity run manager
Acceptance Criteria
Notes
The log file path needs to be known by the run manager's cleanup function to ensure
upload happens reliably.