Skip to content

Harden DownloadSecFilingsStage: validate_config + injection-safe command building - #14

Open
Anilreddy2309 wants to merge 1 commit into
NVIDIA:mainfrom
Anilreddy2309:security/download-sec-filings-safe-cmd-and-validation
Open

Anilreddy2309 wants to merge 1 commit into
NVIDIA:mainfrom
Anilreddy2309:security/download-sec-filings-safe-cmd-and-validation

Conversation

@Anilreddy2309

Copy link
Copy Markdown

Summary

DownloadSecFilingsStage.execute() (nvflow/recipes/finance/stages/download/download_sec_filings.py) had two gaps relative to the rest of the finance recipe's stages:

1. No validate_config() override. A workflow YAML missing output_dir/sec_identity_email/sec_identity_company/tickers/start_year/end_year raised a bare KeyError with no indication of which field (or which YAML file) to fix. Added validate_config() following the pattern already used elsewhere in this recipe (e.g. train_validation_split.py), including the alternate "config" file path where tickers/start_year/end_year come from a separate referenced YAML instead of the stage config directly.

2. Command built via raw f-string interpolation instead of the repo's own safe helper. The shell command was built with manual f"..." string concatenation and hand-placed double quotes, instead of nvflow.lib.cli_cmd.build_python_cmd — the shlex.quote-based helper this repo already uses for every other stage that submits a command this way (validate_questions, data_transformation, apply_prompt_template, convert_to_responses_api, prepare_data, prefetch_cache). A ticker or company name containing backticks or $(...) was not safely escaped under the old approach. Migrated to build_python_cmd, which also standardizes on python3 (this repo's documented interpreter convention per nvflow/lib/cli_cmd.py's own docstring) instead of the unversioned python entry point the old code used.

Test plan

Added tests/test_download_sec_filings.py:

  • validate_config coverage for every required field, both the direct-config and config-file-reference paths
  • execute() tests that stub nemo_skills.pipeline.cli in sys.modules (the stage module doesn't import nemo_skills at module level, only inside execute(), so this works without the real dependency) to capture the exact rendered shell command
  • A regression test asserting that shell metacharacters in a ticker value (AAPL; rm -rf /, $(whoami)) are safely single-quoted in the rendered command rather than being interpretable by the shell

Verified:

  • ruff check / ruff format --check pass
  • mypy passes on both files
  • New suite passes (12/12) in an environment mirroring the exact CI install (pip install -e . --no-deps + tests/requirements-ci.txt, no nemo-skills/torch)
  • Full pytest tests/ suite passes (351 passed, 3 pre-existing skips) — no regressions

🤖 Generated with Claude Code

…and building

DownloadSecFilingsStage.execute() had two gaps relative to the rest of
the finance recipe's stages:

1. No validate_config() override, so a workflow YAML missing
   output_dir/sec_identity_email/sec_identity_company/tickers/
   start_year/end_year raised a bare KeyError with no indication of
   which field or file to fix. Add validate_config() following the
   pattern used elsewhere in this recipe (e.g.
   train_validation_split.py), including the "config" file alt-path
   where tickers/start_year/end_year come from a separate YAML instead.

2. The shell command was built via raw f-string interpolation with
   manual double-quotes instead of nvflow.lib.cli_cmd.build_python_cmd,
   the shlex.quote-based helper this repo already uses for every other
   stage that submits a command this way (validate_questions,
   data_transformation, apply_prompt_template,
   convert_to_responses_api, prepare_data, prefetch_cache). A ticker
   or company name containing backticks or $(...) would not have been
   safely escaped under the old f-string approach. Migrate to
   build_python_cmd, which also standardizes on python3 (this repo's
   documented interpreter convention) instead of the unversioned
   python entry point the old code used.

Add tests/test_download_sec_filings.py: validate_config coverage for
every required field (direct-config and config-file-reference paths),
plus execute() tests that stub nemo_skills.pipeline.cli in
sys.modules (the module doesn't import nemo_skills at module level, so
this works in the lightweight CI environment) to capture the exact
rendered command -- including a regression test asserting that shell
metacharacters in a ticker value are safely single-quoted rather than
interpreted by the shell.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Anil Balireddy <anilbalireddi@gmail.com>
@Anilreddy2309
Anilreddy2309 requested a review from a team September 11, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant