fix(cli): respect ignore files in adk deploy commands#4187
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a great improvement by making the adk deploy commands respect various ignore files (.gitignore, .gcloudignore, .ae_ignore). The implementation is clean, centralizing the logic in a new _get_ignore_patterns_func helper. This function is then correctly applied to to_cloud_run, to_agent_engine, and to_gke deployment functions. The refactoring in to_agent_engine is particularly nice as it replaces specific logic with the new generic helper. The addition of comprehensive unit tests in test_cli_deploy_ignore.py ensures the new functionality is well-tested and robust.
I have one suggestion to make the error handling in the new helper function more specific and robust. Overall, this is a solid contribution.
| try: | ||
| with open(filepath, 'r') as f: | ||
| for line in f: | ||
| line = line.strip() | ||
| if line and not line.startswith('#'): | ||
| # If it ends with /, remove it for fnmatch compatibility | ||
| if line.endswith('/'): | ||
| line = line[:-1] | ||
| patterns.add(line) | ||
| except Exception as e: | ||
| click.secho(f'Warning: Failed to read {filename}: {e}', fg='yellow') |
There was a problem hiding this comment.
This try...except block can be made more robust.
- Specify file encoding: It's good practice to explicitly specify the file encoding when opening text files to ensure consistent behavior across different systems.
utf-8is a safe default for ignore files. - Use specific exceptions: Catching the generic
Exceptionis too broad and can hide unexpected bugs. It's better to catch more specific exceptions, such asOSErrorfor file I/O problems andUnicodeDecodeErrorwhen an encoding is specified.
| try: | |
| with open(filepath, 'r') as f: | |
| for line in f: | |
| line = line.strip() | |
| if line and not line.startswith('#'): | |
| # If it ends with /, remove it for fnmatch compatibility | |
| if line.endswith('/'): | |
| line = line[:-1] | |
| patterns.add(line) | |
| except Exception as e: | |
| click.secho(f'Warning: Failed to read {filename}: {e}', fg='yellow') | |
| try: | |
| with open(filepath, 'r', encoding='utf-8') as f: | |
| for line in f: | |
| line = line.strip() | |
| if line and not line.startswith('#'): | |
| # If it ends with /, remove it for fnmatch compatibility | |
| if line.endswith('/'): | |
| line = line[:-1] | |
| patterns.add(line) | |
| except (OSError, UnicodeDecodeError) as e: | |
| click.secho(f'Warning: Failed to read {filename}: {e}', fg='yellow') |
|
Hi @kotaitos, Thank you for your contribution! We appreciate you taking the time to submit this pull request. Your PR has been received by the team and is currently under review. We will provide feedback as soon as we have an update to share. |
|
Hi @wukath , can you please review this. LGTM. |
|
Hi, @ryanaiagent, Thanks for the review! I'm looking forward to your feedback. |
fc259c3 to
bb41f59
Compare
|
Hi @ryanaiagent and @wukath, I've rebased this PR onto the latest Ready for another look. Thanks! |
|
@kotaitos This looks good! Can you rebase from main and fix conflicts before we can merge? |
b5ad9c8 to
397604c
Compare
The adk deploy commands (cloud_run, agent_engine, gke) were not properly respecting .gitignore, .gcloudignore, or .ae_ignore files, causing unwanted files (like venv, .git, etc.) to be uploaded. This change: - Adds a unified _get_ignore_patterns_func helper that reads all three ignore files. - Updates to_cloud_run, to_agent_engine, and to_gke to use this helper. - Removes hardcoded ignore patterns to strictly follow user configuration. - Adds comprehensive unit tests to verify the fix. Fixes google#4183
397604c to
15f6d62
Compare
|
@haranrk During the rebase, I resolved a conflict in I also updated the unit tests in All tests are now passing successfully. Ready for another review/merge! |
Merge #4187 **Please ensure you have read the [contribution guide](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) before creating a pull request.** ### Link to Issue or Description of Change **1. Link to an existing issue (if applicable):** - Closes: #4183 **Problem:** The `adk deploy` commands (`cloud_run`, `agent_engine`, `gke`) were not properly respecting `.gitignore`, `.gcloudignore`, or `.ae_ignore` files. This caused all files in the source directory, including large or sensitive ones like `venv/`, `.git/`, and `.env`, to be copied to the temporary staging directory and subsequently uploaded to hosted environments. **Solution:** - Implemented a unified `_get_ignore_patterns_func` helper in `src/google/adk/cli/cli_deploy.py` that reads and combines patterns from `.gitignore`, `.gcloudignore`, and `.ae_ignore`. - Updated `to_cloud_run`, `to_agent_engine`, and `to_gke` to use this helper as an ignore filter in `shutil.copytree`. - This ensures that only the files intended by the user are staged and deployed. ### Testing Plan **Unit Tests:** - [x] I have added or updated unit tests for my change. - [x] All unit tests pass locally. Summary of `pytest` results: ```text tests/unittests/cli/utils/test_cli_deploy_ignore.py ... [100%] 3 passed in 2.20s ``` **Manual End-to-End (E2E) Tests:** Verified the fix by following these steps: 1. **Setup:** Created a dummy agent directory with a `.gitignore` file. ```bash mkdir -p verify_agent touch verify_agent/agent.py verify_agent/__init__.py touch verify_agent/ignored_file.txt echo "ignored_file.txt" > verify_agent/.gitignore ``` 2. **Execution:** Ran the deploy command pointing to a local temp folder. ```bash adk deploy cloud_run --temp_folder ./debug_staged_out ./verify_agent ``` 3. **Verification:** Temporarily disabled the `shutil.rmtree` cleanup in code to inspect `./debug_staged_out`. 4. **Result:** Confirmed that `agent.py` and `.gitignore` were present, but `ignored_file.txt` was correctly excluded from the staging area. ### Checklist - [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document. - [x] I have performed a self-review of my own code. - [x] I have commented my code, particularly in hard-to-understand areas. - [x] I have added tests that prove my fix is effective or that my feature works. - [x] New and existing unit tests pass locally with my changes. - [x] I have manually tested my changes end-to-end. - [x] Any dependent changes have been merged and published in downstream modules. Co-authored-by: Haran Rajkumar <haranrk@google.com> COPYBARA_INTEGRATE_REVIEW=#4187 from kotaitos:fix/issue-4183-ignore-files 15f6d62 PiperOrigin-RevId: 934458832
The test_to_agent_engine_respects_multiple_ignore_files unit test added in PR #4187 failed in GitHub Actions. On CI runners there are no Application Default Credentials and no project/region, so to_agent_engine falls back to the interactive `gcloud auth application-default login` onboarding flow and aborts with click.Abort. The test only appeared to pass locally because ambient ADC let it return early (region unset) before reaching the deploy path. - Pass project, region and adk_version explicitly so the onboarding flow is skipped, exercising the full deploy path deterministically. - Give the mocked Agent Engine client a realistic resource name so the downstream console-URL formatting does not fail on a bare Mock. Co-authored-by: Haran Rajkumar <haranrk@google.com> PiperOrigin-RevId: 934630935
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
Problem:
The
adk deploycommands (cloud_run,agent_engine,gke) were not properly respecting.gitignore,.gcloudignore, or.ae_ignorefiles. This caused all files in the source directory, including large or sensitive ones likevenv/,.git/, and.env, to be copied to the temporary staging directory and subsequently uploaded to hosted environments.Solution:
_get_ignore_patterns_funchelper insrc/google/adk/cli/cli_deploy.pythat reads and combines patterns from.gitignore,.gcloudignore, and.ae_ignore.to_cloud_run,to_agent_engine, andto_gketo use this helper as an ignore filter inshutil.copytree.Testing Plan
Unit Tests:
Summary of
pytestresults:Manual End-to-End (E2E) Tests:
Verified the fix by following these steps:
.gitignorefile.shutil.rmtreecleanup in code to inspect./debug_staged_out.agent.pyand.gitignorewere present, butignored_file.txtwas correctly excluded from the staging area.Checklist