Repository navigation
feat(azure): add the localstack-azure-client tool and Azure lifecycle support #79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
f95ae5e
feat(azure): add the Azure emulator tool and Azure lifecycle support
DrisDary 371eb9a
chore(azure): keep internal references out of the repository
DrisDary b4a1dcd
docs(azure): tighten the README's Azure section and the test wording
DrisDary bb11f44
ci(azure): no model evals in CI, fail fast on licence errors, scope t…
DrisDary 7fb62bb
fix(azure): address the review of the Azure tool
DrisDary 8d6ba79
ci(azure): run the Azure jobs with the LOCALSTACK_AUTH_TOKEN_AZURE se…
DrisDary 67dd802
fix(azure): bind-mount the emulator's state folder so app deployments…
DrisDary 89ed16a
ci: run every job with the LOCALSTACK_AUTH_TOKEN_AZURE secret
DrisDary 1f12bf3
ci: run every job with the one LOCALSTACK_AUTH_TOKEN secret
DrisDary File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # Line endings that must not depend on the checkout's core.autocrlf (Windows runners convert to | ||
| # CRLF by default): bash cannot run a script with CRLF, and the samples shim's tests run | ||
| # tests/azure/samples-shim/az through Git Bash on Windows. | ||
| *.sh text eol=lf | ||
| tests/azure/samples-shim/az text eol=lf | ||
| # cmd.exe shims keep CRLF everywhere (tests/azure/samples-shim/quote.test.ts checks it). | ||
| *.cmd text eol=crlf | ||
| # The L2 matrix files are checked for LF (tests/azure/matrix/schema.test.ts). | ||
| tests/azure/matrix/*.yaml text eol=lf | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # Code owners: GitHub requests their review on any pull request that touches these paths. | ||
| # | ||
| # The Azure tool, with its tests, fixtures, CI and image parts: the Azure team (smurf) and the | ||
| # repository's maintainer. Files shared with the AWS and Snowflake tools (the lifecycle tool, the | ||
| # launcher, the wizard, the Dockerfile, the README) are left out on purpose, so that AWS and | ||
| # Snowflake changes do not request Azure reviewers. | ||
|
|
||
| # The tool | ||
| /src/lib/azure/ @localstack/smurf @HarshCasper | ||
| /src/tools/localstack-azure-client.ts @localstack/smurf @HarshCasper | ||
| /src/tools-tests/localstack-azure-client.test.ts @localstack/smurf @HarshCasper | ||
| /src/lib/wizard/azure-steps.ts @localstack/smurf @HarshCasper | ||
| /src/lib/wizard/azure-steps.test.ts @localstack/smurf @HarshCasper | ||
| /src/cli/azure-addons.ts @localstack/smurf @HarshCasper | ||
| /src/cli/azure-addons.test.ts @localstack/smurf @HarshCasper | ||
|
|
||
| # Tests, fixtures and sample data | ||
| /tests/azure/ @localstack/smurf @HarshCasper | ||
| /tests/fixtures/azure/ @localstack/smurf @HarshCasper | ||
| /tests/mcp/azure-offline.spec.mjs @localstack/smurf @HarshCasper | ||
| /tests/mcp/evals-gemini-azure.spec.mjs @localstack/smurf @HarshCasper | ||
| /data/sample-azure/ @localstack/smurf @HarshCasper | ||
| /data/evals/gemini-azure.json @localstack/smurf @HarshCasper | ||
| /jest.azure-live.config.js @localstack/smurf @HarshCasper | ||
|
|
||
| # CI and scripts | ||
| /.github/workflows/azure-live.yml @localstack/smurf @HarshCasper | ||
| /.github/workflows/azure-weekly.yml @localstack/smurf @HarshCasper | ||
| /.github/actions/azure-live-setup/ @localstack/smurf @HarshCasper | ||
| /scripts/ci/azure-emulator-up.sh @localstack/smurf @HarshCasper | ||
| /scripts/ci/scan-for-secret.mjs @localstack/smurf @HarshCasper | ||
| /scripts/ci/merge-op-catalogue.cjs @localstack/smurf @HarshCasper | ||
| /scripts/install-azure-extensions.mjs @localstack/smurf @HarshCasper | ||
| /scripts/gen-az-file-args.py @localstack/smurf @HarshCasper | ||
|
|
||
| # The image's Azure layers (the Azure CLI, its extensions and Bicep) | ||
| /docker/azure-extensions.txt @localstack/smurf @HarshCasper | ||
| /docker/image-size.json @localstack/smurf @HarshCasper | ||
| /tests/docker/image-size.mjs @localstack/smurf @HarshCasper | ||
| /tests/docker/l5-image-assertions.sh @localstack/smurf @HarshCasper |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| name: Azure live setup | ||
| description: >- | ||
| Node, the build, the pinned azure-cli in its own venv, the curated extensions and the | ||
| pinned Bicep. The calling job sets LOCALSTACK_AZ_PATH and | ||
| LOCALSTACK_AZ_BICEP_PATH to /home/runner/az/bin/python3 and /home/runner/bicep-bin/bicep. | ||
| inputs: | ||
| az-version: | ||
| description: azure-cli version (the image pins 2.90.0; DR3 also runs the minimum) | ||
| default: "2.90.0" | ||
| extensions: | ||
| description: install the curated extensions (docker/azure-extensions.txt) | ||
| default: "true" | ||
| runs: | ||
| using: composite | ||
| steps: | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 22.x | ||
|
|
||
| - shell: bash | ||
| run: yarn install --frozen-lockfile && yarn build | ||
|
|
||
| - name: Pinned azure-cli in its own venv (never the runner's /usr/bin/az) | ||
| shell: bash | ||
| run: | | ||
| python3 -m venv ~/az | ||
| if [ "${{ inputs.az-version }}" = "latest" ]; then | ||
| ~/az/bin/pip install --quiet azure-cli | ||
| else | ||
| ~/az/bin/pip install --quiet "azure-cli==${{ inputs.az-version }}" | ||
| fi | ||
| ~/az/bin/python3 -c "import azure.cli.core as c; print('azure-cli-core', c.__version__)" | ||
|
|
||
| - name: Curated extensions | ||
| if: inputs.extensions == 'true' | ||
| shell: bash | ||
| run: node scripts/install-azure-extensions.mjs --az-python ~/az/bin/python3 --dir ~/.localstack/azure/mcp-extensions | ||
|
|
||
| - name: Pinned Bicep v0.47.16, sha256-checked | ||
| shell: bash | ||
| run: | | ||
| mkdir -p ~/bicep-bin | ||
| curl -fsSL -o ~/bicep-bin/bicep https://github.com/Azure/bicep/releases/download/v0.47.16/bicep-linux-x64 | ||
| echo "64c345a58e0c3e48b1bc98a4e62d6b3adb1d238281297de3400aeafb2697aa5a ${HOME}/bicep-bin/bicep" | sha256sum -c - | ||
| chmod +x ~/bicep-bin/bicep |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,251 @@ | ||
| name: Azure live tests | ||
|
|
||
| # The Azure tool against a real LocalStack Azure emulator: the L2 matrix | ||
| # subset, L3 egress and L1 (the harness stage) in one job; the L4 samples subset in its own | ||
| # job with its own emulator. Path-filtered, so AWS-only PRs do not pay for the emulator. | ||
| on: | ||
| pull_request: | ||
| branches: [main] | ||
| paths: | ||
| - "src/lib/azure/**" | ||
| - "src/lib/cli/**" | ||
| - "src/tools/localstack-azure-client.ts" | ||
| - "src/tools/localstack-management.ts" | ||
| # The Azure tool imports core/ (analytics, the response builder); L1 runs dist/cli.js. | ||
| - "src/core/**" | ||
| - "src/cli/**" | ||
| - "src/lib/docker/**" | ||
| - "src/lib/localstack/**" | ||
| - "tests/azure/**" | ||
| # Read by the live suites: the leak commands, the Bicep fixtures, L1's deployments. | ||
| - "tests/fixtures/azure/**" | ||
| - "data/sample-azure/**" | ||
| - "tests/docker/validate-image.mjs" | ||
| - "docker/azure-extensions.txt" | ||
| - "scripts/install-azure-extensions.mjs" | ||
| - "scripts/ci/**" | ||
| - "jest.azure-live.config.js" | ||
| - ".github/actions/azure-live-setup/**" | ||
| - ".github/workflows/azure-live.yml" | ||
| push: | ||
| branches: [main] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: azure-live-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| env: | ||
| AZ_VERSION: "2.90.0" | ||
| # The samples repo's main (2026-09-23). The samples corpus (tests/fixtures/azure/corpus) was | ||
| # built from 4193d67, whose sample scripts are byte-identical to this commit's; only | ||
| # run-samples.sh differs (it registers two more Bicep samples). | ||
| SAMPLES_COMMIT: "5ae698478bd4810b619469959b73c61f326bc569" | ||
|
|
||
| jobs: | ||
| live: | ||
| # Secrets are not available to PRs from forks. | ||
| if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 120 | ||
| env: | ||
| LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} | ||
| # Job level, so every step (the installer included) uses the pinned az and Bicep, | ||
| # never the runner's /usr/bin/az. | ||
| LOCALSTACK_AZ_PATH: /home/runner/az/bin/python3 | ||
| LOCALSTACK_AZ_BICEP_PATH: /home/runner/bicep-bin/bicep | ||
| AZURE_CI_EMULATOR_CONTAINER: ls-azure-ci | ||
| MCP_ANALYTICS_DISABLED: "1" | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| # Node, the build, the pinned az (own venv), the curated extensions, the pinned Bicep. | ||
| - uses: ./.github/actions/azure-live-setup | ||
| with: | ||
| az-version: ${{ env.AZ_VERSION }} | ||
|
|
||
| # Before the emulator starts, so that "~/.azure unchanged" has a baseline even when the | ||
| # start fails. | ||
| - name: "~/.azure fingerprint (before)" | ||
| run: node tests/azure/tools/azure-home-fingerprint.mjs > "$RUNNER_TEMP/azure-home-before.json" | ||
|
|
||
| - name: Start the job's own emulator | ||
| run: bash scripts/ci/azure-emulator-up.sh | ||
|
|
||
| - name: L2 matrix subset | ||
| run: AZURE_LIVE=1 npx jest -c jest.azure-live.config.js --selectProjects matrix-subset --runInBand | ||
| env: | ||
| AZURE_MATRIX_BACKING: "1" | ||
|
|
||
| # Last against this emulator: its stop case stops the container and starts it again. | ||
| - name: L3 egress and the home guard | ||
| run: AZURE_LIVE=1 npx jest -c jest.azure-live.config.js --selectProjects egress --runInBand | ||
| env: | ||
| AZURE_EGRESS_CI: "1" | ||
| AZURE_EGRESS_REPORT: test-results/egress-steps.jsonl | ||
|
|
||
| - name: Emulator logs, then remove the job's emulator (L1 starts its own) | ||
| if: always() | ||
| run: | | ||
| mkdir -p emulator-logs | ||
| docker logs "$AZURE_CI_EMULATOR_CONTAINER" > emulator-logs/emulator-live.log 2>&1 || true | ||
| docker rm -f "$AZURE_CI_EMULATOR_CONTAINER" || true | ||
|
|
||
| - name: L1 through the npx path (its own start, scenario, stop) | ||
| run: node tests/docker/validate-image.mjs -- node dist/cli.js | ||
| env: | ||
| HARNESS_ONLY: azure | ||
| HARNESS_TOKEN_REAL: "1" | ||
| HARNESS_AZURE_BICEP: "1" | ||
|
|
||
| - name: "~/.azure unchanged" | ||
| if: always() | ||
| run: node tests/azure/tools/azure-home-fingerprint.mjs --compare "$RUNNER_TEMP/azure-home-before.json" | ||
|
|
||
| - name: Scan logs and results for the token before any upload | ||
| id: scan | ||
| if: always() | ||
| run: node scripts/ci/scan-for-secret.mjs --env LOCALSTACK_AUTH_TOKEN emulator-logs test-results | ||
|
|
||
| # Never upload when the scan failed: the artifacts would carry the token. | ||
| - name: Upload logs and results on failure | ||
| if: failure() && steps.scan.outcome == 'success' | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: azure-live-logs | ||
| path: | | ||
| emulator-logs/ | ||
| test-results/ | ||
| retention-days: 7 | ||
|
|
||
| - name: Upload the operation catalogue (for the portal) | ||
| if: always() && steps.scan.outcome == 'success' | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: azure-op-catalogue | ||
| path: test-results/azure-op-catalogue.json | ||
| if-no-files-found: ignore | ||
| retention-days: 30 | ||
|
|
||
| # Every run, not only failed ones: the per-case results show which known gaps still | ||
| # fail and which started passing (tests/azure/matrix/README.md). | ||
| - name: Upload the L2 case results | ||
| if: always() && steps.scan.outcome == 'success' | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: azure-matrix-results | ||
| path: test-results/azure-matrix.jsonl | ||
| if-no-files-found: ignore | ||
| retention-days: 30 | ||
|
|
||
| samples: | ||
| name: L4 samples subset | ||
| if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 90 | ||
| env: | ||
| LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} | ||
| LOCALSTACK_AZ_PATH: /home/runner/az/bin/python3 | ||
| LOCALSTACK_AZ_BICEP_PATH: /home/runner/bicep-bin/bicep | ||
| AZURE_CI_EMULATOR_CONTAINER: ls-azure-samples | ||
| MCP_ANALYTICS_DISABLED: "1" | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: ./.github/actions/azure-live-setup | ||
| with: | ||
| az-version: ${{ env.AZ_VERSION }} | ||
|
|
||
| - name: Samples repo at the pinned commit | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: localstack/localstack-azure-samples | ||
| ref: ${{ env.SAMPLES_COMMIT }} | ||
| path: samples-repo | ||
|
|
||
| # What the three PR samples call besides az (tests/azure/samples-shim/README.md). | ||
| - name: JDK 17 for servicebus/java (Maven is preinstalled) | ||
| uses: actions/setup-java@v4 | ||
| with: | ||
| distribution: temurin | ||
| java-version: "17" | ||
|
|
||
| - name: Python with azure-eventhub for the Event Hubs validate.sh scripts | ||
| run: | | ||
| python3 -m venv "$RUNNER_TEMP/samples-py" | ||
| "$RUNNER_TEMP/samples-py/bin/pip" install --quiet azure-eventhub | ||
| echo "PYTHON_BIN=$RUNNER_TEMP/samples-py/bin/python" >> "$GITHUB_ENV" | ||
|
|
||
| # Before the emulator starts, so that "~/.azure unchanged" has a baseline even when the | ||
| # start fails. | ||
| - name: "~/.azure fingerprint (before)" | ||
| run: node tests/azure/tools/azure-home-fingerprint.mjs > "$RUNNER_TEMP/azure-home-before.json" | ||
|
|
||
| - name: Start the job's own emulator | ||
| run: bash scripts/ci/azure-emulator-up.sh | ||
|
|
||
| - name: L4 samples subset through the az shim | ||
| run: AZURE_LIVE=1 npx jest -c jest.azure-live.config.js --selectProjects samples-subset --runInBand | ||
| env: | ||
| AZURE_SAMPLES_DIR: ${{ github.workspace }}/samples-repo | ||
| AZURE_SAMPLES_COMMIT: ${{ env.SAMPLES_COMMIT }} | ||
| AZURE_SAMPLES_RESULTS_DIR: ${{ github.workspace }}/test-results/samples | ||
|
|
||
| - name: Emulator logs, then remove the job's emulator | ||
| if: always() | ||
| run: | | ||
| mkdir -p emulator-logs | ||
| docker logs "$AZURE_CI_EMULATOR_CONTAINER" > emulator-logs/emulator-samples.log 2>&1 || true | ||
| docker rm -f "$AZURE_CI_EMULATOR_CONTAINER" || true | ||
|
|
||
| - name: "~/.azure unchanged" | ||
| if: always() | ||
| run: node tests/azure/tools/azure-home-fingerprint.mjs --compare "$RUNNER_TEMP/azure-home-before.json" | ||
|
|
||
| - name: Scan logs and results for the token before any upload | ||
| id: scan | ||
| if: always() | ||
| run: node scripts/ci/scan-for-secret.mjs --env LOCALSTACK_AUTH_TOKEN emulator-logs test-results | ||
|
|
||
| - name: Upload logs and results on failure | ||
| if: failure() && steps.scan.outcome == 'success' | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: azure-samples-logs | ||
| path: | | ||
| emulator-logs/ | ||
| test-results/ | ||
| retention-days: 7 | ||
|
|
||
| alert: | ||
| name: Tracking issue on failure (main) | ||
| # A PR's failure shows on the PR, to its author and its code-owner reviewers. A failure on | ||
| # main has no PR, so it opens or updates an issue instead, as azure-weekly.yml does. | ||
| needs: [live, samples] | ||
| if: failure() && github.event_name == 'push' | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| issues: write | ||
| steps: | ||
| - name: Open or update the tracking issue | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} | ||
| # The Azure code owners (.github/CODEOWNERS): every alert mentions them, and a new | ||
| # issue is assigned to ASSIGNEE. | ||
| OWNERS: "@localstack/smurf @HarshCasper" | ||
| ASSIGNEE: HarshCasper | ||
| run: | | ||
| title="Azure live tests failing on main" | ||
| existing=$(gh issue list --repo "$GITHUB_REPOSITORY" --state open --search "$title in:title" --json number --jq '.[0].number' --limit 100) | ||
| body="The Azure live tests failed on main at ${GITHUB_SHA::7}: $RUN_URL (jobs: live, samples). cc $OWNERS" | ||
| if [ -n "$existing" ]; then | ||
| gh issue comment "$existing" --repo "$GITHUB_REPOSITORY" --body "$body" | ||
| else | ||
| url=$(gh issue create --repo "$GITHUB_REPOSITORY" --title "$title" --body "$body") | ||
| # Assigned separately, so that an assignee GitHub refuses cannot lose the alert. | ||
| gh issue edit "$url" --repo "$GITHUB_REPOSITORY" --add-assignee "$ASSIGNEE" || echo "::warning::could not assign $ASSIGNEE to $url" | ||
| fi |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.