Run Weaviate E2E with the Kubernetes Agent backend by vitkyrka · Pull Request #24830 · DataDog/integrations-core

vitkyrka · 2026-08-11T14:31:22Z

What does this PR do?

Runs the Weaviate Kind E2E with the Kubernetes Agent backend introduced by #24639.

The weaviate_api_endpoint now uses the in-cluster Service DNS endpoint, http://weaviate.weaviate.svc.cluster.local:80, backed by the weaviate Service (port: 80 -> targetPort: 8080).
The openmetrics_endpoint (port 2112) has no matching Service, so it falls back to the pod IP of the single-replica weaviate StatefulSet, fetched via kubectl get pods -l app=weaviate.
The data-seeding POST to /v1/batch/objects previously ran on the host via requests. It now runs from a disposable curlimages/curl pod inside the cluster instead.
The host-side readiness poll that previously ran before seeding (ready_check(), hitting /v1/.well-known/live through a kubectl port-forward tunnel) has been removed rather than converted to an in-cluster equivalent.
/v1/.well-known/live (DEFAULT_LIVENESS_ENDPOINT, what the removed check polled) is an unconditional 200 with no real check behind it. The actual startup check lives behind /v1/.well-known/ready, which is exactly what the StatefulSet's readinessProbe hits (see weaviate_install.yaml), and the Kubernetes Pod Ready condition is driven by the readiness probe, not the liveness probe. So kubectl wait pods --for=condition=Ready, which setup_weaviate() already runs before seeding, provides a strictly stronger readiness guarantee than the removed poll ever did.
setup_weaviate()'s kubectl calls (namespace/manifest creation, rollout status, readiness wait) now all pass check=True, so a cluster that never becomes ready fails loudly there instead of surfacing as a confusing seed-request error further down.

Motivation

https://datadoghq.atlassian.net/browse/DSCVR-659

This PR migrates Weaviate to the new backend, following the same pattern already applied to argo_rollouts (#24674) and calico (#24673), plus the pod-IP fallback pattern from velero (#24645) for the port with no Service.

Review checklist (to be filled by reviewers)

Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

This PR description was generated by Claude Code, on behalf of Vincent Whitchurch.

Switch from host-side kube_port_forward to the new Kubernetes Agent E2E backend: the API endpoint uses Service DNS, the metrics endpoint falls back to the pod IP since no Service targets that port, and the readiness check plus data-seeding POST now run from disposable in-cluster pods since the host can no longer reach the cluster directly. Environment: Datadog workspace Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

vitkyrka · 2026-08-11T14:32:03Z

@codex review

datadog-datadog-prod-us1-2 · 2026-08-11T14:34:21Z

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 21.43%
• Overall Coverage: 78.37% (-10.05%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 50e6399 | Docs | Datadog PR Page | Give us feedback!}

cit-pr-commenter-54b7da · 2026-08-11T14:34:27Z

evalya-impact-summary

evalya impact analysis

Impact analysis: RUN-ALL — every test task will run
Trigger:         empty diff (default branch, scheduled run, or shallow-clone fallback)
Test tasks:      0 (all selected)
Publish tasks:   1 (always emitted)
Diff:            empty (no diff information)

Learn more about CI impact filtering

chatgpt-codex-connector · 2026-08-11T14:35:53Z

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 95d4230654

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

`/v1/.well-known/live` (DEFAULT_LIVENESS_ENDPOINT) is an unconditional 200 in Weaviate v1.20.0 with no actual readiness check, while the Pod `Ready` condition that `kubectl wait --for=condition=Ready` already blocks on is driven by the StatefulSet's readinessProbe (`/v1/.well-known/ready`, the real `DB.StartupComplete() && ClusterHealthScore() == 0` check). The extra busybox-based WaitFor poll was checking a strictly weaker signal than what setup_weaviate() already waits for, so it can't add any real confidence and was only burning disposable pods. Environment: Datadog workspace Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

vitkyrka · 2026-08-13T09:08:59Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9102cc59b7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

`run_command` defaults to `check=False`, so a timed-out or transient failure in the `kubectl create ns`/`apply`/`rollout status`/`wait` calls previously fell through silently into the pod-IP lookup and one-shot data-seeding curl, turning a clear "cluster never became Ready" failure into a confusing generic curl error. Add `check=True` to all four calls, matching the pattern already used for kuma's equivalent `setup_kuma()` (#24642). Environment: Datadog workspace Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

vitkyrka · 2026-08-13T09:32:59Z

@codex review

chatgpt-codex-connector · 2026-08-13T09:36:33Z

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 50e639990e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

dd-octo-sts · 2026-08-17T08:17:29Z

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
`agent-reqs`	Verify check versions match the Agent requirements file	✅
`ci`	Validate CI configuration and code coverage settings	✅
`codeowners`	Validate every integration has a CODEOWNERS entry	✅
`config`	Validate default configuration files against spec.yaml	✅
`dep`	Verify dependency pins are consistent and Agent-compatible	✅
`http`	Validate integrations use the HTTP wrapper correctly	✅
`imports`	Validate check imports do not use deprecated modules	✅
`integration-style`	Validate check code style conventions	✅
`jmx-metrics`	Validate JMX metrics definition files and config	✅
`labeler`	Validate PR labeler config matches integration directories	✅
`legacy-signature`	Validate no integration uses the legacy Agent check signature	✅
`license-headers`	Validate Python files have proper license headers	✅
`licenses`	Validate third-party license attribution list	✅
`metadata`	Validate metadata.csv metric definitions	✅
`models`	Validate configuration data models match spec.yaml	✅
`openmetrics`	Validate OpenMetrics integrations disable the metric limit	✅
`package`	Validate Python package metadata and naming	✅
`qa-label`	Validate the pull request declares whether it needs QA for the next Agent release	✅
`readmes`	Validate README files have required sections	✅
`saved-views`	Validate saved view JSON file structure and fields	✅
`version`	Validate version consistency between package and changelog	✅

View full run

temporal-github-worker-1 Bot added agent/review-requested ecosystems/review-requested product/review-requested labels Aug 11, 2026

vitkyrka added the qa/skip-qa Automatically skip this PR for the next QA label Aug 11, 2026 — with ddtool CLI

dd-octo-sts Bot added the integration/weaviate label Aug 11, 2026

chatgpt-codex-connector Bot reviewed Aug 13, 2026

View reviewed changes

Comment thread weaviate/tests/conftest.py

vitkyrka marked this pull request as ready for review August 17, 2026 08:08

vitkyrka requested a review from a team as a code owner August 17, 2026 08:08

dd-octo-sts Bot added the team/agent-integrations label Aug 17, 2026

nubtron approved these changes Aug 17, 2026

View reviewed changes

temporal-github-worker-1 Bot added agent/approved and removed agent/review-requested labels Aug 17, 2026

vitkyrka added this pull request to the merge queue Aug 17, 2026

Merged via the queue into master with commit 9373eb0 Aug 17, 2026
53 of 56 checks passed

vitkyrka deleted the vwhitchurch/weaviate-kubernetes-agent-e2e branch August 17, 2026 08:24

dd-octo-sts Bot added this to the 7.83.0 milestone Aug 17, 2026

This was referenced Aug 18, 2026

Run cert-manager E2E with the Kubernetes Agent backend #24896

Draft

Run Traefik Mesh E2E with the Kubernetes Agent backend #24897

Merged

nubtron mentioned this pull request Aug 18, 2026

Run Istio E2E with the Kubernetes Agent backend #24898

Draft

3 tasks

Navigation Menu

Uh oh!

Run Weaviate E2E with the Kubernetes Agent backend - #24830

Run Weaviate E2E with the Kubernetes Agent backend#24830
vitkyrka merged 3 commits into
masterfrom
vwhitchurch/weaviate-kubernetes-agent-e2e

vitkyrka commented Aug 11, 2026 •

edited

Loading

Uh oh!

vitkyrka commented Aug 11, 2026

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Aug 11, 2026 •

edited

Loading

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 11, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented Aug 11, 2026

Uh oh!

vitkyrka commented Aug 13, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

vitkyrka commented Aug 13, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented Aug 13, 2026

Uh oh!

dd-octo-sts Bot commented Aug 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

vitkyrka commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Review checklist (to be filled by reviewers)

Uh oh!

vitkyrka commented Aug 11, 2026

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

evalya-impact-summary

Uh oh!

chatgpt-codex-connector Bot commented Aug 11, 2026

Uh oh!

vitkyrka commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

vitkyrka commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Aug 13, 2026

Uh oh!

dd-octo-sts Bot commented Aug 17, 2026

Validation Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vitkyrka commented Aug 11, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented Aug 11, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Aug 11, 2026 •

edited

Loading

vitkyrka commented Aug 13, 2026 •

edited

Loading

vitkyrka commented Aug 13, 2026 •

edited

Loading