feat: Make Vector PostgreSQL schema configurable via POSTGRES_SCHEMA

[ricardozanini]

Summary

Vector and Flyway migrations both interact with PostgreSQL tables, but lacked a mechanism to ensure they use the same schema. This PR makes PostgreSQL schema explicitly configurable across Vector, Flyway, and DataIndex services, ensuring consistent table resolution and migration creation.

Addresses #83

Technical Details

The Fix

Vector PostgreSQL sink now uses the search_path connection parameter:

endpoint: "postgres://...?options=-c%20search_path%3D${POSTGRES_SCHEMA}"

DataIndex JDBC datasource (Flyway + JPA) now includes schema via currentSchema:

jdbc:postgresql://...?currentSchema=${POSTGRES_SCHEMA}

Flyway migrations target the configured schema:

QUARKUS_FLYWAY_SCHEMAS=${POSTGRES_SCHEMA}

This ensures all components consistently target the same schema:

1. Flyway creates tables in the specified schema
2. Vector inserts into the same schema
3. DataIndex JPA queries the same schema

Schema Configuration

• Default: logicapps (for testing and production use cases)
• Configurable: Via Helm postgresql.schema value
• Propagation:
  • Vector DaemonSet: POSTGRES_SCHEMA environment variable
  • DataIndex/Ingestion: JDBC URL parameter via currentSchema
  • Flyway: Target schema via QUARKUS_FLYWAY_SCHEMAS environment variable
  • E2E verification: Schema-aware table and trigger checks

Usage in logic-operator

When logic-operator deploys DataIndex with a custom schema:

helm install data-index ... --set postgresql.schema=logicflow

All components automatically use the same schema:

• Flyway creates tables in logicflow schema
• Vector targets logicflow schema for inserts
• DataIndex queries logicflow schema
• E2E verification checks logicflow schema for tables

Changes

• Vector endpoint: Add options=-c%20search_path%3D${POSTGRES_SCHEMA} to PostgreSQL sink
• DataIndex JDBC: Add currentSchema parameter to connection string helper
• DataIndex: Add QUARKUS_FLYWAY_SCHEMAS environment variable
• Vector DaemonSet: Add POSTGRES_SCHEMA environment variable
• Helm values: Set postgresql.schema default to logicapps
• E2E test scripts: Pass POSTGRES_SCHEMA=logicapps to verification
• E2E verification: Make schema checks dynamic (use POSTGRES_SCHEMA env var)
• Vector config validation: Update test to use POSTGRES_SCHEMA=logicapps
• CI MODE 2: Fix build command to use Maven reactor with -am flag
• Documentation: Update vector-config.adoc with schema configuration examples

Verification

Unit Tests ✅

• 2 Vector Config Validation Tests — PASS (validates endpoint with logicapps schema)

E2E Tests ✅

• MODE 1 KIND E2E Test — PASS
  • PostgreSQL accepts connections
  • Flyway creates tables in logicapps schema
  • Vector DaemonSet running and configured
  • DataIndex service healthy
  • Schema verification detects 4 tables + triggers in logicapps schema

Implementation Notes

• The PostgreSQL schema must exist before Vector connects
• Default logicapps supports the operator use case while remaining configurable
• E2E verification script uses ${POSTGRES_SCHEMA:-public} fallback for backward compatibility
• When using custom schemas, ensure all three components (Vector, Flyway, DataIndex) are configured consistently
• This makes deployments fully schema-aware, critical for multi-tenant or production scenarios

[Copilot]

Copilot review overview

🟡 Changes recommended

Critical issues remain with the sink connection syntax and propagation of the selected schema to migrations and application datasources.

Review effort: Lite
Findings: 3 · 2

Open (5)

• Unsupported currentSchema option will not set PostgreSQL search_path · New
• Schema change is not propagated to migrations and applications · New
• Selected schema is not wired through Helm deployment · New
• Validation test does not verify PostgreSQL schema routing · New
• Documentation omits PostgreSQL schema configuration · New

What changed in this PR

This PR makes Vector’s PostgreSQL schema configurable through Helm and environment variables.

Changes:

• Adds POSTGRES_SCHEMA to Vector sink configuration.
• Exposes postgresql.schema through Helm.
• Updates Vector deployment wiring and validation configuration.

File	Summary
data-index/​helm/​data-index/​values.yaml	Adds the PostgreSQL schema value.
data-index/​helm/​data-index/​templates/​vector-daemonset.yaml	Injects the schema into Vector.
data-index/​collectors/​vector/​mode1-postgresql/​vector.yaml	Adds schema-aware sink configuration and documentation.
data-index/​collectors/​src/​test/​java/​org/​kubesmarts/​logic/​apps/​dataindex/​collectors/​VectorConfigValidationIT.java	Supplies the schema during validation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.