Skip to content

docs: Update DataIndex integration documentation - #54

Merged
ricardozanini merged 11 commits into
kubesmarts:mainfrom
ricardozanini:docs-dataindex
Oct 1, 2026
Merged

ricardozanini merged 11 commits into
kubesmarts:mainfrom
ricardozanini:docs-dataindex

Conversation

@ricardozanini

@ricardozanini ricardozanini commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Comprehensive documentation updates for DataIndex integration feature from PR #52:

  • Renamed "Workflow Data Persistence" → "Workflow Instance History" for clarity
  • New concept page explaining Vector log collection and Data Index GraphQL queries
  • LogicPlatform CRD reference with full field documentation and examples
  • Updated deployment guides (kind-local, production, monitoring, overview)
  • Fixed schema naming (logicflow_runtime, logicflow_dataindex)
  • Fixed condition type names to proper camelCase
  • Added external links to logic-apps Data Index documentation
  • Removed namespace sharding example (one per cluster recommended)

Test plan

  • Review all documentation changes
  • Verify link references are correct
  • Check YAML examples for completeness

- Rename "Workflow Data Persistence" to "Workflow Instance History" for clarity
- Create new workflow-instance-history.adoc concept page explaining Vector + Data Index
- Add LogicPlatform CRD reference page with schema, examples, and troubleshooting
- Update all deployment guides, samples, and references for DataIndex integration
- Fix schema naming: use logicflow_runtime and logicflow_dataindex for clarity
- Fix condition type names to use proper camelCase (DataIndexDeploymentAvailable)
- Add external links to logic-apps Data Index documentation
- Update navigation and getting-started guides
- Remove namespace sharding example (one Data Index per cluster recommended)
- Rename persistence sample section to "Workflow Instance History via Data Index"
@ricardozanini
ricardozanini requested review from mathecruz and a balanced review from Copilot October 1, 2026 01:25

This comment was marked as outdated.

Addresses 25 Copilot review findings:

Code Changes:
- Add DebugEvents field to VectorSpec for debug logging control
- Implement DEBUG_EVENTS env var injection in WithVectorEnvVars()
- Remove unused Version field from LogicPlatformSpec

Documentation Fixes:
- Fix Vector configuration structure (show vector.application nesting)
- Correct watchNamespaces default (self-namespace, not ["*"])
- Clarify serviceRef field requirements (required: name, databaseName, databaseSchema)
- Document debugEvents feature with examples
- Add missing includeTaskPayloads documentation to LogicFlowRuntime
- Update status examples to match actual VectorStatus and DataIndexStatus types
- Correct condition type (VectorReady, not VectorDaemonSetReady)
- Add runtimeDefaults field documentation
- Add Multi-Namespace setup example
- Add security guidance (CAUTION blocks for payload fields)
- Add port-forwarding guidance for accessing Data Index externally

Breaking Changes: None (Version field was never used)
Signed-off-by: Ricardo Zanini <ricardozanini@gmail.com>
Remove the unused JdbcURL field from PersistencePostgreSQL struct:
- Field was never used in any controller logic
- Code always builds JDBC URL from serviceRef
- Simplifies API surface and eliminates dead code
- Makes serviceRef non-optional (required field)

Update all code and tests:
- Remove nil checks for serviceRef in controller
- Change ServiceRef from *PostgreSQLServiceOptions to PostgreSQLServiceOptions
- Update all test fixtures to use non-pointer struct values
- Remove 'omitempty' from json tag and @optional kubebuilder tag

Documentation:
- Clarify serviceRef is the only option (required)
- Remove mentions of mutual exclusivity with jdbcUrl
- Update field requirements table
- Add guidance on database pre-existence requirement
- Remove nil checks for serviceRef in webhook validation (now always required)
- Update buildJdbcURL call to pass address of struct since function expects pointer
- Simplify validation logic since serviceRef is no longer optional
Signed-off-by: Ricardo Zanini <ricardozanini@gmail.com>
Signed-off-by: Ricardo Zanini <ricardozanini@gmail.com>
Signed-off-by: Ricardo Zanini <ricardozanini@gmail.com>
- Add explicit anchor [#section-b-workflow-history] for xref resolution
- Update GraphQL query example to show port-forward requirement first
- Add includeTaskPayloads field to logging sample with default false
- Clarify that includeTaskPayloads is controlled independently from includeWorkflowPayloads
- Fix GraphQL verification curl to use localhost:8080 and add port-forward note
- Correct Key Takeaways to reflect actual Vector namespace default behavior
- Specify isolated schema names in infrastructure description

This comment was marked as outdated.

- Add QUARKUS_HIBERNATE_ORM_DATABASE_DEFAULT_SCHEMA env var so Hibernate knows which schema to create tables in
- Remove redundant QUARKUS_DATASOURCE_DB_KIND (already configured at build time in image)
- Increase MaxProperties validation from 2 to 3 to allow optional TLS field alongside secretRef and serviceRef

Update schema isolation strategy:
- LogicFlowRuntime uses `public` schema (always exists, no pre-creation needed)
- Data Index uses `logicflow_dataindex` (created automatically by Flyway migrations)

Update all documentation and samples:
- logicplatform.adoc: Remove unimplemented runtimeDefaults section, clarify schema creation, update namespace examples to data-index-system
- production.adoc: Document new schema strategy, clarify watchNamespaces default
- persistence samples: Use public for runtime, logicflow_dataindex for Data Index

Update tests for Hibernate schema env var changes:
- Update quarkus_config_test.go indices: removed QUARKUS_DATASOURCE_DB_KIND shifts indices
- Fix TLS tests to use fixed index [2] for JDBC URL instead of len(envs)-1
- Fix logicplatform test to expect logicflow_dataindex schema instead of data-index
- Fix logicflowruntime test to expect public schema instead of logicflow_runtime

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Legacy persistence resources can trigger nil dereferences, and several new documentation links, ports, schemas, and commands are inaccurate.

Review effort: Balanced
Findings: 1 High severity · 3 Medium severity

Open (4)
Resolved since last review (13)
Files not reviewed (1)
  • api/v1/zz_generated.deepcopy.go: Generated file
Previously missed (7)

In code that hasn't changed since last review

Medium severity Remove conflicting guidance on multiple LogicPlatform resources

docs/​antora/​modules/​ROOT/​pages/​crd-reference/​logicplatform.adoc:192

The PR description recommends one LogicPlatform per cluster, but this text recommends deploying multiple LogicPlatform resources per namespace. Remove that conflicting deployment guidance; watchNamespaces: ["*"] is the supported cluster-wide option.

Medium severity Document Vector replicas accurately for its DaemonSet deployment

docs/​antora/​modules/​ROOT/​pages/​crd-reference/​logicplatform.adoc:250

Vector is reconciled as a DaemonSet, and ToDaemonSetSpec ignores ApplicationSpec.Replicas (internal/controller/logicplatform_objects.go:145-156). Describing replicas as configurable and showing replicas: 1 therefore documents a setting that has no effect.

Medium severity Correct status URL example to use Service port 80

docs/​antora/​modules/​ROOT/​pages/​crd-reference/​logicplatform.adoc:295

The controller builds the internal status URL with the Service port 80 (internal/controller/logicplatform_controller.go:306-308); port 8080 is only the container target port. This status example does not match what users will actually see.

Medium severity Remove nonexistent Ready condition from status documentation

docs/​antora/​modules/​ROOT/​pages/​crd-reference/​logicplatform.adoc:315

The LogicPlatform controller never creates a top-level Ready condition; it only sets the four component conditions shown below (internal/controller/logicplatform_controller.go:211-275,367-440,655-685). Remove this nonexistent condition from the status example.

Medium severity Forward Data Index Service port 80 instead of undefined port 8080

docs/​antora/​modules/​ROOT/​pages/​crd-reference/​logicplatform.adoc:356

The Data Index Service exposes port 80 and targets container port 8080 (internal/controller/quarkus_config.go:31-43). Forwarding to Service port 8080 fails because that port is not defined.

Low severity Replace intra-document link syntax with Antora xref

docs/​antora/​modules/​ROOT/​pages/​concepts/​workflow-instance-history.adoc:5

This uses AsciiDoc's intra-document <<...>> syntax for another Antora page, so the cross-reference will not resolve. Use the repository's xref: form for inter-page links.

This issue also appears on line 230 of the same file.

Low severity Use Antora xref for the Troubleshooting cross-reference

docs/​antora/​modules/​ROOT/​pages/​crd-reference/​logicplatform.adoc:496

Antora inter-page links must use xref:; <<...>> is an intra-document cross-reference and this target will not resolve. Point it at the generated Troubleshooting section ID.

Comment thread api/v1/persistence_types.go
Comment thread docs/antora/modules/ROOT/pages/crd-reference/logicplatform.adoc Outdated
Comment thread docs/antora/modules/ROOT/pages/deployment/production.adoc Outdated
Comment thread docs/antora/modules/ROOT/pages/samples/persistence.adoc Outdated
Updated documentation for GraphQL API URL, Vector logging configuration, and PostgreSQL schema usage.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@ricardozanini
ricardozanini merged commit 37d079f into kubesmarts:main Oct 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants