Skip to content

feat: integrate DataLink semantic service#81

Merged
wangjk9527 merged 1 commit into
datagallery-lab:mainfrom
iancaoo:codex/integrate-datalink
Jul 21, 2026
Merged

feat: integrate DataLink semantic service#81
wangjk9527 merged 1 commit into
datagallery-lab:mainfrom
iancaoo:codex/integrate-datalink

Conversation

@iancaoo

@iancaoo iancaoo commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR introduces DataLink, DataFoundry's first-party semantic graph capability for enterprise data analysis. It connects physical schemas and data profiles with business concepts, entities, join paths, and confidence-scored relationships, giving the Data Agent richer context than schema inspection alone.

  • add the DataLink semantic service under services/datalink
  • expose semantic retrieval to agents through MCP and graph management through REST
  • add opt-in DATALINK_ENABLED orchestration for development and formal deployment
  • register a managed builtin-datalink resource with live health reporting while preserving externally configured services
  • support shared DataFoundry model settings and dedicated DATALINK_* overrides
  • add portable Python dependency management, service CI gates, health checks, and bilingual documentation

Feature value

DataLink strengthens DataFoundry in three areas:

  1. Business-aware grounding: maps tables and columns to concepts and entities instead of relying only on physical names.
  2. Relationship-aware analysis: exposes joinable paths, semantic equivalence, profiles, and confidence signals to reduce guesswork in complex multi-table tasks.
  3. Governed first-party integration: keeps semantic retrieval inside DataFoundry's managed resource, audit, configuration, and deployment boundaries.

Deployment behavior

  • default DATALINK_ENABLED=false: existing Web + API topology, with no Python or uv requirement
  • DATALINK_ENABLED=true: npm run dev and npm run start additionally launch DataLink MCP on 8080 and REST on 8081
  • split start:* commands remain available for existing process supervisors
  • external DataLink-compatible services remain supported

Verification

  • npm run build
  • npm run test:web (586 passed)
  • npm run lint:datalink
  • npm run test:datalink (211 passed, 2 skipped)
  • npm run build:datalink
  • npm run test:datalink-stack
  • node scripts/smoke-builtin-datalink.mjs
  • node scripts/smoke-config-api.mjs
  • npm run smoke:docs
  • npm run docs:build
  • live API + DataLink MCP/REST startup, managed-resource discovery, health checks, and shutdown cleanup

All GitHub Actions checks pass.

Scope

This feature keeps DataFoundry's current process-based deployment model. Docker, Kubernetes, multi-tenancy changes, and a language rewrite are intentionally out of scope.

@wangjk9527
wangjk9527 merged commit 3ffe7b7 into datagallery-lab:main Jul 21, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants