Skip to content

Conversation

@dimitri-yatsenko
Copy link
Member

Summary

New how-to guide for production deployment under Project Management.

Covers the new v2.0 settings from datajoint/datajoint-python#1346:

  • database.create_tables — Production mode that prevents automatic table creation
  • database.schema_prefix — Project isolation on shared databases
  • Environment-based configuration patterns (dev/staging/prod)

Content

  1. Prevent Automatic Table Creation

    • Enable production mode via Python, env var, or config file
    • Behavior comparison (development vs production)
    • Override pattern for controlled migrations
  2. Use Schema Prefixes

    • Configuration methods
    • Usage with dj.Schema(prefix + 'name')
    • SQL permission grants by prefix pattern
  3. Environment-Based Configuration

    • Configuration hierarchy (env > secrets > config > defaults)
    • Docker/Kubernetes example
    • Verification script

Files changed

  • New: src/how-to/deploy-production.md
  • Updated: mkdocs.yaml — nav entry under Project Management
  • Updated: src/how-to/index.md — added to listing
  • Updated: src/how-to/manage-pipeline-project.md — cross-reference in See Also

Dependencies

Requires datajoint/datajoint-python#1346 to be merged first (adds the settings).

🤖 Generated with Claude Code

New guide covering:
- database.create_tables setting (production mode)
- database.schema_prefix for project isolation
- Environment-based configuration patterns
- Verification script example

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new production deployment how-to guide documenting best practices for deploying DataJoint applications to production environments. The guide covers new v2.0 settings for production safety including automatic table creation prevention and schema prefixes for project isolation.

Changes:

  • New comprehensive how-to guide for production deployment patterns
  • Cross-references added to existing documentation for discoverability
  • Navigation entries added for the new guide

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
src/how-to/deploy-production.md New production deployment guide covering create_tables setting, schema_prefix configuration, and environment-based config patterns
mkdocs.yaml Added navigation entry under Project Management section
src/how-to/index.md Added guide to Project Management section listing
src/how-to/manage-pipeline-project.md Added cross-reference in See Also section

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 147 to 150
GRANT ALL PRIVILEGES ON `myproject\_%`.* TO 'developer'@'%';

-- Read-only access to another project
GRANT SELECT ON `otherproject\_%`.* TO 'developer'@'%';
Copy link

Copilot AI Jan 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In MySQL GRANT statements, the underscore character is a wildcard and should not be escaped with a backslash. The backslash escape is only necessary in LIKE clauses. Based on other examples in the codebase (see manage-pipeline-project.md line 209 which uses my_pipeline_*.* without escaping), the correct syntax should be myproject_% instead of myproject\_%.

Suggested change
GRANT ALL PRIVILEGES ON `myproject\_%`.* TO 'developer'@'%';
-- Read-only access to another project
GRANT SELECT ON `otherproject\_%`.* TO 'developer'@'%';
GRANT ALL PRIVILEGES ON `myproject_%`.* TO 'developer'@'%';
-- Read-only access to another project
GRANT SELECT ON `otherproject_%`.* TO 'developer'@'%';

Copilot uses AI. Check for mistakes.
Comment on lines 147 to 150
GRANT ALL PRIVILEGES ON `myproject\_%`.* TO 'developer'@'%';

-- Read-only access to another project
GRANT SELECT ON `otherproject\_%`.* TO 'developer'@'%';
Copy link

Copilot AI Jan 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In MySQL GRANT statements, the underscore character is a wildcard and should not be escaped with a backslash. The backslash escape is only necessary in LIKE clauses. The correct syntax should be otherproject_% instead of otherproject\_%.

Suggested change
GRANT ALL PRIVILEGES ON `myproject\_%`.* TO 'developer'@'%';
-- Read-only access to another project
GRANT SELECT ON `otherproject\_%`.* TO 'developer'@'%';
GRANT ALL PRIVILEGES ON `myproject_%`.* TO 'developer'@'%';
-- Read-only access to another project
GRANT SELECT ON `otherproject_%`.* TO 'developer'@'%';

Copilot uses AI. Check for mistakes.
Comment on lines 210 to 211
- DJ_USER_FILE=/run/secrets/db_user
- DJ_PASS_FILE=/run/secrets/db_password
Copy link

Copilot AI Jan 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The environment variables DJ_USER_FILE and DJ_PASS_FILE are not documented in the Configuration Reference or elsewhere in the codebase. While this is a common Docker secrets pattern, it's unclear if DataJoint supports reading credentials from files via these environment variables. If this is a new feature in v2.0, it should be documented in the Configuration Reference. If DataJoint doesn't support this pattern, consider using standard DJ_USER and DJ_PASS with Docker secrets mounted as environment variables instead, or document this as a planned feature.

Suggested change
- DJ_USER_FILE=/run/secrets/db_user
- DJ_PASS_FILE=/run/secrets/db_password
- DJ_USER=prod_service # injected via secret manager or env
- DJ_PASS=/run/secrets/db_password # example secret mount path

Copilot uses AI. Check for mistakes.
Comment on lines 146 to 150
-- Grant access to all schemas with prefix
GRANT ALL PRIVILEGES ON `myproject\_%`.* TO 'developer'@'%';

-- Read-only access to another project
GRANT SELECT ON `otherproject\_%`.* TO 'developer'@'%';
Copy link

Copilot AI Jan 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example GRANT statements create MySQL users with host set to '%', which allows connections from any IP address and significantly broadens the attack surface for the developer account. In a real production deployment, an attacker on any reachable network path could attempt to brute-force or abuse these credentials because there is no host-based restriction. Use more restrictive host patterns (for example limited to specific application hosts or networks) in the GRANT statements instead of '%' for production guidance.

Suggested change
-- Grant access to all schemas with prefix
GRANT ALL PRIVILEGES ON `myproject\_%`.* TO 'developer'@'%';
-- Read-only access to another project
GRANT SELECT ON `otherproject\_%`.* TO 'developer'@'%';
-- Grant access to all schemas with prefix from application network (replace 10.0.0.% with your app subnet/host)
GRANT ALL PRIVILEGES ON `myproject\_%`.* TO 'developer'@'10.0.0.%';
-- Read-only access to another project from the same application network
GRANT SELECT ON `otherproject\_%`.* TO 'developer'@'10.0.0.%';

Copilot uses AI. Check for mistakes.
dimitri-yatsenko and others added 4 commits January 25, 2026 23:50
Ensures clean slate when re-running notebook by dropping
existing schema before creating tables.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The admin module was removed in the remove/dj-kill branch.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove backslash escapes from MySQL GRANT patterns (not needed with backticks)
- Replace '%' host with '10.0.0.%' subnet and add security warning
- Fix Docker/Kubernetes secrets example to use /run/secrets/datajoint/
  directory pattern (DJ_USER_FILE and DJ_PASS_FILE don't exist)
- Add Kubernetes Secret mounting example

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
MilagrosMarin
MilagrosMarin previously approved these changes Jan 26, 2026
@MilagrosMarin MilagrosMarin merged commit 385ecca into main Jan 26, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants