Skip to content
microsoftPublic

About

Synchronize database schemas between a Kusto cluster and the local file system

Resources

Code of conduct

Contributing

Security policy

Stars

47 stars

Watchers

7 watching

Forks

Repository files navigation

Sync Kusto

The Sync Kusto tool was built to help create a maintainable development process around Kusto. The tool makes it easy to sync database schemas using either the local file system or a Kusto cluster as either the source of the target. It will copy full function definitions along with table schemas. Table data is not synced.

Screenshot

Scope

This tool is intended specifically for the developer desktop scenario. It will not help if you need to build an automated syncing process, generate change scripts, sync policies, etc. For all of that, we recommend using Delta Kusto and/or automated provisioning options.

Getting Started

Assuming that the user already has a Kusto database with functions and tables, set the Kusto database as the source and the local file system as the target. Before your first comparison, open Settings and configure a dedicated temporary cluster and database as described below. Press the compare button and the tool will find that none of the database objects are in the target. Press the update button to download everything from Kusto to the local file system.

Recommended Team Process

The general usage pattern on our team is to make changes in a Kusto dev database which starts with a schema that mirrors the production database. Once the developer is satisfied with the changes, they use Sync Kusto with the development database as the source and their local file system as the target. The changes are committed to source control and a pull request is sent out for review. After the request is approved and the code merges to master, the developer syncs the master branch and uses Sync Kusto to sync from their local file system to the production Kusto database. This is a more flexible/manual version of the Azure Data Explorer task for Azure Dev Ops. We've had great success with this process and use the tool daily to help with our development and for deploying to production assets.

Sync Kusto Binaries

The release can be downloaded from the GitHub Releases page. Extract synckusto.zip and run SyncKusto.exe. .NET 10 Windows Desktop Runtime is required.

Settings

Temporary Databases

When the local file system is selected as either the source or the target, Sync Kusto loads its schema definitions into the configured temporary database for Kusto-backed normalization. Specify both a cluster and an existing database where you have permissions to create and delete functions and tables. All selected files on both sides are read and checked before opening a temporary client or changing that database. Files rejected by the schema-command checks are skipped and reported; only accepted definitions are loaded. After checking both sides, all existing functions and tables in the temporary database are deleted before loading each file-backed side. If the database is not empty when you configure it, Settings asks you to confirm that it can be wiped before saving. Use a dedicated scratch database containing no valuable objects or data. Do not point it at a source or target Kusto database. If two users share a temporary database concurrently, results can be incorrect; give each user their own.

Schema file formats and corrections

The selected extension applies to both reading and writing: .kql, or legacy .csl when that setting is enabled. Files beneath the schema root's Functions folder (including its subfolders) must contain one .create-or-alter function definition. Other matching files must contain one .create table or .create-merge table definition with an explicit column list.

Function parameters, bodies, quoted names, comments, multiline strings, dependency references, and cross-database references inside a function body are supported. Optional with properties are folder and docstring strings, plus skipvalidation and view booleans for functions (boolean strings are also accepted). Normalization supplies skipvalidation=true so dependent functions can be loaded in any order.

Compare skips each file rejected by those checks, including unsupported object types such as materialized views, syntax errors, and scripts containing multiple commands. Materialized views are not supported for comparison or sync. A compact, resizable warning lists every skipped file in a scrolling, copyable report with Source or Target, the full path relative to that selected root, the reason and correction guidance, and a parser line/column when available. Reopen it with Skipped files. The comparison is incomplete: skipped source definitions can make target objects eligible for deletion, and file-target updates can overwrite or remove skipped files. Review checked changes before Update. Skipped file contents are never executed.

AAD Authority

For Microsoft Entra ID user authentication, this field is optional. Leave it blank to use the Kusto SDK's default authority resolution. If automatic resolution does not work for your tenant, specify its authority in Settings. Application authentication with a key or certificate requires an explicit authority.

Warnings

  • Ask before dropping objects in the target Kusto database - This optional check is enabled by default and will prompt the user before dropping anything in the target database.

Formatting

  • Place table fields on new lines - When CSL table files are written to a file target, each field is placed on a new line. This can make it easier to diff changes in a pull request.
  • Generated ".create-merge table" commands instead of ".create table" - When CSL table files are written to a file target, the command will be ".create-merge table" if this is checked. If it is unchecked, the command will be ".create table".

Files

  • Use legacy .csl file extensions - Kusto originally used files with .csl file extensions. This extension has since been deprecated and replaced with the .kql file extension. This setting tells synckusto which file extension should be used for reading and writing all files. The default state is checked, which means synckusto will consume and emit files with the .csl file extension.

Contributing

Issues, additional features, and tests are all welcome. See CONTRIBUTING.md for build instructions and contribution guidelines.

About

Synchronize database schemas between a Kusto cluster and the local file system

Resources

Code of conduct

Contributing

Security policy

Stars

47 stars

Watchers

7 watching

Forks

Releases

Used by

Contributors

Languages