Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

52 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

STON.fi Sync Engine

This repository is the public Rust workspace for STON.fi synchronization libraries published on crates.io. Each independently consumable synchronization layer lives in its own package under crates/.

Packages

Package Path Responsibility
stonfi_sync_core crates/sync_core Dependency-aware synchronization of ordered heights, with optional ScyllaDB progress storage
stonfi_distributed_sync crates/distributed_sync Distributed task coordination, gRPC servers, and workers

stonfi_distributed_sync is a separate package depending on stonfi_sync_core. It is not exposed as a core feature because its gRPC, protobuf, server, worker, serialization, and lifecycle concerns have an independent dependency and release boundary.

Installation

Version 0.1.0 is the first crates.io release of both packages:

[dependencies]
stonfi_distributed_sync = "0.1"
stonfi_sync_core = "0.1"

Enable stonfi_sync_core's non-default scylla feature when the engine should persist handler progress in an existing ScyllaDB keyspace.

Package versions remain independent. Release-plz creates a package-specific tag and GitHub Release for each published version.

See the stonfi_sync_core README for its API, runtime requirements, metrics initialization, lifecycle, and complete example. The stonfi_distributed_sync README documents its API, at-least-once delivery contract, trusted-network boundary, metrics, and runnable example.

Workspace validation

This library workspace intentionally leaves Cargo.lock untracked. A clean checkout resolves the current dependency versions allowed by the manifests; Cargo may create an ignored local lockfile while running these commands.

cargo test --workspace --all-features
cargo test --workspace --doc
cargo test --workspace --examples --all-features
cargo test -p stonfi_sync_core --features scylla --test scylla_progress_store -- --ignored --test-threads=1
cargo +nightly fmt --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
RUSTDOCFLAGS="-D warnings -D missing_docs" cargo doc --workspace --no-deps --all-features
cargo +1.95.0 check --workspace --all-features
cargo package --list -p stonfi_sync_core
cargo package --list -p stonfi_distributed_sync
cargo publish --dry-run -p stonfi_sync_core
bash .github/check-external-consumer.sh

Releases

After main passes CI, release-plz publishes any package version not yet on crates.io, creates its tag and GitHub Release, and creates or updates the next release pull request with version and changelog changes. It resolves workspace dependencies so stonfi_sync_core is published before stonfi_distributed_sync.

Configure the repository secret CRATES_IO_REGISTRY_TOKEN with a crates.io token that has publish-new and publish-update scopes before merging this release. The first successful release-plz run publishes version 0.1.0 of both currently new crates. In Settings → Actions → General → Workflow permissions, enable Allow GitHub Actions to create and approve pull requests so release-plz can maintain its release pull request.

About

Composable Rust synchronization engines for ordered data pipelines, with dependency-aware local scheduling and distributed gRPC workers.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages