Skip to content

Commit 14f6fba

Browse files
committed
docs: add simple readme
closes #23
1 parent 662f49b commit 14f6fba

1 file changed

Lines changed: 57 additions & 0 deletions

File tree

README.md

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Multistore
2+
3+
A set of Rust crates for building multi-runtime S3 gateway proxies. Multistore provides the components — S3 request parsing, SigV4 authentication, backend resolution, configuration, and middleware — to assemble a proxy that sits between S3-compatible clients and backend object stores. Combine them into a native Tokio/Hyper server, a Cloudflare Worker at the edge, or your own custom runtime.
4+
5+
> [!WARNING]
6+
> This project is in early stages and should be considered experimental. APIs and interfaces may change significantly between versions.
7+
8+
## Features
9+
10+
- **Unified interface** — Single stable URL per dataset regardless of backend provider. Backend migrations are invisible to data consumers.
11+
- **Native S3 compatibility** — Works with aws-cli, boto3, DuckDB, obstore, GDAL, or any S3-compatible client. Just set the endpoint URL.
12+
- **Flexible authentication** — Anonymous access, long-lived access keys, or OIDC/STS token exchange for temporary credentials.
13+
- **Multi-runtime** — Same core logic deploys as a native server in containers or as a Cloudflare Worker at the edge.
14+
- **Zero-copy streaming** — Presigned URLs enable direct streaming between clients and backends without buffering.
15+
- **Extensible** — Pluggable traits for request resolution, configuration, and backend I/O.
16+
17+
## Crate Layout
18+
19+
```
20+
crates/
21+
├── core/ (multistore) # Runtime-agnostic: traits, S3 parsing, SigV4, registries
22+
├── metering/ (multistore-metering) # Usage metering and quota enforcement middleware
23+
├── sts/ (multistore-sts) # OIDC/STS token exchange
24+
├── oidc-provider/ (multistore-oidc-provider) # Outbound OIDC provider
25+
├── static-config/ (multistore-static-config) # Static config provider
26+
├── path-mapping/ (multistore-path-mapping) # Path mapping utilities
27+
└── cf-workers/ (multistore-cf-workers) # Cloudflare Workers runtime (WASM)
28+
29+
examples/
30+
├── server/ (multistore-server) # Tokio/Hyper native server
31+
├── lambda/ (multistore-lambda) # AWS Lambda runtime
32+
└── cf-workers/ (multistore-cf-workers) # Cloudflare Workers
33+
```
34+
35+
## Getting Started
36+
37+
See the [full documentation](https://developmentseed.org/multistore/getting-started/) for configuration, deployment, and usage guides.
38+
39+
## Development
40+
41+
```bash
42+
# Format
43+
cargo fmt
44+
45+
# Lint
46+
cargo clippy --fix --allow-dirty --allow-staged
47+
48+
# Check (native)
49+
cargo check
50+
51+
# Check (WASM)
52+
cargo check -p multistore-cf-workers --target wasm32-unknown-unknown
53+
```
54+
55+
## License
56+
57+
[MIT](LICENSE)

0 commit comments

Comments
 (0)