diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 0000000..d53e704 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,44 @@ +name: Docs + +on: + push: + branches: [main] + paths: + - 'docs/**' + - 'mkdocs.yml' + - '.github/workflows/docs.yml' + workflow_dispatch: + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: '3.12' + cache: pip + - run: pip install "mkdocs-material>=9.5" pymdown-extensions + - run: mkdocs build --strict --site-dir site + - uses: actions/upload-pages-artifact@v3 + with: + path: site + + deploy: + needs: build + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - id: deployment + uses: actions/deploy-pages@v4 diff --git a/README.md b/README.md index 57c87b5..6162bc8 100644 --- a/README.md +++ b/README.md @@ -4,361 +4,125 @@ [![build](https://github.com/Insei/gerpo/actions/workflows/go.yml/badge.svg)](https://github.com/Insei/gerpo/actions/workflows/go.yml) [![Goreport](https://goreportcard.com/badge/github.com/insei/gerpo)](https://goreportcard.com/report/github.com/insei/gerpo) [![GoDoc](https://godoc.org/github.com/insei/gerpo?status.svg)](https://godoc.org/github.com/insei/gerpo) +[![Docs](https://img.shields.io/badge/docs-insei.github.io%2Fgerpo-blue)](https://insei.github.io/gerpo/) -Welcome to the **GERPO** repository! This document provides a brief overview of the project, build and run instructions, and other helpful information. +**GERPO** (Golang + Repository) is a generic repository pattern for Go with pluggable adapters and a tiny footprint. It is **not an ORM** β€” no migrations, no relations, no struct tags. All SQL behavior is declared once in the repository configuration; columns are bound to struct fields through pointers. -## About GERPO -**GERPO** (Golang + Repository) is a generic repository implementation with advanced configuration capabilities and easy to use builders. +> πŸ“š Full documentation: **[insei.github.io/gerpo](https://insei.github.io/gerpo/)** Β· API reference: **[pkg.go.dev/github.com/insei/gerpo](https://pkg.go.dev/github.com/insei/gerpo)** -This project under active development. +## Install -Release 1.0.0 Road Map: - * Repository builder changes: - * ~~Add Caching engine configuration in repository builder.~~ [#46](https://github.com/Insei/gerpo/pull/46) - * Column builder changes: - * New API for configuring virtual (calculated fields). Current Virtual fields configuration API marked as deprecated. - * **All other API is stable and not planned to change in 1.0.0 release.** - -Release 1.1.0 Road Map: - * Add support to retrieve inserted ID, Time/Timestamps and some other returning values from db. - * Add multiple insert support. - -### Why GERPO? -1. Support any database drivers via simple [db adapters wrappers](https://github.com/Insei/gerpo/tree/main/executor/adapters#executor-db-adapters). -2. Fast repository level implementation, oriented on microservices. -3. Easily handle CRUD operations (Create, Read, Update, Delete) with powerful filtering and sorting. -4. Already implemented pagination for list queries. -5. Straightforward configuration user-friendly builders and there needed you can use SQL commands. -6. All SQL code in one place β€” inside the configuration. -7. Virtual (calculated, joined) columns with mapping to struct fields. -8. No dependent to other libraries. Only [fmap](https://github.com/Insei/fmap) was used for working with fields pointers. -9. [Caching support](https://github.com/Insei/gerpo/tree/main/executor/cache) (currently only context-oriented cache is supported, but it’s easy to implement other caching mechanisms). +```bash +go get github.com/insei/gerpo@latest +``` -### Ideology +Minimum Go version: **1.21**. -The GERPO ideology consists of several rules: -1) If SQL code is used, then only in the repository configuration. -2) All columns are attached to entity fields via pointers to them in the repository settings. -3) There are no references in the entities that they are stored in the database (i.e. there are no tags) -4) We do not implement entity relationships. -5) We do not do migrations or other actions on database structure. +## Quick start -## Features -Essentially, **GERPO** is generic repository pattern implementation, -yes GERPO looks like ORM in some cases, but it's not an ORM. - -- **Database adapters support**: - - Any database can be used. - - You can use tracing wrappers and any other wrappers. -- **Caching Engine** - - Cache results in context for do not thinking about duplicated queries to database. - - Cache in external store, like redis. -- **Repository configuration** - - Map struct fields to SQL columns via pointers. - - Easy rename and refactor them. - - Easy delete fields. Errors can be found at build time. - - You always know where you can found columns mapping settings. - - Protect fields to insert/update in database. - - Define virtual(calculated)/joined fields. - - Add callbacks and hooks. - - Before insert. - - Before update. - - After select. - - Define persistent filters, groupings, and joins. - - Configure soft deletion. - - Use special builder that replace delete function with update with needed fields update. - - Configure Persistent filters for excluding soft deleted entities. - - Configure Errors transformer to transform GERPO errors to you business Errors. - -- **Per-query configuration**: - - Query builder allows you to set up some query rules: - - Execution fields selector allows you manage fields at update/insert operations: - - Exclude certain fields by fields pointers. - - Select only needed columns by fields pointers. - - Where builder allows you: - - Configure grouped filters. - - Support OR/AND cases. - - All of this via fields pointers. - - Order builder: - - Configure order via fields pointers. - - Work with transactions. - - Use already implemented pagination in your List queries. +```go +type User struct { + ID uuid.UUID + Name string + Email *string + Age int + CreatedAt time.Time +} -## Performance -GERPO uses a minimal amount of reflection and is designed to have minimal allocations when used, -most allocations are initialized during configuration. -We work with unsafe pointers and offsets to determine the necessary fields in -the repository configuration and when querying the database. - -I made 2 tests with absolutely identical conditions. Pure PGX V4 Pool vs GERPO over PGX V4 Pool. -Yes, we do 2x more allocations in a heap. -But I think our functionality is worth it. -In terms of time per operation, we are behind pure PGX v4 Pool by 8%. - -Well, I didn't know what the results would be at the beginning of the design. But I have ideas for optimizations. -#### Pure PGX v4 pool: -``` -BenchmarkGetOneFromDb-32 18049 65033 ns/op 1554 B/op 21 allocs/op -BenchmarkGetOneFromDb-32 18288 66617 ns/op 1555 B/op 21 allocs/op -BenchmarkGetOneFromDb-32 17860 66640 ns/op 1555 B/op 21 allocs/op -BenchmarkGetOneFromDb-32 18193 64665 ns/op 1555 B/op 21 allocs/op -BenchmarkGetOneFromDb-32 18198 65171 ns/op 1559 B/op 21 allocs/op -``` -#### GERPO: -``` -BenchmarkGetFirst/GetFirst-32 16808 69738 ns/op 2961 B/op 51 allocs/op -BenchmarkGetFirst/GetFirst-32 16614 70462 ns/op 2961 B/op 51 allocs/op -BenchmarkGetFirst/GetFirst-32 16905 70796 ns/op 2961 B/op 51 allocs/op -BenchmarkGetFirst/GetFirst-32 17059 70737 ns/op 2961 B/op 51 allocs/op -BenchmarkGetFirst/GetFirst-32 17184 69587 ns/op 2961 B/op 51 allocs/op +repo, err := gerpo.NewBuilder[User](). + DB(pgx5.NewPoolAdapter(pool)). + Table("users"). + Columns(func(m *User, c *gerpo.ColumnBuilder[User]) { + c.Field(&m.ID).AsColumn().WithUpdateProtection() + c.Field(&m.Name).AsColumn() + c.Field(&m.Email).AsColumn() + c.Field(&m.Age).AsColumn() + c.Field(&m.CreatedAt).AsColumn().WithUpdateProtection() + }). + Build() + +users, _ := repo.GetList(ctx, func(m *User, h query.GetListHelper[User]) { + h.Where().Field(&m.Age).GTE(18) + h.OrderBy().Field(&m.CreatedAt).DESC() + h.Page(1).Size(20) +}) ``` -``` -+------------------------------------------------------------+ -| | Get One/First | -+------------------+---------------------+-------------------+ -| | PURE GPX Pool v4 | GERPO PGX Pool v4 | -| | | v4 Adapter | -+------------------+---------------------+-------------------+ -| B/op | 0% | +100% | -| allocs/op | 0% | +120% | -| ns/op | 0% | +7.75% | -+------------------+---------------------+-------------------+ -``` +Full runnable samples live in [`examples/`](examples/) and in the [integration tests](tests/integration/). -## Installation -Go minimal version is `1.21`. -```bash -go get github.com/insei/gerpo@latest -``` +## Features -## Examples -Below you’ll find various configurations and usage examples. +| Area | Highlights | Docs | +|---|---|---| +| Repository | Type-safe builder, thread-safe, `sync.Pool` backed statements | [Repository builder](https://insei.github.io/gerpo/features/repository/) | +| Columns | `AsColumn` / `AsVirtual`, insert/update protection, aliases | [Columns](https://insei.github.io/gerpo/features/columns/), [Virtual columns](https://insei.github.io/gerpo/features/virtual-columns/) | +| Queries | 14 WHERE operators + IC variants, AND/OR/Group, ordering, pagination | [WHERE operators](https://insei.github.io/gerpo/features/where/), [Ordering & pagination](https://insei.github.io/gerpo/features/order-pagination/) | +| Operations | GetFirst / GetList / Count / Insert / Update / Delete with `Only` / `Exclude` | [CRUD operations](https://insei.github.io/gerpo/features/crud/), [Exclude & Only](https://insei.github.io/gerpo/features/exclude-only/) | +| Persistent queries | Always-on WHERE, JOIN, GROUP BY via `WithQuery` | [Persistent queries](https://insei.github.io/gerpo/features/persistent-queries/) | +| Soft delete | Rewrite DELETE as UPDATE of a marker field | [Soft delete](https://insei.github.io/gerpo/features/soft-delete/) | +| Hooks | Before/After for Insert/Update, AfterSelect | [Hooks](https://insei.github.io/gerpo/features/hooks/) | +| Transactions | `repo.Tx(tx)` binds a repository to any driver transaction | [Transactions](https://insei.github.io/gerpo/features/transactions/) | +| Cache | Context-scoped cache out of the box, pluggable backend | [Cache](https://insei.github.io/gerpo/features/cache/) | +| Error handling | `WithErrorTransformer` maps gerpo errors to domain errors | [Error transformer](https://insei.github.io/gerpo/features/error-transformer/) | -### Repository Configuration +## Supported drivers -#### Columns -```go -package main +| Adapter | Package | Placeholders | +|---|---|---| +| pgx v5 | `executor/adapters/pgx5` | `$1, $2, …` | +| pgx v4 | `executor/adapters/pgx4` | `$1, $2, …` | +| database/sql | `executor/adapters/databasesql` | `?` or `$1` (configurable) | -import ( - "time" - "github.com/insei/gerpo" -) +Writing a custom adapter is three methods (`ExecContext`, `QueryContext`, `BeginTx`) β€” see [Adapters](https://insei.github.io/gerpo/features/adapters/) and [adapter internals](https://insei.github.io/gerpo/architecture/adapters-internals/). -type test struct { - ID int - CreatedAt time.Time - UpdatedAt *time.Time - Name string - Age int -} +## Ideology -func main() { - repo, err := gerpo.NewBuilder[test](). - DB(dbWrap). - Table("tests"). - Columns(func(m *test, columns *gerpo.ColumnBuilder[test]) { - columns.Field(&m.ID).AsColumn().WithUpdateProtection() - columns.Field(&m.CreatedAt).AsColumn().WithUpdateProtection() - columns.Field(&m.UpdatedAt).AsColumn().WithInsertProtection() - columns.Field(&m.Name).AsColumn() - columns.Field(&m.Age).AsColumn() - }). - Build() - - // Handle err and proceed with repo usage -} -``` +1. SQL lives only in the repository configuration. +2. Columns are bound to struct fields through pointers. +3. Entities carry no database markers (no tags, no interfaces). +4. gerpo does not implement relations between entities. +5. gerpo does not modify the database schema. -#### Joins -```go -package main +Details and rationale: [Ideology](https://insei.github.io/gerpo/architecture/ideology/). -import ( - "context" - "time" - "github.com/insei/gerpo" - "github.com/insei/gerpo/query" -) +## Performance -type test struct { - ID int - CreatedAt time.Time - UpdatedAt *time.Time - Name string - Age int - Joined string -} +gerpo uses minimal reflection and pools statement objects to keep allocations under control. Measured against a real pgx v4 pool on the same query: -func main() { - repo, err := gerpo.NewBuilder[test](). - DB(dbWrap). - Table("tests"). - Columns(func(m *test, columns *gerpo.ColumnBuilder[test]) { - columns.Field(&m.ID).AsColumn().WithUpdateProtection() - columns.Field(&m.CreatedAt).AsColumn().WithUpdateProtection() - columns.Field(&m.UpdatedAt).AsColumn().WithInsertProtection() - columns.Field(&m.Name).AsColumn() - columns.Field(&m.Age).AsColumn() - columns.Field(&m.Joined).AsColumn().WithTable("joined_table") - }). - WithQuery(func(m *test, h query.PersistentHelper[test]) { - h.LeftJoin(func(ctx context.Context) string { - return "" - }) - }). - Build() - - // Handle err and proceed with repo usage -} -``` +- `ns/op`: **+8%** compared to the raw driver. +- `B/op` / `allocs/op`: ~2Γ— β€” the price of generic SQL generation and struct-field mapping. -#### Soft Deletion -```go -package main +Against a mock adapter (IO = 0) the relative overhead is larger, but absolute cost per call is β‰ˆ0.5–1.5 Β΅s. In a real database, network and query time dwarf the framework overhead. The full mock comparison matrix is produced by `GERPO_BENCH_REPORT=1 go test -run=TestCompareDirectVsGerpo -v ./tests/`. -import ( - "context" - "time" - "github.com/insei/gerpo" - "github.com/insei/gerpo/query" -) +## Roadmap -type test struct { - ID int - CreatedAt time.Time - UpdatedAt *time.Time - Name string - Age int - DeletedAt *time.Time -} +**1.0.0** -func main() { - repo, err := gerpo.NewBuilder[test](). - DB(dbWrap). - Table("tests"). - Columns(func(m *test, columns *gerpo.ColumnBuilder[test]) { - columns.Field(&m.ID).AsColumn().WithUpdateProtection() - columns.Field(&m.CreatedAt).AsColumn().WithUpdateProtection() - columns.Field(&m.UpdatedAt).AsColumn().WithInsertProtection() - columns.Field(&m.Name).AsColumn() - columns.Field(&m.Age).AsColumn() - columns.Field(&m.DeletedAt).AsColumn().WithInsertProtection() // configure soft deletion field/column - }). - WithSoftDeletion(func(m *User, softDeletion *gerpo.SoftDeletionBuilder[User]) { - //Configure set value for soft deletion fields/columns - softDeletion.Field(&m.DeletedAt).SetValueFn(func(ctx context.Context) any { - deletedAt := time.Now().UTC() - return &deletedAt - }) - }). - WithQuery(func(m *test, h query.PersistentHelper[test]) { - // Permanently exclude deleted elements from all queries - h.Where().Field(&m.DeletedAt).EQ(nil) - }). - Build() - - // Handle err and proceed with repo usage -} -``` +- [x] Caching engine configuration in the repository builder (#46). +- [ ] New API for configuring virtual columns (current one marked deprecated). -### Per-request Configuration +The rest of the API is stable and not expected to change in 1.0.0. -#### Exclude -Exclude certain fields from commands like SELECT/UPDATE/INSERT (Update/GetFirst/Insert/GetList): -```go -package main - -import ( - "context" - "github.com/insei/gerpo" - "github.com/insei/gerpo/query" -) - -type test struct { - ID int - CreatedAt time.Time - UpdatedAt *time.Time - Name string - Age int - Joined string -} +**1.1.0** -func main() { - var repo gerpo.Repository[test] // Already initialized - list, err := repo.GetList(ctx, func(m *test, h query.GetListHelper[test]) { - h.Page(1).Size(2) // Pagination - h.Exclude(&m.UpdatedAt, &m.ID) - }) - // Handle err and work with the list -} -``` +- Return inserted IDs and generated timestamps. +- Batch Insert. -#### Where -Available for Count/GetFirst/GetList/Delete/Update, supporting where grouping (AND/OR): -```go -package main - -import ( - "context" - "github.com/insei/gerpo" - "github.com/insei/gerpo/query" -) - -type test struct { - ID int - CreatedAt time.Time - UpdatedAt *time.Time - Name string - Age int - Joined string -} +## Contributing -func main() { - var repo gerpo.Repository[test] // Already initialized - list, err := repo.GetList(ctx, func(m *test, h query.GetListHelper[test]) { - h.Where().Field(&m.ID).LT(7) // Items with ID < 7 - }) - // Handle err and use the list -} -``` +- Unit tests: `go test ./...` +- Integration tests (Docker required): -#### Order -Available for GetFirst/GetList: -```go -package main - -import ( - "context" - "github.com/insei/gerpo" - "github.com/insei/gerpo/query" -) - -type test struct { - ID int - CreatedAt time.Time - UpdatedAt *time.Time - Name string - Age int - Joined string -} + ```bash + docker compose -f tests/integration/docker-compose.yml up -d + GERPO_INTEGRATION_DB_URL="postgres://gerpo:gerpo@localhost:5433/gerpo?sslmode=disable" \ + go test -tags=integration ./tests/integration/... + ``` -func main() { - var repo gerpo.Repository[test] // Already initialized - item, err := repo.GetFirst(ctx, func(m *test, h query.GetFirstHelper[test]) { - h.OrderBy().Field(&m.CreatedAt).DESC() - }) - // Handle err and use 'item' -} -``` +- Every PR runs a mock-db benchmark diff via `benchstat` and posts the summary as a PR comment. + +More in [Contributing](https://insei.github.io/gerpo/architecture/contributing/). ---- +## License -We hope this information helps you quickly get started with **GERPO** and integrate it into your own projects. If you have any questions or suggestions, feel free to open an issue or contribute to the repository. -## Documentation: -* repository (main repository code - uses sqlstmt, query and executor for work, execute hooks, callbacks and error transformer) -* query (User API Query interface - works with sqlstmt via linq API) - * linq (Internal Query API interface - works with sqlstmt) -* sqlstmt (Internal SQL queries generator interface - generates SQL queries and stores arguments) -* [executor](https://github.com/Insei/gerpo/tree/main/executor) (Internal SQL Queries executor API - execute SQL queries and map values to entities) +MIT β€” see [LICENSE.md](LICENSE.md). diff --git a/docs/architecture/adapters-internals.md b/docs/architecture/adapters-internals.md new file mode 100644 index 0000000..564ce06 --- /dev/null +++ b/docs/architecture/adapters-internals.md @@ -0,0 +1,70 @@ +# Adapter internals + +An adapter is a thin wrapper that turns `executor.DBAdapter` calls into driver-specific calls. All bundled adapters follow the same layout; differences come down to placeholder rewriting and transaction semantics. + +## Anatomy of a wrapper + +``` +executor/adapters// + pool.go β€” NewPoolAdapter / NewAdapter + ExecContext / QueryContext / BeginTx + tx.go β€” txWrap implementing executor.Tx + rows.go β€” rowsWrap adapting driver rows to types.Rows + result.go β€” resultWrap adapting driver result to types.Result +``` + +The rest is boilerplate. + +## Placeholder rewriting + +gerpo's SQL uses `?`. Each adapter rewrites placeholders exactly once, right before handing the query to the driver: + +```go +sql, err := placeholder.Dollar.ReplacePlaceholders(query) +``` + +`executor/adapters/placeholder/` provides two formats: + +- `placeholder.Question` β€” no-op (input already `?`). +- `placeholder.Dollar` β€” scan-and-emit rewriter that turns `?` into `$1, $2, …`. + +`databasesql.NewAdapter` defaults to `Question`. `pgx4` / `pgx5` always use `Dollar`. + +## Rows wrapper + +pgx returns `pgx.Rows`, `database/sql` returns `*sql.Rows`. Both shapes are close enough to the `types.Rows` interface, but they differ in `Scan` behavior (nullable types, text decoding). `rowsWrap` exists so gerpo can pretend both are identical. + +## Result wrapper + +`types.Result` exposes only `RowsAffected() (int64, error)`. Both pgx and `database/sql` return something richer, but gerpo only needs this one metric. + +## Transaction wrapper + +`txWrap` stores: + +```go +type txWrap struct { + commited bool + rollbackUnlessCommittedNeeded bool + tx .Tx // or *sql.Tx +} +``` + +- `Commit()` β€” calls driver commit, then sets `commited = true` on **success**. +- `Rollback()` β€” sets `rollbackUnlessCommittedNeeded = false`, then calls driver rollback. +- `RollbackUnlessCommitted()` β€” if `!commited && rollbackUnlessCommittedNeeded`, delegates to `Rollback()`; otherwise no-op. Designed to be safe as a `defer`. + +All three methods use pointer receivers so the state mutations actually stick. + +!!! warning "Historical bug" + pgx v4 and v5 adapters originally used value receivers and also forgot to set `commited`. `RollbackUnlessCommitted()` after `Commit()` returned `tx is closed`. The integration test `TestTx_RollbackUnlessCommitted_AfterCommit` catches this; fixed in the `test: cover hooks, soft delete, …` commit. + +## Writing your own + +Walk through `executor/adapters/pgx5/` as a template. You will need: + +- decide whether to rewrite placeholders (most non-PG drivers keep `?`; PG-derived drivers want `$N`); +- wrap the driver's `Rows` type in something satisfying `types.Rows`; +- wrap the driver's transaction type in `txWrap` following the rules above; +- return a `types.DBAdapter` implementation. + +A good smoke test is `TestSmoke` in `tests/integration/` β€” `forEachAdapter` will pick up your new bundle once you add it to `allAdapters()`. diff --git a/docs/architecture/caching.md b/docs/architecture/caching.md new file mode 100644 index 0000000..9c3f35f --- /dev/null +++ b/docs/architecture/caching.md @@ -0,0 +1,71 @@ +# Caching internals + +The cache is a plug-in. `executor/cache/types.Storage` is the interface; `executor/cache/ctx` is the bundled context-scoped implementation. + +## The Storage interface + +```go +type Storage interface { + Get(ctx context.Context, stmt string, args ...any) (any, error) + Set(ctx context.Context, value any, stmt string, args ...any) + Clean(ctx context.Context) +} +``` + +- **Get** returns the cached value or `cache.ErrNotFound`. +- **Set** records a value. +- **Clean** wipes the cache (the executor calls it after INSERT/UPDATE/DELETE so stale reads don't linger). + +The executor is the only caller. `executor/cache.go` wraps `Storage` in three helpers (`get[T]`, `set`, `clean`) that accept a nil storage and no-op β€” so switching the cache off is as simple as not passing `executor.WithCacheStorage`. + +## CtxCache + +Source: `executor/cache/ctx/source.go`, `executor/cache/ctx/storage.go`. + +The idea: the cache payload lives **inside the request context**. A repo-level `CtxCache` object has a stable UUID-shaped `key` that lets it partition the context-scoped storage by repo. + +``` + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” + β”‚ context.Context β”‚ + β”‚ β”‚ + β”‚ ctxCacheKey β†’ *cacheStorage { β”‚ + β”‚ mtx: sync.Mutex β”‚ + β”‚ c: map[string]map[string]any β”‚ + β”‚ β”‚ + β”‚ ↑ ↑ β”‚ + β”‚ repo A's key repo B's key β”‚ + β”‚ β”‚ + β”‚ } β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +- `ctx.NewCtxCache(ctx)` installs a `cacheStorage` into the context. +- Repo A's reads/writes go to `storage.c["repo-A-uuid"]`. +- On `Clean` only the repo's bucket is cleared, not the whole tree. + +`cacheStorage.Get` looks up `modelKey β†’ key β†’ value`. `modelKey` is the repo UUID; `key` is `sql + args`. + +## Cache key + +`CtxCache.Get/Set` build a string key from the SQL statement and its arguments. Until recently this went through `fmt.Sprintf("%s%v", sql, args)`, which allocated a slice of strings plus the formatted copy. Now keys go through a `strings.Builder` with a type switch covering common argument types (`string`, integers, `uuid.UUID`, `[]byte`, `bool`, `nil`) β€” each of those writes directly into the builder. Uncommon types fall back to `fmt.Fprint`. + +Result: 3–5 fewer allocations per cache operation, no change in key identity. + +## Invalidation + +The executor calls `clean(ctx, cacheSource)` after successful `InsertOne`, `Update`, `Delete`. Read operations don't invalidate. Thus: + +- a repo-managed mutation always clears the repo's cache bucket for the current request context; +- changes made to the database through a different path (raw SQL, another repo, an external service) are invisible to the cache until someone writes through the repo in that context. + +## Thread safety + +`cacheStorage` is protected by a `sync.Mutex`. The lock is fine-grained per write and per read β€” contention is low in practice because most requests touch distinct keys. + +## Building your own Storage + +Anything satisfying `cache.Storage` works. Common candidates: + +- Redis β€” for cross-instance sharing within a trace/tenant. +- `sync.Map` with TTL β€” for a process-wide cache, but mind invalidation semantics. +- `cache.NewModelBundle` β€” already bundled; combines several storages into one (e.g. ctx + redis). diff --git a/docs/architecture/contributing.md b/docs/architecture/contributing.md new file mode 100644 index 0000000..a9aca7b --- /dev/null +++ b/docs/architecture/contributing.md @@ -0,0 +1,71 @@ +# Contributing + +## Development environment + +- Go 1.21+. +- Docker for integration tests. +- `mkdocs-material` if you want to preview the docs locally (`pip install mkdocs-material && mkdocs serve`). + +## Check-in loop + +```bash +# Unit tests + race detector +go test -race ./... + +# Integration tests (PostgreSQL in Docker) +docker compose -f tests/integration/docker-compose.yml up -d +GERPO_INTEGRATION_DB_URL="postgres://gerpo:gerpo@localhost:5433/gerpo?sslmode=disable" \ + go test -tags=integration ./tests/integration/... + +# Direct-vs-gerpo allocation benchmarks +go test -bench='^Benchmark(GetFirst|GetList|Count|Insert|Update|Delete)_(Direct|Gerpo)$' \ + -benchmem -run=^$ -count=5 ./tests/ + +# Formatted summary +GERPO_BENCH_REPORT=1 go test -run=TestCompareDirectVsGerpo -v ./tests/ +``` + +## Code style + +- Package names are lowercase and short. +- Interfaces end in `-er` / `-or` when they describe behaviour (`DBAdapter`, `WhereTarget`, `Operation`). +- Every public API ships with godoc β€” keep the tone concise. +- Generic parameter for the model is `[TModel any]`, consistently. + +## Tests + +- Unit tests live beside the code (`*_test.go`). Use `go-sqlmock` for `database/sql` paths. +- Integration tests go under `tests/integration/` with the `//go:build integration` tag. They target every adapter in a single run through `forEachAdapter`. +- Benchmarks live in `tests/` (no build tag β€” `go test -bench=`). + +## Commit style + +Follow the existing log: lowercase type prefix, imperative subject, optional body with bullet points: + +``` +perf: replace closures in query/linq builders with structured ops + +Where/Order/Exclude/Group/Join no longer store per-condition closures … +``` + +Common types used in the repo: `feat:`, `fix:`, `perf:`, `test:`, `docs:`, `ci:`, `build:`, `refactor:`, `src:`. + +## Opening a PR + +A PR to `main` runs three jobs: + +- `unit` β€” build, race detector, full `go test`. +- `integration` β€” `//go:build integration` against a PG service container. +- `bench-diff` β€” runs mock benchmarks on head and on base, posts a [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat) summary as a PR comment. + +`bench-diff` is `allow_failure: true` β€” a perf regression shows up in the comment but doesn't block merging on its own. Look at the comment before asking for review. + +## Updating the docs + +- English only. +- Prefer runnable snippets. If the snippet would need imports to compile, pick an example from `examples/` or from the integration tests and copy it verbatim. +- Don't duplicate godoc β€” link to `pkg.go.dev` instead. + +## Releasing + +Tag `vX.Y.Z`, push the tag. Prior to 1.0.0 the API is not guaranteed to be stable β€” call out anything breaking in the release notes. diff --git a/docs/architecture/field-mapping.md b/docs/architecture/field-mapping.md new file mode 100644 index 0000000..2e654cc --- /dev/null +++ b/docs/architecture/field-mapping.md @@ -0,0 +1,51 @@ +# Field mapping + +gerpo's pointer-based column binding is powered by [`github.com/insei/fmap/v3`](https://github.com/insei/fmap), a small library that learns the layout of a struct once and exposes O(1) access to each field through its pointer address. + +## Why not reflect.Type? + +Classic reflection identifies a field by name or index. Both are brittle: renames silently shift indices, typos escape the compiler. gerpo wants code like + +```go +c.Field(&m.Email).AsColumn() +``` + +where `&m.Email` is the reference. That needs a trick: convert a pointer-to-field into a stable field identifier. + +## How fmap does it + +1. On `gerpo.NewBuilder[T]()`, gerpo creates a zero `T` on the heap once. +2. `fmap` walks the struct using reflection and records each field's `unsafe.Offsetof`. +3. For every `c.Field(&m.X)` the builder takes the pointer `&m.X`, subtracts the base address of the zero-struct, and looks up the field by offset. O(1). +4. At query time, when the repo has a real `*T`, it reads/writes the field by adding the offset back β€” `unsafe.Pointer` arithmetic without `reflect.Value`. + +Pointer-to-field resolution happens once per repo build, not per request. Per-request, gerpo only does pointer arithmetic and interface boxing β€” no `reflect.Value` on hot paths. + +## What fmap returns + +`fmap.Field` implements enough to: + +- fetch the Go type (`GetType`, `GetDereferencedType`) β€” used when generating operators for a given column; +- read the value (`Get(model)`) β€” used by `GetModelValues` when building INSERT/UPDATE; +- write the value (`Set(model, val)`) β€” used by `SoftDeletionBuilder.SetValueFn`; +- obtain a pointer (`GetPtr(model)`) β€” used by `GetModelPointers` for `rows.Scan`. + +`column.Column` wraps an `fmap.Field` plus the operator table, table name, alias, and allowed SQL actions. + +## Allocation behaviour + +fmap allocates once per struct (not per field, not per query). Its `Set` uses a compiled set of pointer-tricks per Go kind β€” no `reflect.Call`, no boxing on the hot path. + +gerpo's own overhead on top is: + +- one `[]any` for `GetModelPointers` (required by `rows.Scan`); +- one `[]any` for `GetModelValues` on INSERT/UPDATE (required by the driver to marshal parameters); +- one allocation per closure stored in the WHERE/ORDER plan. + +Those slices are a known allocation source β€” see the backlog of allocation ideas in the repository's memory. + +## Limitations + +- Nested anonymous structs are supported; unexported fields are not. +- Interface-typed fields can be columns, but conversion to SQL values depends on the driver. Prefer concrete types. +- `fmap` uses `unsafe`, so the usual caveats apply: struct layout must match what `fmap` saw at init, i.e. no hot-swapping types or binary-incompatible upgrades. diff --git a/docs/architecture/ideology.md b/docs/architecture/ideology.md new file mode 100644 index 0000000..d60987d --- /dev/null +++ b/docs/architecture/ideology.md @@ -0,0 +1,63 @@ +# Ideology + +The five rules gerpo commits to (from the README): + +1. **If SQL exists in your project, it lives only in the repository configuration.** +2. **Every column is bound to a struct field through a pointer.** Not by name, not by tag. +3. **Entities carry no database markers.** No tags, no special interfaces. +4. **We do not implement relations between entities.** No `hasMany`, no `belongsTo` β€” that's the business layer's job. +5. **We do not modify the database schema.** No migrations, no `CREATE`/`ALTER` β€” gerpo only reads and writes data. + +These rules aren't stylistic preferences β€” they are constraints that define the shape of the whole library. Any proposal that breaks one of them is almost certainly heading toward an ORM, and gerpo is deliberately not an ORM. + +## Why "not an ORM"? + +- **ORM models** try to hide SQL and table structure behind an object model. But SQL leaks: through N+1, through implicit migrations, through queries you can't express. Sooner or later, the team has to understand both layers anyway. +- **gerpo's repository model** is honest: yes, this is a SQL database, yes, there are tables. You work with them directly β€” just with the convenience of a type-safe configuration. + +## The cost and the payoff of each rule + +### 1. SQL only in the config + +**Cost:** you have to describe columns and persistent conditions upfront; you cannot slip a JOIN into a handler "just for this call". + +**Payoff:** exactly one place where the schema is read from. SQL changes are always visible in the PR. + +### 2. Bindings via pointers + +**Cost:** one struct field = one pointer in the config. No `c.Column("name")`. + +**Payoff:** renames are a plain refactor, typos are caught by the compiler. You can't misspell a column name in a string. + +### 3. No database markers on entities + +**Cost:** a plain Go struct with no schema information β€” you keep it in a separate config. + +**Payoff:** a domain entity stays a domain entity. It doesn't drag `json:"foo" db:"bar" validate:"…"` along with it β€” those concerns live where they should. + +### 4. No relations + +**Cost:** if there's a 1-N relationship between `User` and `Post`, you write the matching methods yourself (`FindPostsByUser`) β€” gerpo offers no magic navigation. + +**Payoff:** no lazy-load tornadoes, a predictable number of queries. + +### 5. No migrations + +**Cost:** the database schema is managed by a separate tool (`golang-migrate`, `goose`, `atlas`, …). + +**Payoff:** one layer, one responsibility. gerpo happily survives any schema-versioning scheme. + +## Side-by-side summary + +| | GORM / ent | gerpo | +|---|---|---| +| SQL hidden | βœ” | ✘ (visible in the config) | +| Migrations | βœ” | ✘ | +| Relations | βœ” | ✘ | +| Struct tags | βœ” | ✘ | +| Field pointers | ✘ | βœ” | +| CRUD + WHERE DSL | βœ” | βœ” | +| Multiple drivers | βœ” | βœ” | +| Context-aware cache | partial | βœ” | + +gerpo is intentionally **smaller**, and that is its value proposition. diff --git a/docs/architecture/index.md b/docs/architecture/index.md new file mode 100644 index 0000000..ba61715 --- /dev/null +++ b/docs/architecture/index.md @@ -0,0 +1,30 @@ +# Architecture + +This section is for people who want to know gerpo from the inside β€” contributors, reviewers, and anyone building custom adapters or features on top of it. + +| Page | Topic | +|---|---| +| [Ideology](ideology.md) | The five rules that shape the library | +| [Layers](layers.md) | How a request travels from `Repository` down to the driver | +| [SQL generation](sql-generation.md) | `sqlstmt` and `sqlpart` β€” assembling the SQL text | +| [Field mapping](field-mapping.md) | How gerpo sees struct fields through pointers | +| [Caching internals](caching.md) | Inside `CtxCache` | +| [Adapter internals](adapters-internals.md) | How an adapter is written β€” placeholder rewrite, `Rows`, transactions | +| [Contributing](contributing.md) | Building, testing, and shipping a change | + +## TL;DR + +``` +Repository[T] ─► query/*Helper[T] ─► query/linq (internal builders) + β”‚ + β–Ό + sqlstmt (SQL codegen) ──► sqlstmt/sqlpart + β”‚ + β–Ό + executor (run + cache + map) ──► executor/cache/* + β”‚ + β–Ό + executor/adapters/{pgx5,pgx4,databasesql} +``` + +Each arrow is a request object moving down a layer. `Repository` doesn't know about the driver; the driver doesn't know about the model. In between there's pure SQL generation and value mapping. diff --git a/docs/architecture/layers.md b/docs/architecture/layers.md new file mode 100644 index 0000000..a297cd7 --- /dev/null +++ b/docs/architecture/layers.md @@ -0,0 +1,87 @@ +# Layers + +A request flows top-down through four layers. Each layer has a narrow job and communicates with the next one through a small interface. + +``` +Repository[T] (gerpo/*.go) + β”‚ user calls: GetFirst / GetList / Count / Insert / Update / Delete + β–Ό +query/*Helper[T] (query/*.go) + β”‚ per-request helpers that collect user intent + β–Ό +query/linq (query/linq/*.go) + β”‚ struct-based Where/Order/Exclude builders + β–Ό +sqlstmt + sqlpart (sqlstmt/*.go, sqlstmt/sqlpart/*.go) + β”‚ emit SQL text and bound arguments + β–Ό +executor (executor/*.go) + β”‚ run the statement, scan rows into the model, manage the cache + β–Ό +executor/adapters/* (executor/adapters/pgx5 | pgx4 | databasesql) + β”‚ driver-specific IO and placeholder rewriting + β–Ό +database +``` + +## Public layer β€” `Repository[T]` + +Source: `repository.go`, `builder.go`, `types.go`, `options.go`, `soft.go`, `column.go`. + +Every public method follows the same recipe: + +1. Obtain a pooled `stmt` object from `sqlstmt` (`NewGetFirst`, `NewInsert`, …). +2. `defer stmt.Release()` to return the object to its `sync.Pool`. +3. Apply the persistent query: `r.persistentQuery.Apply(stmt)`. +4. Apply the per-call `query.Helper`: `q.Apply(stmt)`. +5. Hand the statement over to the executor. +6. Run `afterSelect` / `afterInsert` / `afterUpdate` as appropriate. +7. Pass errors through `errorTransformer`. + +Every `func` in this layer is a thin orchestration step β€” no SQL, no reflection. + +## `query/*Helper[T]` + +Source: `query/*.go`. + +One helper per operation: `GetFirstHelper`, `GetListHelper`, `CountHelper`, `InsertHelper`, `UpdateHelper`, `DeleteHelper`, and the special `PersistentHelper` for `WithQuery`. + +They do not run queries themselves β€” they only collect user intent into structured objects from `query/linq` (WhereBuilder, OrderBuilder, ExcludeBuilder, PaginationBuilder). When `Apply` runs, those builders walk their internal op-slices and push work into `sqlpart` builders. + +## `query/linq` β€” struct-based builders + +Source: `query/linq/*.go`. + +Prior to an internal refactor (see the `perf: replace closures…` commit), every operation was stored as a closure. Now each builder keeps a typed slice of `OpEntry` structs, and `Apply` is a `switch` dispatching on `kind`. This saved one allocation per condition and made the flow traceable in a debugger. + +## `sqlstmt` + `sqlpart` + +Source: `sqlstmt/*.go` and `sqlstmt/sqlpart/*.go`. + +`sqlstmt` has one type per operation (`GetFirst`, `GetList`, `Count`, `Insert`, `Update`, `Delete`) that knows the SQL shape of its statement. `sqlpart` supplies reusable assemblers: `WhereBuilder`, `OrderBuilder`, `JoinBuilder`, `GroupBuilder`, `LimitOffsetBuilder`. + +Every `sqlstmt` struct exposes `SQL() (string, []any, error)` β€” the final text and bound arguments. `GetFirst/GetList/Count` also have a `Release()` method that resets state and returns to a `sync.Pool`. See [SQL generation](sql-generation.md). + +## `executor` + +Source: `executor/executor.go`, `executor/types.go`, `executor/cache.go`. + +The executor is the one place where IO happens. Responsibilities: + +- Call the cache (`get`/`set`/`clean`) if a `cache.Storage` is wired. +- Delegate `ExecContext`/`QueryContext` to the adapter (or to a `Tx` when we're inside a transaction). +- Scan result rows via `stmt.Columns().GetModelPointers(model)`. + +Caching, if any, is driven entirely by the `cache.Storage` interface β€” no assumptions about the backend. + +## `executor/adapters/*` + +Source: `executor/adapters/{pgx5,pgx4,databasesql,placeholder}/*.go`. + +Three things live here: + +- `NewPoolAdapter` / `NewAdapter` constructors producing a `DBAdapter`. +- `poolWrap` / `dbWrap` that translate the interface into driver-specific calls and rewrite placeholders. +- `txWrap` implementing `Tx` on top of the driver's own transaction type. + +See [Adapter internals](adapters-internals.md). diff --git a/docs/architecture/sql-generation.md b/docs/architecture/sql-generation.md new file mode 100644 index 0000000..4222b06 --- /dev/null +++ b/docs/architecture/sql-generation.md @@ -0,0 +1,67 @@ +# SQL generation + +Every operation has a dedicated `sqlstmt` type; each type emits SQL by concatenating pieces produced by `sqlstmt/sqlpart` builders. + +## Statement types + +| Type | File | SQL shape | +|---|---|---| +| `GetFirst` | `sqlstmt/first.go` | `SELECT … FROM t [JOIN] [WHERE] [GROUP BY] [ORDER BY] LIMIT 1` | +| `GetList` | `sqlstmt/list.go` | `SELECT … FROM t [JOIN] [WHERE] [GROUP BY] [ORDER BY] [LIMIT … OFFSET …]` | +| `Count` | `sqlstmt/count.go` | `SELECT count(*) over() AS count FROM t [JOIN] [WHERE] [GROUP BY] LIMIT 1` | +| `Insert` | `sqlstmt/insert.go` | `INSERT INTO t (cols…) VALUES (?, …)` | +| `Update` | `sqlstmt/update.go` | `UPDATE t SET col = ?, … [WHERE]` | +| `Delete` | `sqlstmt/delete.go` | `DELETE FROM t [JOIN] [WHERE]` | + +Each type implements `executor.Stmt` (or `CountStmt`) and exposes `SQL() (string, []any, error)`. + +## sqlpart builders + +`sqlstmt/sqlpart/` holds reusable fragment builders. Each of them keeps an internal buffer plus a `Reset(ctx)` method so the parent statement can recycle it across calls. + +| Builder | Output | +|---|---| +| `WhereBuilder` | ` WHERE a = ? AND (b = ? OR c IS NULL)` | +| `JoinBuilder` | ` LEFT JOIN posts ON … INNER JOIN tags ON …` | +| `OrderBuilder` | ` ORDER BY created_at DESC, id ASC` | +| `GroupBuilder` | ` GROUP BY id, name` | +| `LimitOffsetBuilder` | ` LIMIT 20 OFFSET 40` | + +Each returns an empty string when it has nothing to emit, so the parent can unconditionally concatenate without worrying about stray whitespace. + +## Operator-to-SQL mapping + +`sqlstmt/sqlpart/where.go` holds one factory per operator (`genEQFn`, `genLTFn`, `genINFn`, …). Factories are invoked once per column at repository build time, producing `func(ctx, value) (string, bool)` closures cached inside the column object. At query time a WHERE condition looks up the matching factory by operator name and appends the resulting SQL fragment with the argument (if any). + +The LIKE family wraps parameters in `CAST(? AS text)` so PostgreSQL can infer the parameter type inside `CONCAT(…)`. + +## Object pooling + +`GetFirst`, `GetList`, and `Count` live in a `sync.Pool`. Lifecycle: + +1. `NewGetFirst(ctx, table, cols)` β†’ take from pool, call `reset(ctx, cols)`, return ready object. +2. Repository uses it. +3. `defer stmt.Release()` β†’ zero mutable fields, return to pool. + +`sqlselect` (the shared part of `GetFirst`/`GetList`/`Count`) owns an instance of each `sqlpart` builder. `sqlselect.reset` re-parents them to the new context and clears their internal buffers without giving up the backing memory β€” so a hot repository steadily reuses the same byte slices. + +The pool can shrink under GC pressure; statement objects are designed to be cheap to allocate from scratch too, so there's no risk of starvation. + +## Assembly flow (GetFirst example) + +``` +repository.GetFirst(ctx, qFns...) + β”œβ”€β”€ NewGetFirst(ctx, table, columns) // from pool + β”œβ”€β”€ persistentQuery.Apply(stmt) // WHERE, JOIN, GROUP BY injected + β”œβ”€β”€ query.NewGetFirst(model).HandleFn(qFns...) + β”‚ .Apply(stmt) // per-request WHERE, ORDER, EXCLUDE + β”œβ”€β”€ executor.GetOne(ctx, stmt) + β”‚ β”œβ”€β”€ stmt.SQL() + β”‚ β”œβ”€β”€ cache.get(ctx, sql, args...) // optional + β”‚ β”œβ”€β”€ adapter.QueryContext(...) + β”‚ β”œβ”€β”€ rows.Scan(columns.GetModelPointers(m)...) + β”‚ └── cache.set(...) + └── stmt.Release() // back to pool +``` + +The executor, the adapter, and the cache are interchangeable; the pipeline stays the same. diff --git a/docs/features/adapters.md b/docs/features/adapters.md new file mode 100644 index 0000000..cf85f57 --- /dev/null +++ b/docs/features/adapters.md @@ -0,0 +1,119 @@ +# Adapters + +gerpo never talks to a specific driver directly β€” it communicates through the `executor.DBAdapter` interface. Three implementations ship in the box. + +## Bundled adapters + +### pgx v5 + +```go +import ( + "github.com/insei/gerpo/executor/adapters/pgx5" + "github.com/jackc/pgx/v5/pgxpool" +) + +pool, _ := pgxpool.New(ctx, dsn) +adapter := pgx5.NewPoolAdapter(pool) +``` + +Placeholders: `$1, $2, …`. + +### pgx v4 + +```go +import ( + "github.com/insei/gerpo/executor/adapters/pgx4" + "github.com/jackc/pgx/v4/pgxpool" +) + +pool, _ := pgxpool.Connect(ctx, dsn) +adapter := pgx4.NewPoolAdapter(pool) +``` + +Identical API, just a different pgx major. + +### database/sql + +A universal adapter for any `*sql.DB`. Defaults to `?` placeholders (MySQL-compatible). For PostgreSQL switch to `$1` explicitly: + +```go +import ( + "database/sql" + _ "github.com/jackc/pgx/v5/stdlib" + + "github.com/insei/gerpo/executor/adapters/databasesql" + "github.com/insei/gerpo/executor/adapters/placeholder" +) + +db, _ := sql.Open("pgx", dsn) +adapter := databasesql.NewAdapter(db, databasesql.WithPlaceholder(placeholder.Dollar)) +``` + +## The `DBAdapter` interface + +To write a custom adapter β€” implement three methods: + +```go +type DBAdapter interface { + ExecContext(ctx context.Context, query string, args ...any) (Result, error) + QueryContext(ctx context.Context, query string, args ...any) (Rows, error) + BeginTx(ctx context.Context) (Tx, error) +} +``` + +`Result`, `Rows`, `Tx` live in `executor/types`: + +```go +type Rows interface { + Next() bool + Scan(dest ...any) error + Close() error +} + +type Result interface { + RowsAffected() (int64, error) +} + +type Tx interface { + ExecQuery + Commit() error + Rollback() error + RollbackUnlessCommitted() error +} +``` + +## Why write a custom adapter + +- **Tracing** β€” wrap an existing adapter and add spans/logs around `ExecContext`/`QueryContext`. +- **A different driver** β€” ClickHouse, SQLite, MSSQL. +- **Mocks** β€” this is exactly how the mock benchmarks and some unit tests are wired (see `tests/mockdb_test.go`). + +A small tracing wrapper: + +```go +type tracingAdapter struct { + inner executor.DBAdapter + tr trace.Tracer +} + +func (a *tracingAdapter) QueryContext(ctx context.Context, q string, args ...any) (types.Rows, error) { + ctx, span := a.tr.Start(ctx, "db.query") + span.SetAttributes(attribute.String("db.statement", q)) + rows, err := a.inner.QueryContext(ctx, q, args...) + if err != nil { + span.RecordError(err) + } + span.End() + return rows, err +} +// same idea for ExecContext and BeginTx +``` + +## Placeholder rewriting + +Internally gerpo emits `?` placeholders. Each adapter decides whether to rewrite them: + +- `pgx5` / `pgx4` β€” rewrite to `$1, $2, …` (`placeholder.Dollar`). +- `databasesql` β€” configurable via `WithPlaceholder(placeholder.Question | placeholder.Dollar)`. + +If your driver accepts `?`, no rewriting is needed. diff --git a/docs/features/cache.md b/docs/features/cache.md new file mode 100644 index 0000000..6f65ea1 --- /dev/null +++ b/docs/features/cache.md @@ -0,0 +1,68 @@ +# Cache + +gerpo can attach a `cache.Storage` to the executor. The built-in implementation is `CtxCache`: a cache scoped to a single `context.Context`. It helps when one business operation fetches the same records multiple times. + +## Wiring + +```go +import ( + "github.com/insei/gerpo" + "github.com/insei/gerpo/executor" + cachectx "github.com/insei/gerpo/executor/cache/ctx" +) + +c := cachectx.New() // one instance per repo + +repo, _ := gerpo.NewBuilder[User](). + DB(adapter, executor.WithCacheStorage(c)). + Table("users"). + Columns(/* … */). + Build() +``` + +For each request you must **wrap the ctx**: + +```go +reqCtx := cachectx.NewCtxCache(ctx) + +// every subsequent call should go through reqCtx +repo.GetFirst(reqCtx, whereByID) +repo.GetFirst(reqCtx, whereByID) // ← hit, served from cache +``` + +Without `NewCtxCache(ctx)` the cache just does nothing β€” a warning goes to the log, and the queries themselves still work. + +## Behavior + +| Operation | Effect | +|---|---| +| `GetFirst`, `GetList`, `Count` | Read and fill the cache by `sql + args` | +| `Insert`, `Update`, `Delete` | **Clear** the repo's cache (`Clean`) | +| External change to the DB | Not observed by the cache β€” a stale value is served until the next `Insert/Update/Delete` through the repo or until the context ends | + +## When it helps + +- **N+1 protection:** one business call hits `repo.GetFirst(ctx, id)` from several places β€” the cache returns the first result. +- **Proxying middleware:** wrap an incoming HTTP request in `NewCtxCache`, and the whole handler/service tree shares the cache. + +## When to skip + +- You need up-to-date reads more than you need fewer round-trips (e.g. view-after-write). +- A long-running business operation β€” the cache has no TTL. +- The same gerpo repository is read across different contexts β€” distinct contexts have independent caches by design. + +## Custom cache backend + +`executor.WithCacheStorage` accepts any `cache.Storage` β€” the interface is defined in `executor/cache/types`. You can implement Redis, memcached, or a `sync.Map` with your own policy. + +```go +type Storage interface { + Get(ctx context.Context, stmt string, args ...any) (any, error) + Set(ctx context.Context, value any, stmt string, args ...any) + Clean(ctx context.Context) +} +``` + +## Key performance + +Starting with the version that introduced the integration tests, cache keys are built with `strings.Builder` plus a type switch over common parameter types β€” no `fmt.Sprintf`. That saves 3–5 allocations per cache operation. diff --git a/docs/features/columns.md b/docs/features/columns.md new file mode 100644 index 0000000..e01949a --- /dev/null +++ b/docs/features/columns.md @@ -0,0 +1,76 @@ +# Columns + +Columns are described inside `Columns(func(m *T, c *gerpo.ColumnBuilder[T]))`. The binding is done **through a pointer to the struct field** β€” not via a string, not via a tag. The benefit: renaming a field is a regular refactor, and the compiler catches every mistake. + +## Two kinds of columns + +```go +c.Field(&m.Name).AsColumn() // regular, read and written +c.Field(&m.FullName).AsVirtual() // computed, read-only + .WithSQL(func(ctx context.Context) string { return "first_name || ' ' || last_name" }) +``` + +### `AsColumn` β€” regular column + +Maps to a physical table column. The default column name is `snake_case(field)` (e.g. `CreatedAt β†’ created_at`). Override with `WithColumnName`. + +### `AsVirtual` β€” virtual column + +Not stored in the table β€” computed by a SQL expression on the fly. Automatically protected from both INSERT and UPDATE. See [Virtual columns](virtual-columns.md). + +## Options on regular columns + +| Option | Effect | +|---|---| +| `WithColumnName(string)` | SQL column name (defaults to snake_case of the field name) | +| `WithTable(string)` | Table name β€” useful for columns coming from a JOIN | +| `WithAlias(string)` | Alias in SELECT | +| `WithInsertProtection()` | Exclude from INSERT (e.g. for a PK with DEFAULT) | +| `WithUpdateProtection()` | Exclude from UPDATE SET (e.g. for `created_at`) | + +## Common patterns + +### PK with a database-side DEFAULT + +```go +c.Field(&m.ID).AsColumn().WithInsertProtection().WithUpdateProtection() +``` + +`ID` appears only in WHERE and SELECT; it is never in INSERT, so the database generates it. + +### created_at / updated_at + +```go +c.Field(&m.CreatedAt).AsColumn().WithUpdateProtection() // inserted, never updated +c.Field(&m.UpdatedAt).AsColumn().WithInsertProtection() // set by a trigger on UPDATE +``` + +### Column from a JOIN + +```go +c.Field(&m.PostTitle).AsColumn().WithTable("posts") +``` + +SELECT will read `posts.post_title`. The JOIN itself must be configured via [Persistent queries](persistent-queries.md). + +### Nullable columns + +Use a pointer type β€” `*string`, `*time.Time`, `*bool`. The `EQ(nil)` / `NEQ(nil)` operators work and gerpo generates `IS NULL` / `IS NOT NULL`. + +```go +type User struct { + Email *string // nullable email +} +c.Field(&m.Email).AsColumn() + +// Query: +repo.Count(ctx, func(m *User, h query.CountHelper[User]) { + h.Where().Field(&m.Email).EQ(nil) // WHERE email IS NULL +}) +``` + +## Columns storage and ExecutionColumns + +`repo.GetColumns()` returns `types.ColumnsStorage` β€” the collection of every configured `Column`. On each request, gerpo creates an `ExecutionColumns` β€” a slice filtered by the specific action (SELECT / INSERT / UPDATE / …) taking protections and `Exclude/Only` helpers into account. + +Direct access to `ColumnsStorage` is useful if you need to build a custom query and bypass the repo β€” rarely needed in practice. diff --git a/docs/features/crud.md b/docs/features/crud.md new file mode 100644 index 0000000..6ba1eb6 --- /dev/null +++ b/docs/features/crud.md @@ -0,0 +1,105 @@ +# CRUD operations + +The `Repository[T]` interface provides six methods: `GetFirst`, `GetList`, `Count`, `Insert`, `Update`, `Delete`. Each one accepts a `context.Context` and a variadic list of query functions that configure a single call. + +## GetFirst + +Return the first matching record. + +```go +u, err := repo.GetFirst(ctx, func(m *User, h query.GetFirstHelper[User]) { + h.Where().Field(&m.Email).EQ("alice@example.com") + h.OrderBy().Field(&m.CreatedAt).DESC() +}) +``` + +If no row is found, the error is `gerpo.ErrNotFound`: + +```go +if errors.Is(err, gerpo.ErrNotFound) { /* … */ } +``` + +## GetList + +Return a slice of records. An empty result is an empty slice and no error. + +```go +users, err := repo.GetList(ctx, func(m *User, h query.GetListHelper[User]) { + h.Where().Field(&m.Age).GTE(18) + h.OrderBy().Field(&m.Name).ASC() + h.Page(2).Size(50) +}) +``` + +See [Ordering & pagination](order-pagination.md) for details on `Page`/`Size`. + +## Count + +Returns a `uint64`. + +```go +n, err := repo.Count(ctx, func(m *User, h query.CountHelper[User]) { + h.Where().Field(&m.Age).GTE(18) +}) +``` + +## Insert + +Inserts a single record. Mutates the model through `WithBeforeInsert` / `WithAfterInsert` hooks (see [Hooks](hooks.md)). + +```go +u := &User{ID: uuid.New(), Name: "Bob", Age: 25, CreatedAt: time.Now()} +err := repo.Insert(ctx, u) +``` + +`Exclude`/`Only` narrows the column set (handy when the database should apply a `DEFAULT`): + +```go +repo.Insert(ctx, u, func(m *User, h query.InsertHelper[User]) { + h.Exclude(&m.CreatedAt) // let the database default NOW() +}) +``` + +## Update + +Updates records by WHERE. Returns the number of affected rows. When zero rows match, returns `gerpo.ErrNotFound`. + +```go +u.Name = "Bob The Builder" +affected, err := repo.Update(ctx, u, func(m *User, h query.UpdateHelper[User]) { + h.Where().Field(&m.ID).EQ(u.ID) +}) +``` + +`Only`/`Exclude` let you update a subset of fields ([Exclude & Only](exclude-only.md)): + +```go +repo.Update(ctx, u, func(m *User, h query.UpdateHelper[User]) { + h.Where().Field(&m.ID).EQ(u.ID) + h.Only(&m.Name) // SET name = ?, and nothing else +}) +``` + +## Delete + +Deletes records by WHERE. If the repo was configured with `WithSoftDeletion`, this is rewritten as an UPDATE instead ([Soft delete](soft-delete.md)). When zero rows match, returns `gerpo.ErrNotFound`. + +```go +n, err := repo.Delete(ctx, func(m *User, h query.DeleteHelper[User]) { + h.Where().Field(&m.ID).EQ(u.ID) +}) +``` + +!!! warning "Delete without WHERE wipes the table" + The repo does not block an unconditional `Delete` unless a persistent query puts a WHERE in front. Always pass a WHERE explicitly. + +## Error semantics + +| Method | ErrNotFound when | +|---|---| +| `GetFirst` | no rows returned | +| `Update` | `RowsAffected == 0` | +| `Delete` | `RowsAffected == 0` (including the UPDATE from soft delete) | +| `GetList`, `Count`, `Insert` | **never** | + +Any other error (FK, unique, syntax, network) is returned as-is and passed through [`WithErrorTransformer`](error-transformer.md) if configured. diff --git a/docs/features/error-transformer.md b/docs/features/error-transformer.md new file mode 100644 index 0000000..3f735ee --- /dev/null +++ b/docs/features/error-transformer.md @@ -0,0 +1,53 @@ +# Error transformer + +`WithErrorTransformer(fn func(error) error)` pipes every error returned by the repository through your function. The typical use case is to stop leaking `gerpo.ErrNotFound` outwards and map it to a domain error instead. + +## Replacing ErrNotFound + +```go +var ErrUserNotFound = errors.New("user not found") + +repo, _ := gerpo.NewBuilder[User](). + DB(adapter). + Table("users"). + Columns(/* … */). + WithErrorTransformer(func(err error) error { + if errors.Is(err, gerpo.ErrNotFound) { + return ErrUserNotFound + } + return err + }). + Build() +``` + +Now the hexagonal layer knows nothing about gerpo. + +## What flows through the transformer + +- `GetFirst` β€” no rows β‡’ `gerpo.ErrNotFound`. +- `Update`, `Delete` β€” `rowsAffected == 0` β‡’ `gerpo.ErrNotFound`. +- Any DB error (FK, unique, syntax, network). +- `gerpo.ErrApplyQuery`, `gerpo.ErrApplyPersistentQuery` β€” when WHERE/ORDER/etc. could not be assembled. + +## What does **not** + +- The happy path β€” the transformer isn't invoked when `err == nil`. +- Logic errors raised before any DB call (e.g. an empty `Build()` state) β€” they come out of `NewBuilder.Build()`, which is outside the transformer. + +## Passing back the wrapped error + +```go +.WithErrorTransformer(func(err error) error { + switch { + case errors.Is(err, gerpo.ErrNotFound): + return ErrUserNotFound + default: + // wrap with domain context, keep the original for logs + return fmt.Errorf("user repo: %w", err) + } +}) +``` + +## Behavior with tests + +`errors.Is` is transitive: `errors.Is(err, ErrUserNotFound)` β†’ true. But `errors.Is(err, gerpo.ErrNotFound)` β†’ **false**, because the wrapping chain starts from the replacement. That's by design: the transformer is the boundary between infrastructure and domain. diff --git a/docs/features/exclude-only.md b/docs/features/exclude-only.md new file mode 100644 index 0000000..5834bb5 --- /dev/null +++ b/docs/features/exclude-only.md @@ -0,0 +1,52 @@ +# Exclude & Only + +`Exclude` and `Only` narrow the set of columns that participate in a specific operation. Fields are referenced by pointer, same as when configuring columns. + +## GetFirst / GetList + +Affect **SELECT**: excluded fields are never fetched and stay at their zero value. + +```go +// SELECT id, name FROM users WHERE … +u, _ := repo.GetFirst(ctx, func(m *User, h query.GetFirstHelper[User]) { + h.Where().Field(&m.ID).EQ(id) + h.Only(&m.ID, &m.Name) +}) + +// Opposite: fetch everything except the password +u, _ := repo.GetFirst(ctx, func(m *User, h query.GetFirstHelper[User]) { + h.Where().Field(&m.ID).EQ(id) + h.Exclude(&m.PasswordHash) +}) +``` + +!!! tip + Excluded columns keep working in WHERE β€” this is metadata of the operation, not of the schema. + +## Insert + +Affects **INSERT**: excluded fields don't appear in the statement β€” the database uses its `DEFAULT`. + +```go +repo.Insert(ctx, u, func(m *User, h query.InsertHelper[User]) { + h.Exclude(&m.ID, &m.CreatedAt) // ID and CreatedAt are filled by the DB +}) +``` + +## Update + +Affects **SET**: only the selected subset is touched. + +```go +// UPDATE users SET name = ? WHERE id = ? +repo.Update(ctx, u, func(m *User, h query.UpdateHelper[User]) { + h.Where().Field(&m.ID).EQ(u.ID) + h.Only(&m.Name) +}) +``` + +`Exclude` works symmetrically β€” all fields except the listed ones are updated. + +## Interaction with `With*Protection` + +Protected columns are **always** excluded from the matching operation, regardless of helpers. For example, `WithUpdateProtection()` on `ID` means `ID` never enters the SET β€” even if you explicitly pass it into `Only`. That's a schema property; `Only`/`Exclude` is a query property. diff --git a/docs/features/hooks.md b/docs/features/hooks.md new file mode 100644 index 0000000..deb6b23 --- /dev/null +++ b/docs/features/hooks.md @@ -0,0 +1,58 @@ +# Hooks + +Hooks are functions that gerpo runs around repository operations. Useful for generated fields, auditing, projections. + +## Hook kinds + +| Option | Signature | Called | +|---|---|---| +| `WithBeforeInsert` | `func(ctx, *T)` | before SQL `INSERT` | +| `WithAfterInsert` | `func(ctx, *T)` | after a successful `INSERT` | +| `WithBeforeUpdate` | `func(ctx, *T)` | before SQL `UPDATE` | +| `WithAfterUpdate` | `func(ctx, *T)` | after a successful `UPDATE` (rowsAffected > 0) | +| `WithAfterSelect` | `func(ctx, []*T)` | after Scan of `GetFirst`/`GetList` | + +For `GetFirst`, `afterSelect` receives a single-element slice. For `GetList` β€” the full slice. + +## Mutating the model + +Changes made in a `Before*` hook land in the SQL. This is the standard way to fill generated fields: + +```go +.WithBeforeInsert(func(ctx context.Context, u *User) { + if u.ID == uuid.Nil { + u.ID = uuid.New() + } + if u.CreatedAt.IsZero() { + u.CreatedAt = time.Now().UTC() + } +}). +WithBeforeUpdate(func(ctx context.Context, u *User) { + now := time.Now().UTC() + u.UpdatedAt = &now +}) +``` + +`After*` hooks can also mutate the model, but their changes never reach the database β€” they only affect the caller's copy. + +## Stacking + +`WithBeforeInsert` can be called multiple times β€” hooks run in registration order: + +```go +.WithBeforeInsert(setDefaults). +WithBeforeInsert(validatePayload). +WithBeforeInsert(audit.LogAttempt) +``` + +## Typical uses + +- **Field auto-fill:** IDs, timestamps, tenant_id. +- **Auditing:** log INSERT/UPDATE/DELETE together with the user and payload. +- **Denormalization:** recompute a derived column after UPDATE. +- **Post-processing in `AfterSelect`:** e.g. decrypting encrypted fields. + +## What you can't do in a hook + +- Open transactions on the same connection β€” you're already inside a gerpo call. Run external side effects outside `.Insert`/`.Update`. +- Return an error β€” the signature is `func(ctx, *T)` without a return. To abort an operation, validate upstream (before the repo call) or map the DB error via [Error transformer](error-transformer.md). diff --git a/docs/features/index.md b/docs/features/index.md new file mode 100644 index 0000000..b4bf36e --- /dev/null +++ b/docs/features/index.md @@ -0,0 +1,32 @@ +# Features + +Reference of gerpo capabilities grouped by area. + +## Configuration + +| Page | What's inside | +|---|---| +| [Repository builder](repository.md) | `NewBuilder[T]()`, `DB`, `Table`, `Build`, repository lifecycle | +| [Columns](columns.md) | `AsColumn`, `AsVirtual`, insert/update protection, aliases, columns from other tables | +| [Persistent queries](persistent-queries.md) | `WithQuery`: conditions, JOINs, GROUP BY applied to every request | +| [Soft delete](soft-delete.md) | Turning DELETE into UPDATE | +| [Virtual columns](virtual-columns.md) | Computed fields at the SELECT level | +| [Hooks](hooks.md) | Before/After for Insert/Update/Select | +| [Error transformer](error-transformer.md) | Mapping gerpo errors to domain errors | + +## Operations + +| Page | What's inside | +|---|---| +| [CRUD operations](crud.md) | `GetFirst`, `GetList`, `Count`, `Insert`, `Update`, `Delete` | +| [WHERE operators](where.md) | EQ, NEQ, LT/LTE/GT/GTE, IN/NIN, CT/BW/EW + IC, AND/OR/Group | +| [Ordering & pagination](order-pagination.md) | `OrderBy`, `Page`, `Size` | +| [Exclude & Only](exclude-only.md) | Narrowing columns in SELECT/INSERT/UPDATE | +| [Transactions](transactions.md) | `BeginTx`, `repo.Tx`, `Commit`, `Rollback`, `RollbackUnlessCommitted` | + +## Infrastructure + +| Page | What's inside | +|---|---| +| [Cache](cache.md) | `CtxCache` β€” cache scoped to a request context | +| [Adapters](adapters.md) | pgx v5, pgx v4, database/sql, and custom adapters | diff --git a/docs/features/order-pagination.md b/docs/features/order-pagination.md new file mode 100644 index 0000000..568f2dc --- /dev/null +++ b/docs/features/order-pagination.md @@ -0,0 +1,55 @@ +# Ordering & pagination + +## Sorting + +```go +h.OrderBy().Field(&m.CreatedAt).DESC() +h.OrderBy().Field(&m.Name).ASC() +``` + +Multiple calls stack into a single `ORDER BY`, comma-separated: + +```go +// ORDER BY priority DESC, created_at ASC +h.OrderBy().Field(&m.Priority).DESC() +h.OrderBy().Field(&m.CreatedAt).ASC() +``` + +Available in `GetFirst` and `GetList`. `Count`/`Update`/`Delete` don't benefit from sorting, so it's absent in their helpers. + +## Pagination (GetList only) + +```go +h.Page(1).Size(20) +``` + +- `Page(n)` β€” page number, **1-indexed**. +- `Size(n)` β€” page size (LIMIT). +- OFFSET is computed as `(page - 1) * size`. + +!!! warning "Page without Size" + Calling `Page(...)` without `Size(...)` returns an error on Apply: *"incorrect pagination: size is required then page is set"*. + +### Limit only, no offset + +Just `Size`: + +```go +h.Size(10) // LIMIT 10 OFFSET 0 +``` + +### Page past the end + +If OFFSET goes beyond the data, `GetList` returns an empty slice β€” no error. + +## Combining with filters + +Call order doesn't matter: WHERE, ORDER BY, and LIMIT/OFFSET are assembled independently. + +```go +users, _ := repo.GetList(ctx, func(m *User, h query.GetListHelper[User]) { + h.Where().Field(&m.Age).GTE(18) + h.OrderBy().Field(&m.CreatedAt).DESC() + h.Page(3).Size(25) +}) +``` diff --git a/docs/features/persistent-queries.md b/docs/features/persistent-queries.md new file mode 100644 index 0000000..3a843b0 --- /dev/null +++ b/docs/features/persistent-queries.md @@ -0,0 +1,69 @@ +# Persistent queries + +`WithQuery(func(m *T, h query.PersistentHelper[T]))` defines conditions that apply to **every** request the repository runs β€” SELECT, COUNT, UPDATE, DELETE. Typical uses are soft delete, JOINs for virtual columns, GROUP BY. + +## Four capabilities of PersistentHelper + +| Method | Effect | +|---|---| +| `Where()` | Filters inserted into every query | +| `LeftJoin(fn)` / `InnerJoin(fn)` | JOINs β€” body is returned by a `func(context.Context) string` | +| `GroupBy(fields...)` | A single GROUP BY applied everywhere (required when a JOIN + aggregate shows up) | +| `Exclude(fields...)` | Hide a column from every SELECT | + +## Hiding soft-deleted records + +```go +.WithQuery(func(m *User, h query.PersistentHelper[User]) { + h.Where().Field(&m.DeletedAt).EQ(nil) +}) +``` + +Now every `GetFirst`/`GetList`/`Count` automatically ignores records whose `DeletedAt` is non-null. + +## JOIN + virtual column + +A real example from the integration tests β€” `User` has a virtual `PostCount` field computed through a LEFT JOIN on `posts`: + +```go +.Columns(func(m *User, c *gerpo.ColumnBuilder[User]) { + c.Field(&m.ID).AsColumn() + c.Field(&m.Name).AsColumn() + c.Field(&m.PostCount).AsVirtual().WithSQL(func(ctx context.Context) string { + return "COALESCE(COUNT(posts.id), 0)" + }) +}). +WithQuery(func(m *User, h query.PersistentHelper[User]) { + h.LeftJoin(func(ctx context.Context) string { + return "posts ON posts.user_id = users.id" + }) + h.GroupBy(&m.ID, &m.Name) + h.Where().Field(&m.DeletedAt).EQ(nil) +}) +``` + +Now `PostCount` is automatically included in the SELECT of every request against `users`. + +!!! note "InnerJoin vs LeftJoin" + `InnerJoin` drops users who have no posts β€” handy when you only care about active ones. `LeftJoin` keeps them, the aggregate returns `0` for loners. + +## Context-aware JOIN + +The function returns the JOIN text and takes a `context.Context`. That lets you mix in runtime values (tenant ID, UI locale) into the body: + +```go +h.LeftJoin(func(ctx context.Context) string { + tenantID := ctxpkg.TenantID(ctx) + return fmt.Sprintf( + "posts ON posts.user_id = users.id AND posts.tenant_id = '%s'", + tenantID, + ) +}) +``` + +!!! danger "SQL injection" + Values coming from the context into the JOIN body do not flow through parameter binding. If you interpolate user-supplied data, escape it yourself β€” or, better, switch to a WHERE with a bound parameter. + +## Combining with per-request WHERE + +Persistent conditions are joined with per-request conditions via AND and **always** come first. Your filter cannot disable a persistent WHERE, but it can add extra conditions. diff --git a/docs/features/repository.md b/docs/features/repository.md new file mode 100644 index 0000000..6f7b0ef --- /dev/null +++ b/docs/features/repository.md @@ -0,0 +1,74 @@ +# Repository builder + +A repository is assembled through the fluent `gerpo.NewBuilder[T]()`. The chain `DB β†’ Table β†’ Columns β†’ Build` is mandatory. Everything else is optional `With*` steps between `Columns` and `Build`. + +## Minimal repository + +```go +repo, err := gerpo.NewBuilder[User](). + DB(adapter). + Table("users"). + Columns(func(m *User, c *gerpo.ColumnBuilder[User]) { + c.Field(&m.ID).AsColumn().WithUpdateProtection() + c.Field(&m.Name).AsColumn() + }). + Build() +``` + +`Build()` returns `gerpo.Repository[User]` β€” a thread-safe object, a single instance serves the whole application. + +## Full chain of options + +```go +repo, err := gerpo.NewBuilder[User](). + DB(adapter, executor.WithCacheStorage(ctxCache)). // (1) adapter + executor options + Table("users"). // (2) table name + Columns(colsFn). // (3) column description + WithQuery(persistentFn). // (4) persistent conditions + WithSoftDeletion(softFn). // (5) DELETE replacement + WithBeforeInsert(beforeFn). // (6) hooks + WithAfterInsert(afterFn). + WithBeforeUpdate(beforeUpdFn). + WithAfterUpdate(afterUpdFn). + WithAfterSelect(afterSelectFn). + WithErrorTransformer(mapErr). // (7) error mapping + Build() +``` + +1. **DB** β€” attach a driver adapter. List of bundled adapters: [Adapters](adapters.md). The second argument accepts `executor.Option` (e.g., `WithCacheStorage`). +2. **Table** β€” physical table name. +3. **Columns** β€” mandatory column description. [Columns](columns.md). +4. **WithQuery** β€” persistent filters/joins/groupings applied to every request. [Persistent queries](persistent-queries.md). +5. **WithSoftDeletion** β€” rewrite DELETE as UPDATE. [Soft delete](soft-delete.md). +6. **With{Before,After}{Insert,Update}**, **WithAfterSelect** β€” hooks. Multiple calls stack. [Hooks](hooks.md). +7. **WithErrorTransformer** β€” run every error through a mapping function. [Error transformer](error-transformer.md). + +## Methods of the finished repository + +```go +type Repository[TModel any] interface { + GetFirst(ctx context.Context, qFns ...func(m *TModel, h query.GetFirstHelper[TModel])) (*TModel, error) + GetList(ctx context.Context, qFns ...func(m *TModel, h query.GetListHelper[TModel])) ([]*TModel, error) + Count(ctx context.Context, qFns ...func(m *TModel, h query.CountHelper[TModel])) (uint64, error) + Insert(ctx context.Context, model *TModel, qFns ...func(m *TModel, h query.InsertHelper[TModel])) error + Update(ctx context.Context, model *TModel, qFns ...func(m *TModel, h query.UpdateHelper[TModel])) (int64, error) + Delete(ctx context.Context, qFns ...func(m *TModel, h query.DeleteHelper[TModel])) (int64, error) + Tx(tx executor.Tx) Repository[TModel] + GetColumns() types.ColumnsStorage +} +``` + +Every method is described in [CRUD operations](crud.md). + +## Thread safety + +A repository is safe to share across goroutines. Internally, statement objects are reused through `sync.Pool`, but each call takes its own instance from the pool, so there are no races. + +## Build errors + +`Build()` returns an error if: + +- `DB` or `Table` is not set; +- `Columns` contains no columns; +- a virtual column lacks `WithSQL`; +- `WithSoftDeletion` references a field that is not declared as a column. diff --git a/docs/features/soft-delete.md b/docs/features/soft-delete.md new file mode 100644 index 0000000..2b0658b --- /dev/null +++ b/docs/features/soft-delete.md @@ -0,0 +1,61 @@ +# Soft delete + +`WithSoftDeletion(fn)` turns a physical DELETE into an UPDATE of selected fields. The "mark, don't drop" pattern β€” useful to preserve data and keep foreign keys intact. + +## Setup + +```go +.Columns(func(m *User, c *gerpo.ColumnBuilder[User]) { + c.Field(&m.ID).AsColumn() + c.Field(&m.DeletedAt).AsColumn().WithInsertProtection() +}). +WithSoftDeletion(func(m *User, b *gerpo.SoftDeletionBuilder[User]) { + b.Field(&m.DeletedAt).SetValueFn(func(ctx context.Context) any { + t := time.Now().UTC() + return &t + }) +}). +WithQuery(func(m *User, h query.PersistentHelper[User]) { + h.Where().Field(&m.DeletedAt).EQ(nil) +}) +``` + +Three required pieces: + +1. **Marker column** (`DeletedAt`). Typically nullable β€” `*time.Time`. Add `WithInsertProtection` so it can't be accidentally written at INSERT time. +2. **`WithSoftDeletion`** β€” describes the value to write on "delete". The function runs on every `Delete` call and receives the context (useful for user/clock/tenant). +3. **`WithQuery` with a filter** β€” so soft-deleted records don't leak into SELECTs. Without it they show up in listings. + +!!! warning "SetValueFn return type" + The returned value must match the field type. For `*time.Time` return `*time.Time`, not `time.Time`, or fmap will panic when assigning. + +## How it works + +`repo.Delete(ctx, …)` executes + +```sql +UPDATE users SET deleted_at = ? WHERE … +``` + +instead of `DELETE FROM users WHERE …`. It returns the UPDATE's `RowsAffected`. If zero rows match, it returns `ErrNotFound`. + +## Restoration + +There's no dedicated API β€” restore a row with a direct UPDATE bypassing the repo: + +```sql +UPDATE users SET deleted_at = NULL WHERE id = ?; +``` + +Alternatively, you can run an extra `repo.Update` with `Only(&m.DeletedAt)` and a `nil` value in the model β€” but that bypasses the persistent WHERE, so it only works when the repo's structure allows it. + +## Multiple marker fields + +`SoftDeletionBuilder` supports multiple `Field` calls β€” all of them will be updated on soft-delete: + +```go +WithSoftDeletion(func(m *User, b *gerpo.SoftDeletionBuilder[User]) { + b.Field(&m.DeletedAt).SetValueFn(now) + b.Field(&m.DeletedBy).SetValueFn(userFromCtx) +}) +``` diff --git a/docs/features/transactions.md b/docs/features/transactions.md new file mode 100644 index 0000000..cec0bcb --- /dev/null +++ b/docs/features/transactions.md @@ -0,0 +1,60 @@ +# Transactions + +gerpo does not invent its own transaction layer β€” it works with the `Tx` returned by the adapter. + +## Basic flow + +```go +tx, err := adapter.BeginTx(ctx) +if err != nil { + return err +} +defer tx.RollbackUnlessCommitted() // safety net: rolls back if we forgot to Commit + +txRepo := repo.Tx(tx) + +if err := txRepo.Insert(ctx, u); err != nil { + return err // defer will roll back +} +if _, err := txRepo.Update(ctx, u, whereByID); err != nil { + return err +} + +return tx.Commit() +``` + +## Tx methods + +| Method | Effect | +|---|---| +| `Commit() error` | Commits; all subsequent `Rollback*` calls become no-ops | +| `Rollback() error` | Explicit rollback | +| `RollbackUnlessCommitted() error` | Safe `defer`: rolls back only if Commit wasn't called | +| `ExecContext`/`QueryContext` | Raw SQL β€” useful when you need to bypass the repo | + +## `repo.Tx(tx) Repository[T]` + +This method does not open a new transaction β€” it **wraps** an existing one. Returns a repository bound to this `Tx`. Returns a single value, no error. + +```go +orderRepo := orderRepo.Tx(tx) +itemRepo := itemRepo.Tx(tx) +// both write into the same transaction +``` + +## Isolation + +Isolation is controlled by the driver; gerpo does not set a level. PostgreSQL defaults to Read Committed. For SERIALIZABLE/REPEATABLE READ, open the transaction directly via the adapter's `ExecContext` (`BEGIN ISOLATION LEVEL …`), or pass options via the driver's `BeginTx` (pgx accepts `pgx.TxOptions`). + +## Common pitfall: multiple calls without a transaction + +```go +repo.Insert(ctx, order) // on one pool connection +repo.Insert(ctx, items...) // may land on a different connection; not atomic +``` + +If atomicity matters β€” wrap in one `tx`. + +## Partial rollback: savepoints + +gerpo does not expose a `SAVEPOINT` API. If you need nested rollbacks, issue them via `tx.ExecContext(ctx, "SAVEPOINT sp")` / `RELEASE SAVEPOINT` / `ROLLBACK TO SAVEPOINT`. diff --git a/docs/features/virtual-columns.md b/docs/features/virtual-columns.md new file mode 100644 index 0000000..b7182c2 --- /dev/null +++ b/docs/features/virtual-columns.md @@ -0,0 +1,70 @@ +# Virtual columns + +Virtual columns are SELECT expressions mapped onto struct fields. They are **read-only**: automatically protected from both INSERT and UPDATE. + +## Simple computed field + +```go +type User struct { + FirstName string + LastName string + FullName string // virtual +} + +c.Field(&m.FullName).AsVirtual().WithSQL(func(ctx context.Context) string { + return "first_name || ' ' || last_name" +}) +``` + +SELECT will get `first_name || ' ' || last_name AS full_name` (alias is the snake_case of the field), and the Scan drops the value into `m.FullName`. + +## Aggregations from a JOIN + +Virtual columns often aggregate related tables. You need a pair: a JOIN in the persistent query, and a matching GROUP BY. + +```go +c.Field(&m.PostCount).AsVirtual().WithSQL(func(ctx context.Context) string { + return "COALESCE(COUNT(posts.id), 0)" +}) + +.WithQuery(func(m *User, h query.PersistentHelper[User]) { + h.LeftJoin(func(ctx context.Context) string { + return "posts ON posts.user_id = users.id" + }) + h.GroupBy(&m.ID, &m.Name /*, every non-aggregate */) +}) +``` + +!!! warning "GROUP BY" + When aggregates are in play, you must list every regular column that appears in SELECT in the `GroupBy`, otherwise PostgreSQL returns *"must appear in the GROUP BY clause or be used in an aggregate function"*. + +## Context-aware SQL + +`WithSQL(fn)` takes a `context.Context`, so the SQL is produced per request. You can mix in data from the context (tenant, UI locale), but **values are not parameterized** β€” be careful with user input. + +## Read-only in practice + +Trying to assign a value to a virtual field during `Insert`/`Update` isn't an error β€” gerpo simply ignores it: + +```go +u := &User{FullName: "fake"} // never stored +repo.Insert(ctx, u) +``` + +The next `GetFirst` returns whatever the database computed. + +## Boolean virtuals with custom filter (deprecated API) + +For `bool`-typed virtual columns that need different SQL for `true/false/nil` in WHERE, there is `WithBoolEqFilter`: + +```go +c.Field(&m.IsActive).AsVirtual(). + WithSQL(func(ctx context.Context) string { return "EXISTS (SELECT 1 FROM sessions WHERE user_id = users.id)" }). + WithBoolEqFilter(func(b *virtual.BoolEQFilterBuilder) { + b.AddTrueSQLFn(func(ctx context.Context) string { return "EXISTS (...)" }) + b.AddFalseSQLFn(func(ctx context.Context) string { return "NOT EXISTS (...)" }) + }) +``` + +!!! note "Deprecated" + The current virtual-filter API is marked as deprecated in the 1.0.0 roadmap β€” a new configuration API for virtual columns is planned. It still works today; just be ready to migrate. diff --git a/docs/features/where.md b/docs/features/where.md new file mode 100644 index 0000000..73c9e53 --- /dev/null +++ b/docs/features/where.md @@ -0,0 +1,81 @@ +# WHERE operators + +All operators are built with `h.Where().Field(&m.X).(val)`. The result is an `ANDOR` that lets you keep chaining with `.AND()` / `.OR()`. + +## Comparison + +| Method | SQL | Works for | +|---|---|---| +| `EQ(v)` | `= ?` (or `IS NULL` when `v == nil`) | any type | +| `NEQ(v)` | `!= ?` (or `IS NOT NULL`) | any | +| `LT(v)` | `< ?` | numbers, dates | +| `LTE(v)` | `<= ?` | numbers, dates | +| `GT(v)` | `> ?` | numbers, dates | +| `GTE(v)` | `>= ?` | numbers, dates | + +```go +h.Where().Field(&m.Age).GTE(18) +h.Where().Field(&m.DeletedAt).EQ(nil) // IS NULL +``` + +## Sets + +| Method | SQL | +|---|---| +| `IN(a, b, c)` | `IN (?, ?, ?)` | +| `NIN(a, b, c)` | `NOT IN (?, ?, ?)` | + +Accept variadic `any` or an already-expanded slice: + +```go +h.Where().Field(&m.ID).IN(id1, id2, id3) +h.Where().Field(&m.ID).IN(ids...) // if ids is []uuid.UUID +``` + +## String patterns + +| Method | SQL | Case-insensitive variant | +|---|---|---| +| `CT(v)` | `LIKE CONCAT('%', CAST(? AS text), '%')` | `CT(v, true)` β†’ `LOWER(col) LIKE LOWER(…)` | +| `NCT(v)` | `NOT LIKE CONCAT('%', …, '%')` | `NCT(v, true)` | +| `BW(v)` | `LIKE CONCAT(CAST(? AS text), '%')` | `BW(v, true)` | +| `NBW(v)` | `NOT LIKE CONCAT(…, '%')` | `NBW(v, true)` | +| `EW(v)` | `LIKE CONCAT('%', CAST(? AS text))` | `EW(v, true)` | +| `NEW(v)` | `NOT LIKE CONCAT('%', …)` | `NEW(v, true)` | + +```go +h.Where().Field(&m.Title).CT("go", true) // case-insensitive contains +h.Where().Field(&m.Email).BW("admin@") // starts with +h.Where().Field(&m.Path).EW(".log", true) // ends with, case-insensitive +``` + +!!! note "CAST(? AS text)" + A bare `CONCAT(…)` breaks PostgreSQL type inference, so gerpo casts the parameter to `text` explicitly. The same form is valid in MySQL. + +## Logic: AND, OR, Group + +- Consecutive `Field(...).Op(...)` calls are joined by an **implicit AND**. +- `.OR()` / `.AND()` in the chain add explicit joiners. +- `Group(func(t WhereTarget))` produces parentheses. + +```go +// (age >= 18 AND email IS NOT NULL) OR role = 'admin' +h.Where().Group(func(t types.WhereTarget) { + t.Field(&m.Age).GTE(18). + AND().Field(&m.Email).NEQ(nil) +}).OR().Field(&m.Role).EQ("admin") +``` + +## Custom operation β€” `OP` + +If a column exposes a custom filter (relevant for [virtual columns](virtual-columns.md) with `WithBoolEqFilter`), invoke it by name: + +```go +h.Where().Field(&m.IsActive).OP(types.OperationEQ, true) +``` + +## Limitations + +- `LT`/`GT`/`LTE`/`GTE` do not type-check at runtime β€” the database does. gerpo passes values through as-is. +- For string LIKE operators the value must be a string; for `EQ`/`NEQ` the value type must match the field type (nullable types accept `nil`). +- A type mismatch produces a descriptive error wrapped with `gerpo.ErrApplyQuery`. diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..28431cb --- /dev/null +++ b/docs/index.md @@ -0,0 +1,106 @@ +# gerpo + +**gerpo** (Golang + Repository) is a generic repository pattern for Go with pluggable adapters and minimal reflection. It is **not an ORM**: no migrations, no relations between entities, no struct tags. The entire SQL behavior is described declaratively in the repository configuration. + +!!! tip "One-line ideology" + *Schema and SQL live inside the repository configuration; columns are bound to struct fields via pointers β€” not strings, not tags.* + +## Install + +```bash +go get github.com/insei/gerpo@latest +``` + +Minimum Go version is **1.21**. + +## Quick start + +```go +package main + +import ( + "context" + "time" + + "github.com/google/uuid" + "github.com/insei/gerpo" + "github.com/insei/gerpo/executor/adapters/pgx5" + "github.com/insei/gerpo/query" + "github.com/jackc/pgx/v5/pgxpool" +) + +type User struct { + ID uuid.UUID + Name string + Email *string // nullable + Age int + CreatedAt time.Time +} + +func main() { + pool, _ := pgxpool.New(context.Background(), "postgres://...") + + repo, err := gerpo.NewBuilder[User](). + DB(pgx5.NewPoolAdapter(pool)). + Table("users"). + Columns(func(m *User, c *gerpo.ColumnBuilder[User]) { + c.Field(&m.ID).AsColumn().WithUpdateProtection() + c.Field(&m.Name).AsColumn() + c.Field(&m.Email).AsColumn() + c.Field(&m.Age).AsColumn() + c.Field(&m.CreatedAt).AsColumn().WithUpdateProtection() + }). + Build() + if err != nil { + panic(err) + } + + ctx := context.Background() + + // Insert + u := &User{ID: uuid.New(), Name: "Alice", Age: 30, CreatedAt: time.Now()} + _ = repo.Insert(ctx, u) + + // Query + list, _ := repo.GetList(ctx, func(m *User, h query.GetListHelper[User]) { + h.Where().Field(&m.Age).GTE(18) + h.OrderBy().Field(&m.CreatedAt).DESC() + h.Page(1).Size(20) + }) + _ = list +} +``` + +Full runnable samples live in [`examples/`](https://github.com/Insei/gerpo/tree/main/examples) and in the [integration tests](https://github.com/Insei/gerpo/tree/main/tests/integration). + +## What to read next + +
+ +- :material-book-open-variant:{ .lg } **[Features β†’](features/index.md)** + + A walkthrough of every repository capability: CRUD, filters, ordering, hooks, soft delete, virtual columns, cache, transactions, adapters. + +- :material-sitemap:{ .lg } **[Architecture β†’](architecture/index.md)** + + How gerpo is built internally: layers, SQL generation, pointer-based field mapping via fmap, adapter implementations, CtxCache. For contributors. + +- :simple-go:{ .lg } **[API reference β†’](https://pkg.go.dev/github.com/insei/gerpo)** + + Autogenerated on pkg.go.dev. + +- :material-github:{ .lg } **[Repository β†’](https://github.com/Insei/gerpo)** + + Source, issue tracker, pull requests. + +
+ +## Supported drivers + +| Adapter | Package | Placeholder format | +|---|---|---| +| pgx v5 | `executor/adapters/pgx5` | `$1, $2, …` | +| pgx v4 | `executor/adapters/pgx4` | `$1, $2, …` | +| database/sql | `executor/adapters/databasesql` | `?` or `$1` (configurable) | + +You can wrap any driver of your own β€” just implement the `executor.DBAdapter` interface (three methods: `ExecContext`, `QueryContext`, `BeginTx`). See [Adapters](features/adapters.md). diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..fa09ac0 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,96 @@ +site_name: gerpo +site_description: Generic repository pattern for Go with pluggable adapters +site_url: https://insei.github.io/gerpo/ +repo_url: https://github.com/Insei/gerpo +repo_name: Insei/gerpo +edit_uri: edit/main/docs/ + +theme: + name: material + features: + - navigation.instant + - navigation.tracking + - navigation.tabs + - navigation.sections + - navigation.indexes + - navigation.top + - toc.follow + - search.suggest + - search.highlight + - content.code.copy + - content.code.annotate + - content.tabs.link + - content.action.edit + palette: + - media: "(prefers-color-scheme: light)" + scheme: default + primary: indigo + accent: indigo + toggle: + icon: material/weather-sunny + name: Switch to dark mode + - media: "(prefers-color-scheme: dark)" + scheme: slate + primary: indigo + accent: indigo + toggle: + icon: material/weather-night + name: Switch to light mode + icon: + repo: fontawesome/brands/github + +markdown_extensions: + - admonition + - attr_list + - md_in_html + - tables + - toc: + permalink: true + - pymdownx.details + - pymdownx.highlight: + anchor_linenums: true + line_spans: __span + pygments_lang_class: true + - pymdownx.inlinehilite + - pymdownx.superfences + - pymdownx.tabbed: + alternate_style: true + - pymdownx.snippets + - pymdownx.emoji: + emoji_index: !!python/name:material.extensions.emoji.twemoji + emoji_generator: !!python/name:material.extensions.emoji.to_svg + +extra: + social: + - icon: fontawesome/brands/github + link: https://github.com/Insei/gerpo + - icon: fontawesome/brands/go + link: https://pkg.go.dev/github.com/insei/gerpo + +nav: + - Get started: index.md + - Features: + - features/index.md + - Repository builder: features/repository.md + - Columns: features/columns.md + - CRUD operations: features/crud.md + - WHERE operators: features/where.md + - Ordering & pagination: features/order-pagination.md + - Exclude & Only: features/exclude-only.md + - Persistent queries: features/persistent-queries.md + - Soft delete: features/soft-delete.md + - Virtual columns: features/virtual-columns.md + - Hooks: features/hooks.md + - Transactions: features/transactions.md + - Cache: features/cache.md + - Error transformer: features/error-transformer.md + - Adapters: features/adapters.md + - Architecture: + - architecture/index.md + - Ideology: architecture/ideology.md + - Layers: architecture/layers.md + - SQL generation: architecture/sql-generation.md + - Field mapping: architecture/field-mapping.md + - Caching internals: architecture/caching.md + - Adapter internals: architecture/adapters-internals.md + - Contributing: architecture/contributing.md