diff --git a/docs/design/fulltext2-base-file-reuse-experiment.md b/docs/design/fulltext2-base-file-reuse-experiment.md new file mode 100644 index 0000000000000..d9f52115e099b --- /dev/null +++ b/docs/design/fulltext2-base-file-reuse-experiment.md @@ -0,0 +1,165 @@ +# FULLTEXT2 immutable Base-file reuse experiment + +Status: **default-off experiment; not a production rollout approval**. +Related work: [#27445](https://github.com/matrixorigin/matrixone/issues/27445) +and the independently merged CDC read-only cache contract, [#29227](https://github.com/matrixorigin/matrixone/pull/29227). + +## Problem and scope + +Normal stale/TTL replacement reconstructs a search Index. Its immutable Base +chunks may be streamed from SQL and materialized again even when only Tail/Delete +changed. This experiment retains the Base files within one CN service lifecycle, +not the decoded Index. Ordinary CDC must neither evict a warm Search nor access, +fill, invalidate or prewarm this pool. Normal freshness and DDL remain independent. + +Only `fulltext2_base_file_reuse` builds install the service owner. Ordinary builds +keep the existing loader. There is no SQL variable, public capacity setting, +persistent cache format, background refresher, cross-restart reuse or shared +Segment. The experimental budget is 4 GiB / 64 files per service. This is not a +decision about a production default or unified disk budget. + +## Ownership, identity and lifecycle + +``` +CN Start -> generation token -> service owner -> bounded immutable files + | + Search -> lease -> private mmap/Segment/Index +``` + +Every load reads metadata with the current request/transaction before acquisition. +The key includes service, effective account, physical storage/metadata identity, +source/index/PK identity, build segment ID, checksum and size. Tail watermark is +not part of the key. Metadata/permission errors cannot be satisfied by local hits. +Missing identity uses the ordinary loader; checksum alone is not cross-index +identity. CREATE/REBUILD/MERGE use the existing per-build IDs; current and historical +loads may share a file only when their own metadata selects the same identity. + +Files transition ABSENT -> FILLING -> READY. One request fills; waiters retain +their own cancellation and transactions. Filling reserves bytes and an FD before +creating the file. READY contents are not modified. Each lease pins before mmap; +every Search separately decodes and constructs Tail/Delete, liveness and scoring. +Only idle READY files are evicted, in acquisition-recency order. + +Search.Destroy releases loaded resources but retains its original owner/closed +binding because the vector cache can retry that same algorithm object. CN Close +rejects/cancels owner operations, drains only its service's cache entries with +expected-entry identity, then closes the pool. The retryable owner cleanup is +outside CN closeOnce and preserves the CN fail-stop/local-completion contract. +Old generation tokens and delayed entry closers cannot drain a new instance. +Each entry's close mutex spans current-entry validation, service drain, owner +cleanup and matching registry retirement. Pending cleanup retains the closed +entry before releasing that mutex; queued closers revalidate only after retirement. +The registry mutex never spans cache or resource waits. Process-wide pre-cancellation +remains independent of these close mutexes. + +Successful owner cleanup removes the registry entry. Pending entries remain owned +and charged until a later close succeeds. SQL factory never initializes an owner, +including when the registry is empty: only explicit CN Start can do so. The stable +process shutdown dispatcher is registered once and does not retain historical +owner closures. Global cache Destroy first closes registry/operation admission +and cancels every current owner's complete-operation context, before waiting for +the cache serve loop or entry locks. This prephase performs no pool cleanup or +drain; the post-eviction dispatcher drains/releases resources and retains pending +owners for retry. Housekeeping does not close admission. No cleanup worker or +permanent per-UUID tombstone is introduced. + +## Failure and resource rules + +| Condition | Outcome | +|---|---| +| Retention capacity/FD/pinned/validation-byte admission refusal | One ordinary load under original resource admission | +| Healthy waiter whose fill leader was canceled | One ordinary load with waiter's context and transaction | +| Unresolved deferred Segment or linked-file cleanup | Typed pending error before READY or ordinary mapping admission; no fallback | +| Waiter cancellation or closed owner/pool | Error; no resurrection through fallback | +| Published READY size/checksum/decode corruption | Retire, then one ordinary load, without refilling the pool | +| First fill source/permission/checksum/decode failure | Original error, no automatic retry | + +Owner cancellation covers complete Preload/Load and nested ordinary fallback, +without replacing execution identity, snapshot or transaction. The pool reuses immutable +files, never shares decoded Segments. The first fill retains the existing validation +decode followed by the first caller's decode. Historical measurements belong +to the earlier revision described below; final-head net performance remains +unaccepted. This change retains that cost without introducing a handoff state machine. + +Retired pinned files retain byte/FD charges. Failed FILLING cleanup keeps its FD +reservation until close. Failed munmap retains a reachable deferred Segment or +mapping. Reservation is retained while deferred bytes are registered, then +released: transient conservative double charging is allowed, undercharging is +not. Successful Search destruction releases its mappings/leases; idle files may +remain. Successful owner shutdown requires all files, mappings, deferred resources +and reservations to be zero; permanent OS cleanup failure remains explicit pending. + +Experimental ordinary fallbacks pin the cleanup owner from initial mmap until +successful Free, independently of cache map residence. An eviction already removed +from the generic cache cannot make service cleanup report complete before that +consumer releases its mapping/path. Live pins do not authorize pool cleanup to +free a live Search; only failed releases enter the deferred retry collection. + +Linked pool files whose unlink fails retain their exact path and disk-byte debt +after their descriptor is closed once. Failed fill passes the real handle to the +same cleanup owner. These debts quarantine new materialization, remain charged +to retention bytes, and are retried at later cleanup boundaries; successful +unlink or an already absent path clears debt once. No closed FD is charged or +reclosed. LOCAL absence or failure can reach this linked path in production. + +Experimental ordinary fallbacks bind the same retry owner as pooled Segments +immediately after mmap, before checksum/decode validation or publication. Failed +Segment release preserves its mapping plus the cache's per-doc heap-model charge +under that owner after eviction. While any such cleanup remains pending, complete +owner operations and both READY and ordinary materialization reject new work with +a distinct pending error. Existing admitted work can finish and transfer its finite +resources under the original governor/tenant admission; capacity fallback cannot +bypass this quarantine. Successful retry removes each retained charge once and +reopens an otherwise open owner. No background retry worker or new resource budget +is introduced. Default ordinary loading keeps its previous semantics. + +## Alternatives and decision record + +* Ordinary per-load files: simplest and retained as the default/fallback; repeats + Base SQL and file writes on reload. +* Share full Segment/Index or incrementally mutate cached Tail: rejected because + decode/liveness/scoring ownership would cross search generations and the CDC + read-only contract. It expands correctness scope for a narrower I/O problem. +* Retain only immutable files: selected as the narrow prototype. It still pays + checksum, decode, Tail and liveness costs, retains disk/FDs, and does not help + first load, restart or capacity misses. Each independent mmap remains charged by + the existing search governor, even when file backing is shared. + +The current repair scope accepts this default-off immutable-file architecture +and the bounded fallback/service lifecycle obligations. The task evidence +`trusted-design-scope-acceptance.json` records that current engineering decision; +it does not establish a historical human approval on 2026-10-08. Closed UUID +tombstones and query-side owner creation remain rejected. Production enablement +and final-head performance acceptance remain separate gates. + +## Verification and rollout gates + +Package tests cover single fill/private mappings, ordinary fallback, corruption, +context/transaction identity, canceled loads, quota/deferred ownership, cache drain, +old tokens, repeated cleanup and reclaimed/pending registry entries. Concurrent +tests use phase barriers and rescue/join before restoring mocks. The tagged embedded +SQL test covers fixed initial PKs {1,4}, cross-phase alpha score stability, warm +residence, old-versus-new Tail terms and unchanged Base identity across a new +Search reload, nonempty MERGE, REBUILD, same-name +recreation, snapshot/restore, accounts and actual CN close/restart. The canonical +snapshot BVT additionally checks current versus historical results across recreation +and REBUILD; neither SQL test nor BVT proves a file hit or capacity fallback. +A separate real-materializer component fixture compares fixed PKs and exact +score bits against an independently materialized ordinary loader, and counts +actual Base source streams for a new READY Segment and capacity fallback. +This is fixture equivalence, not complete ranking or performance acceptance. + +Two historical Linux D->R pairs on the earlier `986ca0a7` source avoided 2,079,538,688 +bytes of Base streaming and successful writes per reload, with 64 hits / 22 capacity +fallbacks. Reload wall time fell 17.936 -> 11.017 s and 19.082 -> 11.744 s. Initial +load increased 0.505 s and 0.451 s. These are historical same-order paired observations, +not final-main performance tests, not the planned three-pair gate, and remain +`PERF_INCONCLUSIVE_HOST_BUSY`. Both controls also had zero C10 timeouts, so no +claim that the historical timeout is fixed is supported. + +Before Ready/default enablement: complete reversed/fixed repeatability, comparable +CPU/fallback/warm cost and a normal-lifecycle frequency/payback window, establish +production disk/FD admission policy, and finish exact-head CI/deployment QA. No +on-disk change needs migration; rollback is a normal build without the tag and +clean service shutdown. Unverified timing, permission-denial and deployment +scenarios remain explicit gaps rather than successful acceptance. diff --git a/pkg/cnservice/fulltext2_owner_default.go b/pkg/cnservice/fulltext2_owner_default.go new file mode 100644 index 0000000000000..31d57a3062ac2 --- /dev/null +++ b/pkg/cnservice/fulltext2_owner_default.go @@ -0,0 +1,11 @@ +//go:build !fulltext2_base_file_reuse + +package cnservice + +type fulltext2BaseReuseOwnerToken struct{} + +func initializeFulltext2BaseReuseOwner(_ string) (*fulltext2BaseReuseOwnerToken, error) { + return nil, nil +} + +func closeFulltext2BaseReuseOwner(_ string, _ *fulltext2BaseReuseOwnerToken) error { return nil } diff --git a/pkg/cnservice/fulltext2_owner_experimental.go b/pkg/cnservice/fulltext2_owner_experimental.go new file mode 100644 index 0000000000000..1c648bf5468e0 --- /dev/null +++ b/pkg/cnservice/fulltext2_owner_experimental.go @@ -0,0 +1,16 @@ +//go:build fulltext2_base_file_reuse + +package cnservice + +import "github.com/matrixorigin/matrixone/pkg/fulltext2" + +type fulltext2BaseReuseOwnerToken = fulltext2.BaseFileReuseOwnerToken + +func initializeFulltext2BaseReuseOwner(service string) (*fulltext2BaseReuseOwnerToken, error) { + return fulltext2.InitializeBaseFileReuseOwner(service) +} + +func closeFulltext2BaseReuseOwner(service string, token *fulltext2BaseReuseOwnerToken) error { + _ = service // retained for the default-build-compatible call signature + return fulltext2.CloseBaseFileReuseOwner(token) +} diff --git a/pkg/cnservice/fulltext2_owner_test.go b/pkg/cnservice/fulltext2_owner_test.go new file mode 100644 index 0000000000000..8be7b03d34803 --- /dev/null +++ b/pkg/cnservice/fulltext2_owner_test.go @@ -0,0 +1,32 @@ +// Copyright 2026 Matrix Origin +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +package cnservice + +import ( + "errors" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestFulltext2OwnerCloseRetryPreservesWithdrawalDiagnostic(t *testing.T) { + diagnostic := errors.New("remote withdrawal diagnostic") + s := &service{cfg: &Config{}, closeErr: diagnostic, fulltext2BaseReuseClosePending: true} + // Local teardown has already completed; only owner cleanup was pending. + s.closeOnce.Do(func() {}) + require.ErrorIs(t, s.Close(), diagnostic) + require.True(t, s.CloseComplete(), "successful owner retry certifies local teardown despite a remote diagnostic") + require.False(t, s.fulltext2BaseReuseClosePending) + require.ErrorIs(t, s.Close(), diagnostic) + require.True(t, s.CloseComplete()) +} + +func TestFulltext2OwnerCloseDoesNotCertifyFailedDrain(t *testing.T) { + failure := errors.New("local producer drain failed") + s := &service{closeErr: failure} + s.closeOnce.Do(func() {}) + require.ErrorIs(t, s.Close(), failure) + require.False(t, s.CloseComplete(), "a nil experimental owner cannot erase a local drain failure") +} diff --git a/pkg/cnservice/server.go b/pkg/cnservice/server.go index 79683dd520633..cf2ddceb1c226 100644 --- a/pkg/cnservice/server.go +++ b/pkg/cnservice/server.go @@ -444,6 +444,9 @@ func (s *service) Start() (err error) { } s.lifecycle = serviceStarted }() + if s.fulltext2BaseReuseOwner, err = initializeFulltext2BaseReuseOwner(s.cfg.UUID); err != nil { + return err + } if err = s.waitForClusterSelfReady(false); err != nil { return err @@ -602,7 +605,20 @@ func (s *service) closeService() error { s.closeComplete = localErr == nil s.closeErr = errors.Join(withdrawErr, localErr) }) - return s.closeErr + // Keep this retryable operation outside closeOnce. A pinned mapping or a + // deferred munmap may make the first owner close return pending; a later + // service Close must be able to retry it after Search handles release their + // leases. The owner itself has already rejected new work, and closeService's + // once body has drained the service entry points before reaching here. + if !s.closeComplete && !s.fulltext2BaseReuseClosePending { + // Preserve the fail-stop drain contract: a local producer/dependency + // teardown failure is not permission to destroy its cached consumers. + return s.closeErr + } + ownerErr := closeFulltext2BaseReuseOwner(s.cfg.UUID, s.fulltext2BaseReuseOwner) + s.fulltext2BaseReuseClosePending = ownerErr != nil + s.closeComplete = ownerErr == nil + return errors.Join(s.closeErr, ownerErr) } // CloseComplete certifies local teardown, not a successful remote generation diff --git a/pkg/cnservice/types.go b/pkg/cnservice/types.go index 12ac04c405303..2a386c83b0af2 100644 --- a/pkg/cnservice/types.go +++ b/pkg/cnservice/types.go @@ -856,14 +856,18 @@ type service struct { beforeViewMetadataAdmissionHandoff func() // viewMetadataCloseFn is a deterministic test hook for the asynchronous // close request issued after synchronous ingress revocation. - viewMetadataCloseFn func() error - aicm *defines.AutoIncrCacheManager - lifecycleMu sync.Mutex - frontendLifecycleMu sync.Mutex - lifecycle serviceLifecycleState - closeOnce sync.Once - closeErr error - closeComplete bool + viewMetadataCloseFn func() error + aicm *defines.AutoIncrCacheManager + lifecycleMu sync.Mutex + frontendLifecycleMu sync.Mutex + lifecycle serviceLifecycleState + closeOnce sync.Once + closeErr error + closeComplete bool + fulltext2BaseReuseOwner *fulltext2BaseReuseOwnerToken + // Only owner cleanup may be retried after local teardown. Remote admission + // withdrawal errors are diagnostic and must not prevent that retry finishing. + fulltext2BaseReuseClosePending bool task struct { sync.RWMutex diff --git a/pkg/fulltext2/base_file_cleanup_final_test.go b/pkg/fulltext2/base_file_cleanup_final_test.go new file mode 100644 index 0000000000000..2d6dfccdec168 --- /dev/null +++ b/pkg/fulltext2/base_file_cleanup_final_test.go @@ -0,0 +1,310 @@ +// Copyright 2026 Matrix Origin +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +//go:build (darwin || linux) && fulltext2_base_file_reuse + +package fulltext2 + +import ( + "context" + "errors" + "math" + "os" + "path/filepath" + "sync" + "testing" + "time" + + "github.com/matrixorigin/matrixone/pkg/container/types" + "github.com/matrixorigin/matrixone/pkg/util/executor" + veccache "github.com/matrixorigin/matrixone/pkg/vectorindex/cache" + "github.com/matrixorigin/matrixone/pkg/vectorindex/sqlexec" + "github.com/stretchr/testify/require" +) + +type detachedCleanupSearch struct { + *Fulltext2Search + entered chan struct{} + resume chan struct{} +} + +func (s *detachedCleanupSearch) Destroy() { close(s.entered); <-s.resume; s.Fulltext2Search.Destroy() } + +func TestBaseFileCleanupDetachedEvictionLifetime(t *testing.T) { + for _, mode := range []string{"fallback-fault", "fallback-clean", "pooled"} { + t.Run(mode, func(t *testing.T) { + sp, cfg, buf := cleanupReworkSource(t, nil, "") + capacity := int64(1) + if mode == "pooled" { + capacity = int64(len(buf)) * 2 + } + owner := newBaseFileOwner(capacity, 2) + previous := veccache.Cache + c := veccache.NewVectorIndexCache() + veccache.Cache = c + t.Cleanup(func() { veccache.Cache = previous }) + withExperimentalOwnerState(t, owner, "cleanup-rework", false) + token, err := InitializeBaseFileReuseOwner("cleanup-rework") + require.NoError(t, err) + p, err := owner.poolForSearch() + require.NoError(t, err) + selectMapping, restore, released := cleanupReworkFault(t, owner) + seg, err := loadFromStorageWithOwner(sp, cfg, "seg0", owner, p) + require.NoError(t, err) + path := seg.mmapPath + ptr := &seg.mmapData[0] + selectMapping(seg.mmapData) + if mode != "fallback-fault" { + restore() + } + s := newFulltext2SearchWithBaseOwner(cfg, owner) + s.idx = NewIndex([]*Segment{seg}, nil) + s.loaded = true + blocked := &detachedCleanupSearch{Fulltext2Search: s, entered: make(chan struct{}), resume: make(chan struct{})} + entry := &veccache.VectorIndexSearch{Algo: blocked} + entry.Cond = sync.NewCond(entry.Mutex.RLocker()) + entry.Status.Store(veccache.STATUS_LOADED) + c.IndexMap.Store("detached", entry) + joined := make(chan struct{}) + var resume sync.Once + go func() { c.Remove("detached"); close(joined) }() + t.Cleanup(func() { + resume.Do(func() { close(blocked.resume) }) + <-joined + restore() + _ = owner.close() + seg.Free() + if path != "" { + _ = os.Remove(path) + } + }) + select { + case <-blocked.entered: + case <-time.After(5 * time.Second): + t.Fatal("real eviction did not reach destructor") + } + _, present := c.IndexMap.Load("detached") + require.False(t, present) + require.NotNil(t, seg.mmapData, "mapping remains live after cache removal") + require.ErrorIs(t, CloseBaseFileReuseOwner(token), errBaseFileOwnerPending, "pre-Free close must not unregister a live consumer") + _, err = InitializeBaseFileReuseOwner("cleanup-rework") + require.ErrorIs(t, err, errBaseFileOwnerPending) + resume.Do(func() { close(blocked.resume) }) + <-joined + if mode == "fallback-fault" { + require.ErrorIs(t, CloseBaseFileReuseOwner(token), errBaseFileOwnerPending) + requireDeferredSegmentCharge(t, p, int64(len(buf))+estBytesPerDocHeap) + require.Zero(t, released[ptr]) + restore() + } + require.NoError(t, CloseBaseFileReuseOwner(token)) + require.False(t, owner.hasResources()) + requireDeferredSegmentCharge(t, p, 0) + require.Equal(t, 1, released[ptr]) + require.Nil(t, seg.mmapData) + if path != "" { + _, err = os.Stat(path) + require.True(t, os.IsNotExist(err)) + } + next, err := InitializeBaseFileReuseOwner("cleanup-rework") + require.NoError(t, err) + require.NoError(t, CloseBaseFileReuseOwner(token)) + require.False(t, experimentalOwnerForSQL("cleanup-rework").closed) + require.NoError(t, CloseBaseFileReuseOwner(next)) + }) + } +} + +func TestBaseFileCleanupLinkedRemoveFailure(t *testing.T) { + if os.Geteuid() == 0 { + t.Skip("real directory EACCES requires a non-root uid") + } + for _, mode := range []string{"ready-close", "ready-eviction", "failed-fill", "already-removed"} { + t.Run(mode, func(t *testing.T) { + dir := t.TempDir() + t.Setenv("TMPDIR", dir) + sp, cfg, buf := cleanupReworkSource(t, nil, "") + owner := newBaseFileOwner(int64(len(buf)), 1) + withExperimentalOwnerState(t, owner, "cleanup-rework", false) + token, err := InitializeBaseFileReuseOwner("cleanup-rework") + require.NoError(t, err) + p, err := owner.poolForSearch() + require.NoError(t, err) + var path string + var fd *os.File + t.Cleanup(func() { + _ = os.Chmod(dir, 0700) + _ = owner.close() + if path != "" { + _ = os.Remove(path) + } + }) + if mode == "failed-fill" { + swapRunStreamingSql(t, func(_ context.Context, _ *sqlexec.SqlProcess, _ string, _ chan executor.Result, _ chan error) (executor.Result, error) { + paths, e := filepath.Glob(filepath.Join(dir, "ft2idx_pool*")) + require.NoError(t, e) + require.Len(t, paths, 1) + path = paths[0] + require.NoError(t, os.Chmod(dir, 0500)) + return executor.Result{}, errors.New("selected source error") + }) + _, err = loadFromStorageWithOwner(sp, cfg, "seg0", owner, p) + require.ErrorContains(t, err, "selected source error") + } else { + seg, e := loadFromStorageWithOwner(sp, cfg, "seg0", owner, p) + require.NoError(t, e) + t.Cleanup(seg.Free) + p.mu.Lock() + for _, entry := range p.entries { + path = entry.handle.path + fd = entry.handle.file + } + p.mu.Unlock() + require.NotEmpty(t, path, "real materializer must use its linked fallback") + seg.Free() + require.NoError(t, os.Chmod(dir, 0500)) + if mode == "ready-eviction" { + p.mu.Lock() + e = p.makeRoomLocked(int64(len(buf))) + p.mu.Unlock() + require.ErrorIs(t, e, errBaseFileCleanupPending, "failed unlink must quarantine new mappings") + } + } + require.ErrorIs(t, CloseBaseFileReuseOwner(token), errBaseFileOwnerPending, "unlink failure cannot report clean") + require.FileExists(t, path) + p.mu.Lock() + require.Equal(t, int64(len(buf)), p.deferredFileBytes) + require.Len(t, p.deferredFiles, 1) + p.mu.Unlock() + if fd != nil { + _, err = fd.Stat() + require.ErrorIs(t, err, os.ErrClosed, "failed unlink must not retain a fictional open FD") + } + p.mu.Lock() + require.Zero(t, p.openFiles) + require.Zero(t, p.reservedFiles) + p.mu.Unlock() + _, err = InitializeBaseFileReuseOwner("cleanup-rework") + require.ErrorIs(t, err, errBaseFileOwnerPending) + require.NoError(t, os.Chmod(dir, 0700)) + if mode == "already-removed" { + require.NoError(t, os.Remove(path)) + } + require.NoError(t, CloseBaseFileReuseOwner(token)) + require.False(t, owner.hasResources()) + p.mu.Lock() + require.Zero(t, p.deferredFileBytes) + require.Empty(t, p.deferredFiles) + p.mu.Unlock() + _, err = os.Stat(path) + require.True(t, os.IsNotExist(err)) + require.NoError(t, CloseBaseFileReuseOwner(token)) + }) + } +} + +// The oracle is the ordinary loader over the same fixed source, independently +// materialized before the experiment. Stream counts distinguish READY reuse +// from a warm Search and from real capacity fallback. This is a component test, +// not an embedded SQL file-hit counter or a complete ranking acceptance gate. +func TestBaseFileCleanupOrdinaryScoreOracle(t *testing.T) { + b := NewBuilder("seg0", int32(types.T_int64)) + feed(t, b, int64(1), "alpha", "beta", "gamma") + feed(t, b, int64(2), "beta", "gamma", "delta") + feed(t, b, int64(3), "gamma", "delta", "epsilon") + feed(t, b, int64(4), "alpha", "zeta") + source, err := b.Finish() + require.NoError(t, err) + t.Cleanup(source.Free) + buf, err := source.Serialize() + require.NoError(t, err) + sp, cfg, _ := cleanupReworkSource(t, buf, "") + originalStream := runStreamingSql + streams := 0 + runStreamingSql = func(ctx context.Context, sp *sqlexec.SqlProcess, q string, sc chan executor.Result, ec chan error) (executor.Result, error) { + streams++ + return originalStream(ctx, sp, q, sc, ec) + } + t.Cleanup(func() { runStreamingSql = originalStream }) + scores := func(seg *Segment) map[int64]uint32 { + t.Helper() + results, e := NewIndex([]*Segment{seg}, nil).SearchQuery([]byte("alpha"), true, ParserDefault, BM25, 10, nil) + require.NoError(t, e) + got := map[int64]uint32{} + for _, r := range results { + got[r.Pk.(int64)] = math.Float32bits(r.Score) + } + require.Len(t, got, 2) + require.Contains(t, got, int64(1)) + require.Contains(t, got, int64(4)) + return got + } + ordinary, err := LoadFromStorage(sp, cfg, "seg0") + require.NoError(t, err) + t.Cleanup(ordinary.Free) + want := scores(ordinary) + require.Equal(t, 1, streams) + pool := newBaseFilePool(int64(len(buf)), 1) + t.Cleanup(pool.Close) + first, err := loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err) + t.Cleanup(first.Free) + require.Equal(t, want, scores(first)) + first.Free() + require.Equal(t, 2, streams) + hit, err := loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err) + t.Cleanup(hit.Free) + require.NotSame(t, first, hit) + require.Equal(t, want, scores(hit)) + require.Equal(t, 2, streams, "READY skips actual source stream for a new Segment") + fallbackPool := newBaseFilePool(1, 1) + t.Cleanup(fallbackPool.Close) + fallback, err := loadFromStorageWithPool(sp, cfg, "seg0", fallbackPool) + require.NoError(t, err) + t.Cleanup(fallback.Free) + require.Nil(t, fallback.mmapRelease, "ordinary fallback owns no pooled lease") + require.True(t, fallback.mmapFallbackPinned) + require.Equal(t, want, scores(fallback)) + require.Equal(t, 3, streams, "capacity fallback rematerializes source once") +} + +func TestBaseFileCleanupFallbackUnlinkAfterUnmap(t *testing.T) { + if os.Geteuid() == 0 { + t.Skip("real directory EACCES requires a non-root uid") + } + dir := t.TempDir() + t.Setenv("TMPDIR", dir) + sp, cfg, buf := cleanupReworkSource(t, nil, "") + owner := newBaseFileOwner(1, 1) + p, err := owner.poolForSearch() + require.NoError(t, err) + selectMapping, restore, released := cleanupReworkFault(t, owner) + seg, err := loadFromStorageWithOwner(sp, cfg, "seg0", owner, p) + require.NoError(t, err) + path := seg.mmapPath + ptr := &seg.mmapData[0] + selectMapping(seg.mmapData) + restore() + t.Cleanup(func() { _ = os.Chmod(dir, 0700); seg.Free(); _ = owner.close(); _ = os.Remove(path) }) + require.NoError(t, os.Chmod(dir, 0500)) + seg.Free() + require.Nil(t, seg.mmapData, "actual munmap already succeeded") + require.Equal(t, 1, released[ptr]) + require.FileExists(t, path) + requireDeferredSegmentCharge(t, p, int64(len(buf))+estBytesPerDocHeap) + require.ErrorIs(t, owner.close(), errBaseFileOwnerPending) + require.Equal(t, 1, released[ptr], "path retry must not repeat successful munmap") + p.mu.Lock() + require.Equal(t, 1, p.liveFallbacks) + require.Zero(t, p.openFiles) + p.mu.Unlock() + require.NoError(t, os.Chmod(dir, 0700)) + require.NoError(t, owner.close()) + require.Equal(t, 1, released[ptr]) + require.False(t, owner.hasResources()) + requireDeferredSegmentCharge(t, p, 0) + _, err = os.Stat(path) + require.True(t, os.IsNotExist(err)) +} diff --git a/pkg/fulltext2/base_file_cleanup_rework_test.go b/pkg/fulltext2/base_file_cleanup_rework_test.go new file mode 100644 index 0000000000000..d8f323b3e40e0 --- /dev/null +++ b/pkg/fulltext2/base_file_cleanup_rework_test.go @@ -0,0 +1,398 @@ +// Copyright 2026 Matrix Origin +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +//go:build (darwin || linux) && fulltext2_base_file_reuse + +package fulltext2 + +import ( + "context" + "errors" + "os" + "reflect" + "runtime" + "strings" + "sync" + "testing" + "time" + + "github.com/matrixorigin/matrixone/pkg/container/batch" + "github.com/matrixorigin/matrixone/pkg/container/types" + "github.com/matrixorigin/matrixone/pkg/util/executor" + "github.com/matrixorigin/matrixone/pkg/vectorindex" + veccache "github.com/matrixorigin/matrixone/pkg/vectorindex/cache" + "github.com/matrixorigin/matrixone/pkg/vectorindex/sqlexec" + "github.com/stretchr/testify/require" +) + +func cleanupReworkSource(t *testing.T, payload []byte, checksum string) (*sqlexec.SqlProcess, TableConfig, []byte) { + sp, mp := mockSqlProcWithIdentity(t, "cleanup-rework") + if payload == nil { + b := NewBuilder("seg0", int32(types.T_int64)) + feed(t, b, int64(1), "hello", "world") + s, err := b.Finish() + require.NoError(t, err) + payload, err = s.Serialize() + require.NoError(t, err) + } + if checksum == "" { + checksum = vectorindex.CheckSumFromBuffer(payload) + } + swapRunSql(t, func(_ *sqlexec.SqlProcess, _ string) (executor.Result, error) { + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, checksum, int64(len(payload)), 7)}}, nil + }) + swapRunStreamingSql(t, func(_ context.Context, _ *sqlexec.SqlProcess, _ string, sc chan executor.Result, _ chan error) (executor.Result, error) { + sc <- executor.Result{Mp: mp, Batches: []*batch.Batch{chunkBatch(mp, payload)}} + return executor.Result{}, nil + }) + return sp, testStorageCfg(), payload +} + +// Retain an independent address ledger even on the old candidate's owner-loss +// path. Cleanup restores munmap, retries the owner, then rescues only mappings +// which were never successfully released; this test cannot itself leak. +func cleanupReworkFault(t *testing.T, owner *baseFileOwner) (func([]byte), func(), map[*byte]int) { + original := munmapFn + selected := map[*byte][]byte{} + released := map[*byte]int{} + failing := true + munmapFn = func(data []byte) error { + ptr := &data[0] + if _, ok := selected[ptr]; ok && failing { + return errors.New("selected cleanup fault") + } + err := original(data) + if err == nil { + released[ptr]++ + } + return err + } + t.Cleanup(func() { + defer func() { munmapFn = original }() + munmapFn = func(data []byte) error { + err := original(data) + if err == nil { + released[&data[0]]++ + } + return err + } + _ = owner.close() + for ptr, data := range selected { + if released[ptr] == 0 { + require.NoError(t, original(data)) + released[ptr]++ + } + } + munmapFn = original + }) + return func(data []byte) { selected[&data[0]] = data; released[&data[0]] = 0 }, func() { failing = false }, released +} + +func cleanupReworkEvict(t *testing.T, cfg TableConfig, owner *baseFileOwner, seg *Segment) { + c := veccache.NewVectorIndexCache() + s := newFulltext2SearchWithBaseOwner(cfg, owner) + s.idx = NewIndex([]*Segment{seg}, nil) + s.loaded = true + entry := &veccache.VectorIndexSearch{Algo: s} + entry.Cond = sync.NewCond(entry.Mutex.RLocker()) + entry.Status.Store(veccache.STATUS_LOADED) + c.IndexMap.Store("cleanup", entry) + c.Remove("cleanup") + _, exists := c.IndexMap.Load("cleanup") + require.False(t, exists) + require.Nil(t, s.idx) +} + +func TestBaseFileCleanupReadyEvictionRejectsUnchargedReload(t *testing.T) { + sp, cfg, buf := cleanupReworkSource(t, nil, "") + owner := newBaseFileOwner(int64(len(buf))*2, 2) + p, err := owner.poolForSearch() + require.NoError(t, err) + selectMapping, restore, released := cleanupReworkFault(t, owner) + first, err := loadFromStorageWithOwner(sp, cfg, "seg0", owner, p) + require.NoError(t, err) + t.Cleanup(first.Free) + second, err := loadFromStorageWithOwner(sp, cfg, "seg0", owner, p) + require.NoError(t, err) + t.Cleanup(second.Free) + selectMapping(first.mmapData) + selectMapping(second.mmapData) + ptr1, ptr2 := &first.mmapData[0], &second.mmapData[0] + cleanupReworkEvict(t, cfg, owner, first) + for i := 0; i < 3; i++ { + var extra *Segment + var err error + if i%2 == 0 { + extra, err = loadFromStorageWithPool(sp, cfg, "seg0", p) + } else { + extra, err = loadFromStorageWithOwner(sp, cfg, "seg0", owner, p) + } + if extra != nil { + extra.Free() + } + require.Error(t, err, "READY reload must reject while an evicted mapping remains owned") + } + cleanupReworkEvict(t, cfg, owner, second) + require.Equal(t, 2, p.deferredCount()) + require.ErrorIs(t, owner.close(), errBaseFileOwnerPending) + require.Zero(t, released[ptr1]) + require.Zero(t, released[ptr2]) + requireDeferredSegmentCharge(t, p, 2*(int64(len(buf))+estBytesPerDocHeap)) + restore() + require.NoError(t, owner.close()) + require.False(t, owner.hasResources()) + requireDeferredSegmentCharge(t, p, 0) + require.Equal(t, 1, released[ptr1]) + require.Equal(t, 1, released[ptr2]) + require.NoError(t, owner.close()) + require.Equal(t, 1, released[ptr1]) + require.Equal(t, 1, released[ptr2]) +} + +func TestBaseFileCleanupCapacityFallbackEvictionRetainsOwner(t *testing.T) { + sp, cfg, buf := cleanupReworkSource(t, nil, "") + owner := newBaseFileOwner(1, 1) + previousCache := veccache.Cache + veccache.Cache = veccache.NewVectorIndexCache() + t.Cleanup(func() { veccache.Cache = previousCache }) + withExperimentalOwnerState(t, owner, "cleanup-rework", false) + token, err := InitializeBaseFileReuseOwner("cleanup-rework") + require.NoError(t, err) + p, err := owner.poolForSearch() + require.NoError(t, err) + selectMapping, restore, released := cleanupReworkFault(t, owner) + seg, err := loadFromStorageWithOwner(sp, cfg, "seg0", owner, p) + require.NoError(t, err) + require.Equal(t, len(buf), len(seg.mmapData)) + require.NotEmpty(t, seg.mmapPath) + path := seg.mmapPath + ptr := &seg.mmapData[0] + selectMapping(seg.mmapData) + t.Cleanup(func() { _ = os.Remove(path) }) + cleanupReworkEvict(t, cfg, owner, seg) + require.ErrorIs(t, owner.close(), errBaseFileOwnerPending, "fallback must prevent false clean owner shutdown") + require.Equal(t, 1, p.deferredCount()) + require.ErrorIs(t, CloseBaseFileReuseOwner(token), errBaseFileOwnerPending) + _, err = InitializeBaseFileReuseOwner("cleanup-rework") + require.ErrorIs(t, err, errBaseFileOwnerPending) + require.Zero(t, released[ptr]) + requireDeferredSegmentCharge(t, p, int64(len(buf))+estBytesPerDocHeap) + restore() + require.NoError(t, owner.close()) + require.False(t, owner.hasResources()) + requireDeferredSegmentCharge(t, p, 0) + _, err = os.Stat(path) + require.True(t, os.IsNotExist(err)) + require.NoError(t, owner.close()) + require.Equal(t, 1, released[ptr]) + require.NoError(t, CloseBaseFileReuseOwner(token)) + nextToken, err := InitializeBaseFileReuseOwner("cleanup-rework") + require.NoError(t, err) + t.Cleanup(func() { require.NoError(t, CloseBaseFileReuseOwner(nextToken)) }) + next := NewFulltext2SearchForExecution(cfg, "cleanup-rework") + require.NotSame(t, owner, next.baseOwner) + nextPool, err := next.baseOwner.poolForSearch() + require.NoError(t, err) + nextSeg, err := loadFromStorageWithOwner(sp, cfg, "seg0", next.baseOwner, nextPool) + require.NoError(t, err) + t.Cleanup(nextSeg.Free) + require.NoError(t, CloseBaseFileReuseOwner(token)) + df, ok := nextSeg.lookupLoadedDF("hello") + require.True(t, ok) + require.Equal(t, 1, df) + nextSeg.Free() + require.NoError(t, CloseBaseFileReuseOwner(nextToken)) +} + +// Validation errors occur before a Segment can be published to Search. Capture +// the actual mapping at its first attempted release, never a fake Segment. +func TestBaseFileCleanupFallbackValidationRetainsOwner(t *testing.T) { + for _, name := range []string{"checksum", "decode"} { + t.Run(name, func(t *testing.T) { + var payload []byte + checksum := "mismatch" + if name == "decode" { + payload = []byte("not a serialized segment") + checksum = "" + } + sp, cfg, _ := cleanupReworkSource(t, payload, checksum) + owner := newBaseFileOwner(1, 1) + p, err := owner.poolForSearch() + require.NoError(t, err) + selectMapping, restore, released := cleanupReworkFault(t, owner) + previous := munmapFn + var ptr *byte + munmapFn = func(data []byte) error { + if ptr == nil { + ptr = &data[0] + selectMapping(data) + } + return previous(data) + } + _, err = loadFromStorageWithOwner(sp, cfg, "seg0", owner, p) + require.Error(t, err) + require.NotNil(t, ptr) + require.Equal(t, 1, p.deferredCount()) + require.Positive(t, deferredSegmentCharge(p)) + require.ErrorIs(t, owner.close(), errBaseFileOwnerPending) + restore() + require.NoError(t, owner.close()) + requireDeferredSegmentCharge(t, p, 0) + require.Equal(t, 1, released[ptr]) + require.False(t, owner.hasResources()) + }) + } +} + +func TestBaseFileCleanupFallbackFullSearchFailureRetainsOwner(t *testing.T) { + for _, mode := range []string{"partial-base", "cancel-after-base"} { + t.Run(mode, func(t *testing.T) { + sp, cfg, buf := cleanupReworkSource(t, nil, "") + _, mp := mockSqlProc(t) + ctx, cancel := context.WithCancel(sp.GetTopContext()) + defer cancel() + sp = sp.WithContext(ctx) + owner := newBaseFileOwner(1, 1) + p, err := owner.poolForSearch() + require.NoError(t, err) + selectMapping, restore, released := cleanupReworkFault(t, owner) + previous := munmapFn + var ptr *byte + munmapFn = func(data []byte) error { + if ptr == nil { + ptr = &data[0] + selectMapping(data) + } + return previous(data) + } + swapRunSql(t, func(_ *sqlexec.SqlProcess, sql string) (executor.Result, error) { + switch { + case strings.Contains(sql, "SUM("): + return executor.Result{}, nil + case strings.Contains(sql, "checksum"): + if strings.Contains(sql, "'s1'") { + return executor.Result{}, errors.New("second base unavailable") + } + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, vectorindex.CheckSumFromBuffer(buf), int64(len(buf)), 7)}}, nil + case strings.Contains(sql, "SELECT index_id"): + if mode == "partial-base" { + return executor.Result{Mp: mp, Batches: []*batch.Batch{twoIdBatch(mp, "seg0", "s1")}}, nil + } + return executor.Result{Mp: mp, Batches: []*batch.Batch{twoIdBatch(mp, "seg0", "seg0")}}, nil + default: + if mode == "cancel-after-base" { + cancel() + return executor.Result{}, context.Canceled + } + return executor.Result{}, nil + } + }) + s := newFulltext2SearchWithBaseOwner(cfg, owner) + t.Cleanup(s.Destroy) + err = s.Load(sp) + require.Error(t, err) + require.Nil(t, s.idx) + require.False(t, s.loaded) + require.NotNil(t, ptr) + require.Positive(t, p.deferredCount()) + require.ErrorIs(t, owner.close(), errBaseFileOwnerPending) + restore() + require.NoError(t, owner.close()) + requireDeferredSegmentCharge(t, p, 0) + require.Equal(t, 1, released[ptr]) + require.False(t, owner.hasResources()) + }) + } +} + +// Reflection keeps the exact regression source executable on the original +// candidate, which lacked this field; absence is zero rather than a PASS. +func deferredSegmentCharge(p *baseFilePool) int64 { + v := reflect.ValueOf(p).Elem().FieldByName("deferredSegmentBytes") + if !v.IsValid() { + return 0 + } + return v.Int() +} +func requireDeferredSegmentCharge(t *testing.T, p *baseFilePool, want int64) { + t.Helper() + require.Equal(t, want, deferredSegmentCharge(p)) +} + +// Hold the actual cleanup lock until cancellation is observed. Stack inspection +// establishes the stable wait phase; it is not the cancellation oracle. +func TestBaseFileCleanupAdmissionDoesNotDelayCancellation(t *testing.T) { + sp, _ := mockSqlProcWithIdentity(t, "cleanup-admission") + owner := newBaseFileOwner(1024, 2) + pool, err := owner.poolForSearch() + require.NoError(t, err) + entered, observed, rescue := make(chan struct{}), make(chan struct{}), make(chan struct{}) + var enteredOnce, observedOnce sync.Once + swapRunSql(t, func(proc *sqlexec.SqlProcess, _ string) (executor.Result, error) { + enteredOnce.Do(func() { close(entered) }) + select { + case <-proc.GetTopContext().Done(): + observedOnce.Do(func() { close(observed) }) + return executor.Result{}, context.Cause(proc.GetTopContext()) + case <-rescue: + return executor.Result{}, context.Canceled + } + }) + first := newFulltext2SearchWithBaseOwner(testStorageCfg(), owner) + second := newFulltext2SearchWithBaseOwner(testStorageCfg(), owner) + var workers sync.WaitGroup + firstDone, secondDone, cancelDone := make(chan error, 1), make(chan error, 1), make(chan struct{}) + locked := false + t.Cleanup(func() { + if locked { + pool.mu.Unlock() + } + close(rescue) + owner.cancelOperations() + workers.Wait() + first.Destroy() + second.Destroy() + require.NoError(t, owner.close()) + }) + workers.Add(1) + go func() { defer workers.Done(); firstDone <- first.Load(sp) }() + select { + case <-entered: + case <-time.After(5 * time.Second): + t.Fatal("first real Search.Load did not enter SQL") + } + pool.mu.Lock() + locked = true + workers.Add(1) + go func() { defer workers.Done(); secondDone <- second.Load(sp) }() + deadline := time.Now().Add(5 * time.Second) + for { + stack := make([]byte, 1<<20) + n := runtime.Stack(stack, true) + if strings.Contains(string(stack[:n]), "(*baseFilePool).mappingAdmissionError(") { + break + } + if time.Now().After(deadline) { + t.Fatal("second Search.Load did not reach controlled pool wait") + } + runtime.Gosched() + } + workers.Add(1) + go func() { defer workers.Done(); owner.cancelOperations(); close(cancelDone) }() + select { + case <-cancelDone: + case <-time.After(2 * time.Second): + t.Fatal("cancellation phase waited for pool cleanup mutex") + } + select { + case <-observed: + case <-time.After(2 * time.Second): + t.Fatal("admitted Search.Load did not observe owner cancellation") + } + pool.mu.Unlock() + locked = false + require.ErrorIs(t, <-firstDone, context.Canceled) + require.ErrorIs(t, <-secondDone, errBaseFileOwnerClosed) + workers.Wait() +} diff --git a/pkg/fulltext2/base_file_generation_finalization_test.go b/pkg/fulltext2/base_file_generation_finalization_test.go new file mode 100644 index 0000000000000..454eee9bf9083 --- /dev/null +++ b/pkg/fulltext2/base_file_generation_finalization_test.go @@ -0,0 +1,189 @@ +// Copyright 2026 Matrix Origin +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +//go:build (darwin || linux) && fulltext2_base_file_reuse + +package fulltext2 + +import ( + "runtime" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + veccache "github.com/matrixorigin/matrixone/pkg/vectorindex/cache" + "github.com/stretchr/testify/require" +) + +func TestExperimentalOwnerCloseFinalizationExcludesNewGeneration(t *testing.T) { + for _, first := range []string{"dedicated", "global"} { + t.Run(first, func(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + previousCache := veccache.Cache + c := veccache.NewVectorIndexCache() + veccache.Cache = c + t.Cleanup(func() { veccache.Cache = previousCache }) + const service = "generation-finalization" + oldToken, err := InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + experimentalOwnerState.Lock() + oldEntry := experimentalOwnerState.owners[service] + experimentalOwnerState.Unlock() + + atFinalization, secondValidated := make(chan struct{}), make(chan struct{}) + releaseFirst, releaseSecond := make(chan struct{}), make(chan struct{}) + var firstOnce, secondOnce sync.Once + resumeFirst := func() { firstOnce.Do(func() { close(releaseFirst) }) } + resumeSecond := func() { secondOnce.Do(func() { close(releaseSecond) }) } + previousCurrent, previousFinalize := experimentalOwnerCurrentBarrier, experimentalOwnerFinalizeBarrier + var validated, finalized atomic.Int32 + experimentalOwnerCurrentBarrier = func(entry *experimentalOwnerEntry) { + if entry == oldEntry && validated.Add(1) == 2 { + close(secondValidated) + <-releaseSecond + } + } + experimentalOwnerFinalizeBarrier = func(entry *experimentalOwnerEntry) { + if entry == oldEntry && finalized.Add(1) == 1 { + close(atFinalization) + <-releaseFirst + } + } + var workers sync.WaitGroup + var next *BaseFileReuseOwnerToken + // Independent rescue releases both controlled windows and joins every + // closer before restoring hooks/cache or freeing any live mapping. + t.Cleanup(func() { + resumeFirst() + resumeSecond() + workers.Wait() + experimentalOwnerCurrentBarrier, experimentalOwnerFinalizeBarrier = previousCurrent, previousFinalize + _ = CloseBaseFileReuseOwner(oldToken) + _ = CloseBaseFileReuseOwner(next) + c.Destroy() + }) + firstDone := make(chan error, 1) + workers.Add(1) + go func() { + defer workers.Done() + if first == "global" { + c.Destroy() + firstDone <- nil + } else { + firstDone <- CloseBaseFileReuseOwner(oldToken) + } + }() + select { + case <-atFinalization: + case <-time.After(5 * time.Second): + t.Fatal("first close did not reach registry finalization") + } + _, err = InitializeBaseFileReuseOwner(service) + require.ErrorIs(t, err, errExperimentalOwnerShutdown, "restart must reject unfinished finalization") + secondDone := make(chan error, 1) + workers.Add(1) + go func() { defer workers.Done(); secondDone <- CloseBaseFileReuseOwner(oldToken) }() + + // Old code validates the entry after A unlocked and before A finalized. + // Fixed code holds the entry mutex until finalization. A durable mutex + // stack only confirms B's phase; actual new cache/mapping survival is + // the oracle, not a scheduling delay or a fake owner identity. + crossed := false + deadline := time.Now().Add(5 * time.Second) + for { + select { + case <-secondValidated: + crossed = true + default: + } + if crossed || generationCloserWaitingOnMutex() { + break + } + if time.Now().After(deadline) { + t.Fatal("second closer reached neither current-check nor entry mutex") + } + runtime.Gosched() + } + resumeFirst() + require.NoError(t, <-firstDone) + if !crossed { + require.NoError(t, <-secondDone, "queued old closer must become a stale no-op") + } + require.False(t, oldToken.owner.hasResources()) + require.True(t, oldToken.owner.isClosing()) + next, err = InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + require.NotSame(t, oldToken.owner, next.owner) + require.NotSame(t, oldToken, next) + experimentalOwnerState.Lock() + newEntry := experimentalOwnerState.owners[service] + experimentalOwnerState.Unlock() + require.NotSame(t, oldEntry, newEntry) + + sp, cfg, _ := cleanupReworkSource(t, nil, "") + p, err := next.owner.poolForSearch() + require.NoError(t, err) + selectMapping, restore, released := cleanupReworkFault(t, next.owner) + restore() // observe real successful munmap, without injecting a fault + seg, err := loadFromStorageWithOwner(sp, cfg, "seg0", next.owner, p) + require.NoError(t, err) + selectMapping(seg.mmapData) + ptr := &seg.mmapData[0] + search := newFulltext2SearchWithBaseOwnerForService(cfg, next.owner, service) + search.idx, search.loaded = NewIndex([]*Segment{seg}, nil), true + entry := &veccache.VectorIndexSearch{Algo: search} + entry.Cond = sync.NewCond(entry.Mutex.RLocker()) + entry.Status.Store(veccache.STATUS_LOADED) + const key = "generation-finalization-entry" + c.IndexMap.Store(key, entry) + t.Cleanup(func() { resumeSecond(); workers.Wait(); c.Remove(key); seg.Free() }) + resumeSecond() + if crossed { + require.NoError(t, <-secondDone) + } + workers.Wait() + resident, present := c.IndexMap.Load(key) + require.True(t, present, "old current-entry closer destroyed the replacement generation") + require.Same(t, entry, resident) + require.Equal(t, int32(veccache.STATUS_LOADED), entry.Status.Load()) + require.NotNil(t, search.idx) + require.NotNil(t, seg.mmapData) + require.Zero(t, released[ptr], "old close must not unmap replacement Base") + require.True(t, next.owner.hasResources()) + require.False(t, next.owner.isClosing()) + experimentalOwnerState.Lock() + current, token := experimentalOwnerState.owners[service], experimentalOwnerState.owners[service].token + experimentalOwnerState.Unlock() + require.Same(t, newEntry, current) + require.Same(t, next, token) + require.NoError(t, CloseBaseFileReuseOwner(oldToken)) + require.False(t, next.owner.isClosing()) + require.NoError(t, CloseBaseFileReuseOwner(next)) + require.Nil(t, seg.mmapData) + require.Equal(t, 1, released[ptr]) + require.False(t, next.owner.hasResources()) + _, present = c.IndexMap.Load(key) + require.False(t, present) + experimentalOwnerState.Lock() + _, present = experimentalOwnerState.owners[service] + experimentalOwnerState.Unlock() + require.False(t, present) + t.Log("controlled finalization/restart window: registry, token, cache and real mapping terminal verified; workers joined") + }) + } +} + +func generationCloserWaitingOnMutex() bool { + stack := make([]byte, 1<<20) + n := runtime.Stack(stack, true) + for _, goroutine := range strings.Split(string(stack[:n]), "\n\n") { + if strings.Contains(goroutine, "closeExperimentalOwnerEntry(") && + (strings.Contains(goroutine, "internal/sync.(*Mutex).lockSlow(") || strings.Contains(goroutine, "sync.(*Mutex).lockSlow(")) { + return true + } + } + return false +} diff --git a/pkg/fulltext2/base_file_identity_test.go b/pkg/fulltext2/base_file_identity_test.go new file mode 100644 index 0000000000000..26c4e095c5951 --- /dev/null +++ b/pkg/fulltext2/base_file_identity_test.go @@ -0,0 +1,34 @@ +// Copyright 2026 Matrix Origin +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +package fulltext2 + +import "testing" + +func TestBaseFileKeySeparatesOwnerAccountAndIncarnation(t *testing.T) { + base := baseFileKey{ + owner: "cn-a", account: 7, db: "db", src: "src", index: "idx", + metadata: "meta", pkey: "id", id: "idx:100:0", checksum: "sum-a", size: 8, + } + if base == (baseFileKey{}) { + t.Fatal("test identity must not be empty") + } + for name, changed := range map[string]baseFileKey{ + "owner": func() baseFileKey { k := base; k.owner = "cn-b"; return k }(), + "account": func() baseFileKey { k := base; k.account = 8; return k }(), + "incarnation": func() baseFileKey { k := base; k.id = "idx:101:0"; return k }(), + "checksum": func() baseFileKey { k := base; k.checksum = "sum-b"; return k }(), + } { + if changed == base { + t.Fatalf("%s must not alias the same immutable file identity", name) + } + } + // Tail generation is deliberately absent from the key: it changes the + // surrounding Index snapshot, not the immutable Base bytes. + sameBase := base + if sameBase != base { + t.Fatal("same Base identity must remain reusable across Tail generations") + } +} diff --git a/pkg/fulltext2/base_file_load_bench_test.go b/pkg/fulltext2/base_file_load_bench_test.go new file mode 100644 index 0000000000000..06e52a07d16c6 --- /dev/null +++ b/pkg/fulltext2/base_file_load_bench_test.go @@ -0,0 +1,115 @@ +//go:build fulltext2_base_file_reuse + +// Copyright 2026 Matrix Origin +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +package fulltext2 + +import ( + "context" + "fmt" + "sync/atomic" + "testing" + + "github.com/matrixorigin/matrixone/pkg/common/mpool" + "github.com/matrixorigin/matrixone/pkg/container/batch" + "github.com/matrixorigin/matrixone/pkg/container/types" + "github.com/matrixorigin/matrixone/pkg/util/executor" + "github.com/matrixorigin/matrixone/pkg/vectorindex" + "github.com/matrixorigin/matrixone/pkg/vectorindex/sqlexec" +) + +// In-memory SQL producer, real spill/checksum/mmap/decode/Free. This isolates +// local materialization cost, NOT remote SQL latency or whole Search.Load CPU. +// Cold pool includes its creation, both existing decodes and final Close. +// Hit excludes only the explicitly prefilled file, never the new Segment decode. +func BenchmarkBaseFileLoadCost(b *testing.B) { + for _, docs := range []int{256, 10000, 50000} { + b.Run(fmt.Sprintf("docs=%d", docs), func(b *testing.B) { + builder := NewBuilder("bench", int32(types.T_int64)) + for d := 0; d < docs; d++ { + for term := 0; term < 8; term++ { + if err := builder.Add(fmt.Sprintf("term%03d", (d+term)%128), int32(term*8), int64(d)); err != nil { + b.Fatal(err) + } + } + } + source, err := builder.Finish() + if err != nil { + b.Fatal(err) + } + b.Cleanup(source.Free) + buf, err := source.Serialize() + if err != nil { + b.Fatal(err) + } + size, checksum := int64(len(buf)), vectorindex.CheckSumFromBuffer(buf) + sp := sqlexec.NewSqlProcessWithContext(sqlexec.NewSqlContext(context.Background(), "load-cost", nil, 0, nil)) + mp := mpool.MustNewZero() + b.Cleanup(func() { mpool.DeleteMPool(mp) }) + var streamed atomic.Int64 + oldSQL, oldContextSQL, oldStream := runSql, runSqlWithContext, runStreamingSql + b.Cleanup(func() { runSql, runSqlWithContext, runStreamingSql = oldSQL, oldContextSQL, oldStream }) + runSql = func(*sqlexec.SqlProcess, string) (executor.Result, error) { + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, checksum, size, 1)}}, nil + } + runSqlWithContext = func(_ context.Context, sp *sqlexec.SqlProcess, sql string) (executor.Result, error) { + return runSql(sp, sql) + } + runStreamingSql = func(_ context.Context, _ *sqlexec.SqlProcess, _ string, sc chan executor.Result, _ chan error) (executor.Result, error) { + streamed.Add(size) + sc <- executor.Result{Mp: mp, Batches: []*batch.Batch{chunkBatch(mp, buf)}} + return executor.Result{}, nil + } + for _, mode := range []string{"ordinary", "cold-pool", "hit", "capacity-fallback"} { + b.Run(mode, func(b *testing.B) { + var pool *baseFilePool + if mode == "hit" || mode == "capacity-fallback" { + budget := size + if mode == "capacity-fallback" { + budget-- + } + pool = newBaseFilePool(budget, 1) + b.Cleanup(pool.Close) + if mode == "hit" { + s, err := loadFromStorageWithPool(sp, testStorageCfg(), "bench", pool) + if err != nil { + b.Fatal(err) + } + s.Free() + } + } + streamed.Store(0) + b.ReportAllocs() + b.ResetTimer() + for i := 0; i < b.N; i++ { + var s *Segment + var err error + if mode == "ordinary" { + s, err = LoadFromStorage(sp, testStorageCfg(), "bench") + } else { + if mode == "cold-pool" { + pool = newBaseFilePool(size, 1) + } + s, err = loadFromStorageWithPool(sp, testStorageCfg(), "bench", pool) + } + if err != nil { + if pool != nil { + pool.Close() + } + b.Fatal(err) + } + s.Free() + if mode == "cold-pool" { + pool.Close() + } + } + b.StopTimer() + b.ReportMetric(float64(size), "base-bytes") + b.ReportMetric(float64(streamed.Load())/float64(b.N), "streamed-B/op") + }) + } + }) + } +} diff --git a/pkg/fulltext2/base_file_owner.go b/pkg/fulltext2/base_file_owner.go new file mode 100644 index 0000000000000..f66d4b5237577 --- /dev/null +++ b/pkg/fulltext2/base_file_owner.go @@ -0,0 +1,218 @@ +// Copyright 2026 Matrix Origin +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// http://www.apache.org/licenses/LICENSE-2.0 + +package fulltext2 + +import ( + "context" + "errors" + "sync" +) + +var ( + errBaseFileOwnerClosed = errors.New("fulltext2 base file owner is closed") + errBaseFileOwnerPending = errors.New("fulltext2 base file owner still owns resources") +) + +// baseFileOwner is the lifetime owner for an experiment-scoped pool. Search +// handles borrow it; they never close it. One owner is intended to correspond +// to one CN service runtime, so deferred mappings and pinned files remain +// reachable even when a cache entry drops its Search handle. +type baseFileOwner struct { + mu sync.Mutex + closeMu sync.Mutex + cond *sync.Cond + pool *baseFilePool + active int + closing bool + nextOp uint64 + operations map[uint64]context.CancelFunc +} + +func newBaseFileOwner(maxBytes int64, maxFiles int) *baseFileOwner { + o := &baseFileOwner{pool: newBaseFilePool(maxBytes, maxFiles), operations: make(map[uint64]context.CancelFunc)} + o.cond = sync.NewCond(&o.mu) + return o +} + +func (o *baseFileOwner) poolForSearch() (*baseFilePool, error) { + if o == nil { + return nil, errBaseFileOwnerClosed + } + o.mu.Lock() + defer o.mu.Unlock() + if o.closing || o.pool == nil { + return nil, errBaseFileOwnerClosed + } + return o.pool, nil +} + +func (o *baseFileOwner) isClosing() bool { + if o == nil { + return true + } + o.mu.Lock() + defer o.mu.Unlock() + return o.closing || o.pool == nil +} + +// cancelOperations closes admission and cancels complete operations without +// waiting for cache entries, active work, or pool/OS cleanup. Global shutdown +// must cancel every owner before starting any of those destructive waits. +func (o *baseFileOwner) cancelOperations() { + if o == nil { + return + } + o.mu.Lock() + o.closing = true + cancels := make([]context.CancelFunc, 0, len(o.operations)) + for _, cancel := range o.operations { + cancels = append(cancels, cancel) + } + o.mu.Unlock() + for _, cancel := range cancels { + cancel() + } +} + +// beginClose preserves the dedicated service-close pool-fill cancellation +// contract, in addition to closing complete-operation admission before drain. +func (o *baseFileOwner) beginClose() { + if o == nil { + return + } + o.cancelOperations() + o.mu.Lock() + p := o.pool + o.mu.Unlock() + if p != nil { + p.Close() + } +} + +// run admits one complete storage operation, including an ordinary-loader +// fallback. Close waits for this count to drain, so a fallback cannot begin +// after shutdown has passed its admission gate. +func (o *baseFileOwner) run(ctx context.Context, fn func(context.Context) (*Segment, error)) (*Segment, error) { + var seg *Segment + err := o.runOperation(ctx, func(ctx context.Context) error { + var err error + seg, err = fn(ctx) + return err + }) + return seg, err +} + +// runOperation covers the entire search load, including reads before/after Base +// materialization. Close cancels and waits for these operations before draining +// cached search objects. Nested Base operations retain their standalone contract. +func (o *baseFileOwner) runOperation(ctx context.Context, fn func(context.Context) error) error { + if o == nil { + return errBaseFileOwnerClosed + } + if ctx == nil { + ctx = context.Background() + } + o.mu.Lock() + if o.closing || o.pool == nil { + o.mu.Unlock() + return errBaseFileOwnerClosed + } + p := o.pool + o.mu.Unlock() + // Pool admission can wait for retained-file cleanup. Never carry the + // owner lock through that wait: cancelOperations must issue cancellation + // before any pool/OS cleanup wait. Recheck ownership after the pool gate. + if err := p.mappingAdmissionError(); err != nil { + return err + } + o.mu.Lock() + if o.closing || o.pool != p { + o.mu.Unlock() + return errBaseFileOwnerClosed + } + opCtx, cancel := context.WithCancel(ctx) + o.nextOp++ + opID := o.nextOp + o.operations[opID] = cancel + o.active++ + o.mu.Unlock() + defer func() { + o.mu.Lock() + delete(o.operations, opID) + o.active-- + if o.active == 0 { + o.cond.Broadcast() + } + o.mu.Unlock() + }() + defer cancel() + select { + case <-opCtx.Done(): + return context.Cause(opCtx) + default: + } + return fn(opCtx) +} + +func (o *baseFileOwner) hasResources() bool { + if o == nil { + return false + } + o.mu.Lock() + p := o.pool + o.mu.Unlock() + if p == nil { + return false + } + p.mu.Lock() + defer p.mu.Unlock() + return p.bytes != 0 || p.reserved != 0 || p.reservedFiles != 0 || p.deferredBytes != 0 || + p.openFiles != 0 || p.deferredSegmentBytes != 0 || len(p.entries) != 0 || + len(p.deferredSegments) != 0 || len(p.deferredMappings) != 0 || + p.liveFallbacks != 0 || len(p.deferredFiles) != 0 || p.deferredFileBytes != 0 +} + +// close is idempotent. A close that cannot finish returns a typed pending +// error; a later close after Search handles release their leases retries the +// same owner rather than losing the only cleanup reference. Ordinary mappings +// retain a lifetime pin through successful Free, including cache eviction +// already removed from the map but not yet inside Destroy. +func (o *baseFileOwner) close() error { + if o == nil { + return nil + } + o.closeMu.Lock() + defer o.closeMu.Unlock() + // close may be called directly by a service owner, without the cache + // drain's separate beginClose step. Publish the same admission barrier and + // cancel every already-admitted operation before waiting for active work; + // otherwise an ordinary-loader fallback can remain blocked in source I/O + // and make owner shutdown wait forever. + o.beginClose() + o.mu.Lock() + if o.pool == nil { + o.mu.Unlock() + return nil + } + p := o.pool + o.mu.Unlock() + + o.mu.Lock() + for o.active != 0 { + o.cond.Wait() + } + o.mu.Unlock() + p.retryDeferred() + if o.hasResources() { + return errBaseFileOwnerPending + } + o.mu.Lock() + o.pool = nil + o.mu.Unlock() + return nil +} diff --git a/pkg/fulltext2/base_file_owner_test.go b/pkg/fulltext2/base_file_owner_test.go new file mode 100644 index 0000000000000..d760966f0081b --- /dev/null +++ b/pkg/fulltext2/base_file_owner_test.go @@ -0,0 +1,98 @@ +// Copyright 2026 Matrix Origin +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +package fulltext2 + +import ( + "context" + "sync/atomic" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestBaseFileOwnerSurvivesBorrowerRelease(t *testing.T) { + owner := newBaseFileOwner(64, 2) + t.Cleanup(func() { _ = owner.close() }) + pool, err := owner.poolForSearch() + require.NoError(t, err) + var fills atomic.Int32 + key := testPoolKey("owner", 8, "abcdefgh") + borrow := func() error { + _, err := owner.run(context.Background(), func(context.Context) (*Segment, error) { + lease, err := pool.acquire(context.Background(), key, testPoolFill(t, &fills, "abcdefgh")) + if err != nil { + return nil, err + } + lease.Release() + return nil, nil + }) + return err + } + require.NoError(t, borrow()) + require.NoError(t, borrow(), "a second Search handle may borrow the same owner") + require.Equal(t, int32(1), fills.Load()) + require.NoError(t, owner.close()) + _, err = owner.poolForSearch() + require.ErrorIs(t, err, errBaseFileOwnerClosed) +} + +func TestBaseFileOwnerCloseWaitsForAdmittedOperation(t *testing.T) { + owner := newBaseFileOwner(64, 2) + started := make(chan struct{}) + release := make(chan struct{}) + t.Cleanup(func() { + select { + case <-release: + default: + close(release) + } + _ = owner.close() + }) + result := make(chan error, 1) + go func() { + _, err := owner.run(context.Background(), func(context.Context) (*Segment, error) { + close(started) + <-release + return nil, nil + }) + result <- err + }() + <-started + closed := make(chan error, 1) + go func() { closed <- owner.close() }() + select { + case err := <-closed: + t.Fatalf("owner close returned before admitted operation ended: %v", err) + default: + } + close(release) + require.NoError(t, <-result) + require.NoError(t, <-closed) + _, err := owner.poolForSearch() + require.ErrorIs(t, err, errBaseFileOwnerClosed) +} + +func TestBaseFileOwnerRejectsOperationAfterClose(t *testing.T) { + owner := newBaseFileOwner(64, 2) + t.Cleanup(func() { _ = owner.close() }) + require.NoError(t, owner.close()) + _, err := owner.run(context.Background(), func(context.Context) (*Segment, error) { + t.Fatal("closed owner must not invoke an operation") + return nil, nil + }) + require.ErrorIs(t, err, errBaseFileOwnerClosed) +} + +func TestBaseFileOwnerReportsPinnedResourcesUntilRelease(t *testing.T) { + owner := newBaseFileOwner(64, 1) + pool, err := owner.poolForSearch() + require.NoError(t, err) + lease, err := pool.acquire(context.Background(), testPoolKey("pinned-owner", 8, "abcdefgh"), testPoolFill(t, new(atomic.Int32), "abcdefgh")) + require.NoError(t, err) + require.ErrorIs(t, owner.close(), errBaseFileOwnerPending) + lease.Release() + require.NoError(t, owner.close()) +} diff --git a/pkg/fulltext2/base_file_pool.go b/pkg/fulltext2/base_file_pool.go new file mode 100644 index 0000000000000..ad65b304bd46d --- /dev/null +++ b/pkg/fulltext2/base_file_pool.go @@ -0,0 +1,692 @@ +// Copyright 2026 Matrix Origin +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package fulltext2 + +import ( + "context" + "crypto/md5" + "encoding/hex" + "errors" + "fmt" + "io" + "os" + "sync" +) + +var errBaseFilePoolCapacity = errors.New("fulltext2 base file pool capacity admission failed") + +// These sentinels classify failures at the optional-file layer. Callers must +// use errors.Is instead of matching messages: capacity and a recoverable cache +// failure may fall back to the ordinary loader, while a closed owner or a +// source-side error must be returned to the query unchanged. +var ( + errBaseFileCleanupPending = errors.New("fulltext2 base cleanup is pending") + errBaseFilePoolClosed = errors.New("fulltext2 base file pool is closed") + errBaseFilePoolLeaderCanceled = errors.New("fulltext2 base file fill leader canceled") + errBaseFilePoolReadyCorrupt = errors.New("fulltext2 ready base file is corrupt") +) + +type baseFilePoolError struct { + kind error + cause error +} + +func (e *baseFilePoolError) Error() string { + if e == nil { + return "fulltext2 base file pool error" + } + if e.cause == nil { + return e.kind.Error() + } + return fmt.Sprintf("%s: %v", e.kind, e.cause) +} + +func (e *baseFilePoolError) Unwrap() error { return e.cause } + +func (e *baseFilePoolError) Is(target error) bool { + return e != nil && (target == e.kind || errors.Is(e.cause, target)) +} + +func wrapBaseFilePoolError(kind, cause error) error { + if cause == nil { + return kind + } + return &baseFilePoolError{kind: kind, cause: cause} +} + +// baseFileKey identifies one immutable Base file. The metadata identity is +// deliberately part of the key: a checksum is an integrity check, not an +// identity, and must not by itself deduplicate files from different indexes. +// The owner and account are captured from the loading CN when available; the +// database/table identity remains part of the key for deterministic tests. +type baseFileKey struct { + owner string + account uint32 + db string + src string + index string + metadata string + pkey string + id string + checksum string + size int64 +} + +type baseFileState uint8 + +const ( + baseFileFilling baseFileState = iota + 1 + baseFileReady +) + +type baseFileEntry struct { + key baseFileKey + handle baseFileHandle + state baseFileState + users int + waiters int + err error + retired bool + fileGone bool + lastUsed uint64 + done chan struct{} + mapMu sync.Mutex // protects fallback mmapReadOnly implementations that seek the shared fd + cancel context.CancelFunc +} + +type deferredBaseMapping struct { + data []byte + size int64 +} + +type baseFileHandle struct { + file *os.File + path string + // validationData remains owned by the fill until it is successfully unmapped. + // It is only non-nil on the exceptional validation-release failure path. + validationData []byte + // validated is a one-shot proof produced by the fill transaction. It avoids + // hashing the same bytes twice for the first lease; later READY hits always + // re-check the complete file before mapping. + validated bool +} + +// baseFilePool is a deliberately small, request-independent pool of immutable +// Base files. It owns files, not decoded Segments. The mutex protects +// the directory, byte/FD accounting and lease state; SQL, checksum and mmap +// work happens outside it. Close/remove retain the existing bounded mutex +// boundary; cancellation never holds the owner mutex while waiting here. +type baseFilePool struct { + mu sync.Mutex + retryMu sync.Mutex + maxBytes int64 + maxFiles int + bytes int64 + reserved int64 + // deferredBytes accounts for validation mappings whose munmap failed after + // their fill reservation was released. They remain charged until retry succeeds. + deferredBytes int64 + openFiles int + // reservedFiles remains charged from FILLING admission until the fill's + // handle has actually been closed. In particular, a failed fill must not + // open a second FD while its first handle is still in closeHandle. + reservedFiles int + seq uint64 + closed bool + entries map[baseFileKey]*baseFileEntry + deferredSegmentBytes int64 + deferredSegments map[*Segment]int64 + deferredMappings map[*deferredBaseMapping]struct{} + // Live ordinary mappings pin the cleanup owner even after cache removal. + liveFallbacks int + // Linked paths whose FD is already closed retain disk debt, never FD debt. + deferredFiles map[string]int64 + deferredFileBytes int64 +} + +func newBaseFilePool(maxBytes int64, maxFiles int) *baseFilePool { + if maxBytes <= 0 { + maxBytes = 1 + } + if maxFiles <= 0 { + maxFiles = 1 + } + return &baseFilePool{ + maxBytes: maxBytes, + maxFiles: maxFiles, + entries: make(map[baseFileKey]*baseFileEntry), + deferredSegments: make(map[*Segment]int64), + deferredMappings: make(map[*deferredBaseMapping]struct{}), + deferredFiles: make(map[string]int64), + } +} + +// baseFileLease pins one pool file. A lease has at most one mapping; every +// loaded Segment gets its own mapping and its own lease. +type baseFileLease struct { + pool *baseFilePool + entry *baseFileEntry + once sync.Once + mu sync.Mutex + data []byte + err error + mapped bool + released bool +} + +func (p *baseFilePool) acquire(ctx context.Context, key baseFileKey, fill func(context.Context) (*baseFileHandle, error)) (*baseFileLease, error) { + if ctx == nil { + ctx = context.Background() + } + for { + p.mu.Lock() + if p.closed { + p.mu.Unlock() + return nil, errBaseFilePoolClosed + } + if len(p.deferredSegments) != 0 || len(p.deferredFiles) != 0 { + p.mu.Unlock() + return nil, errBaseFileCleanupPending + } + if e := p.entries[key]; e != nil { + if e.state == baseFileReady { + p.seq++ + e.lastUsed = p.seq + e.users++ + p.mu.Unlock() + return &baseFileLease{pool: p, entry: e}, nil + } + done := e.done + e.waiters++ + p.mu.Unlock() + select { + case <-done: + p.mu.Lock() + if e.waiters > 0 { + e.waiters-- + } + fillErr := e.err + p.mu.Unlock() + if fillErr != nil { + return nil, fillErr + } + continue + case <-ctx.Done(): + p.mu.Lock() + if e.waiters > 0 { + e.waiters-- + } + p.mu.Unlock() + return nil, ctx.Err() + } + } + + if key.size <= 0 || key.size > p.maxBytes { + p.mu.Unlock() + return nil, fmt.Errorf("%w: base file %s exceeds pool capacity", errBaseFilePoolCapacity, key.id) + } + if err := p.makeRoomLocked(key.size); err != nil { + p.mu.Unlock() + return nil, err + } + e := &baseFileEntry{ + key: key, + state: baseFileFilling, + done: make(chan struct{}), + } + fillCtx, cancel := context.WithCancel(ctx) + e.cancel = cancel + p.entries[key] = e + p.reserved += key.size + p.reservedFiles++ + p.mu.Unlock() + + handle, err := fill(fillCtx) + callerErr := ctx.Err() + cancel() + if err != nil { + err = p.classifyFillError(ctx, err) + p.finishFill(e, handle, err) + return nil, err + } + if handle == nil || handle.file == nil { + err = fmt.Errorf("fulltext2 base file %s fill returned nil file", key.id) + p.finishFill(e, nil, err) + return nil, err + } + if callerErr != nil { + err = wrapBaseFilePoolError(errBaseFilePoolLeaderCanceled, callerErr) + p.finishFill(e, handle, err) + return nil, err + } + p.mu.Lock() + closed := p.closed + if !closed { + e.handle = *handle + e.state = baseFileReady + e.users = 1 + p.seq++ + e.lastUsed = p.seq + p.bytes += key.size + p.openFiles++ + p.reserved -= key.size + p.reservedFiles-- + } + if closed { + e.err = errBaseFilePoolClosed + } + close(e.done) + delete(p.entries, key) + if !closed { + p.entries[key] = e + } + p.mu.Unlock() + if closed { + p.closeHandle(handle, key.size) + p.mu.Lock() + p.reserved -= key.size + p.reservedFiles-- + p.mu.Unlock() + return nil, e.err + } + return &baseFileLease{pool: p, entry: e}, nil + } +} + +func (p *baseFilePool) classifyFillError(ctx context.Context, err error) error { + if err == nil { + return nil + } + p.mu.Lock() + closed := p.closed + p.mu.Unlock() + if closed { + return wrapBaseFilePoolError(errBaseFilePoolClosed, err) + } + // A fill context is derived from the leader's context. Only that leader's + // cancellation is eligible for waiter-side ordinary-loader fallback; source + // errors that happen to wrap context.Canceled are preserved as source errors. + if ctx != nil && ctx.Err() != nil && + (errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded)) { + return wrapBaseFilePoolError(errBaseFilePoolLeaderCanceled, err) + } + return err +} + +func (p *baseFilePool) finishFill(e *baseFileEntry, handle *baseFileHandle, err error) { + p.mu.Lock() + e.err = err + if p.entries[e.key] == e { + delete(p.entries, e.key) + close(e.done) + } + p.mu.Unlock() + p.closeHandle(handle, e.key.size) + // Keep both byte and FD reservations until closeHandle has returned. A + // failing fill may already own an open file even though its directory entry + // is gone, so releasing either budget earlier admits an actual overage. + p.mu.Lock() + p.reserved -= e.key.size + p.reservedFiles-- + p.mu.Unlock() + _ = err +} + +func (p *baseFilePool) makeRoomLocked(need int64) error { + for p.bytes+p.reserved+p.deferredBytes+p.deferredFileBytes+need > p.maxBytes || p.fileCountLocked()+1 > p.maxFiles { + var victim *baseFileEntry + for _, e := range p.entries { + if e.state != baseFileReady || e.users != 0 { + continue + } + if victim == nil || e.lastUsed < victim.lastUsed { + victim = e + } + } + if victim == nil { + return errBaseFilePoolCapacity + } + delete(p.entries, victim.key) + victim.retired = true + p.closeEntryLocked(victim) + if len(p.deferredFiles) != 0 { + return errBaseFileCleanupPending + } + } + return nil +} + +func (p *baseFilePool) closeEntryLocked(e *baseFileEntry) { + if e == nil || e.fileGone { + return + } + e.fileGone = true + p.bytes -= e.key.size + if e.state == baseFileReady && p.openFiles > 0 { + p.openFiles-- + } + if e.handle.validationData != nil { + p.deferMappingLocked(e.handle.validationData) + e.handle.validationData = nil + } + _ = e.handle.file.Close() + p.removeFileLocked(e.handle.path, e.key.size) +} + +func (p *baseFilePool) retire(e *baseFileEntry) { + if e == nil { + return + } + p.mu.Lock() + if !e.retired { + e.retired = true + if p.entries[e.key] == e { + delete(p.entries, e.key) + } + } + if e.users == 0 { + p.closeEntryLocked(e) + } + p.mu.Unlock() +} + +func (p *baseFilePool) fileCountLocked() int { + return p.openFiles + p.reservedFiles +} + +// mappingAdmissionError quarantines new experiment mappings while cleanup +// retains failed Segment or linked-file resources. Capacity fallback +// must not bypass this error. Existing admitted work can finish and transfer its +// finite resources; cleanup success reopens admission without a worker or timer. +func (p *baseFilePool) mappingAdmissionError() error { + if p == nil { + return nil + } + p.mu.Lock() + defer p.mu.Unlock() + if p.closed { + return errBaseFilePoolClosed + } + if len(p.deferredSegments) != 0 || len(p.deferredFiles) != 0 { + return errBaseFileCleanupPending + } + return nil +} + +func (p *baseFilePool) deferSegment(s *Segment) { + if p == nil || s == nil { + return + } + p.mu.Lock() + // Preserve the same mapping + per-doc charge used by cache admission + // after the cache entry has been removed. Repeated failed retries must + // neither duplicate the charge nor permit new READY/fallback mappings. + if _, exists := p.deferredSegments[s]; !exists { + mappingBytes := int64(len(s.mmapData)) + if s.mmapPath != "" { + mappingBytes = max(mappingBytes, s.mmapFileSize) + } + charge := mappingBytes + max(s.N, 0)*estBytesPerDocHeap + p.deferredSegments[s] = charge + p.deferredSegmentBytes += charge + } + p.mu.Unlock() +} + +func (p *baseFilePool) undeferSegment(s *Segment) { + if p == nil || s == nil { + return + } + p.mu.Lock() + if charge, exists := p.deferredSegments[s]; exists { + delete(p.deferredSegments, s) + p.deferredSegmentBytes -= charge + } + p.mu.Unlock() +} + +func (p *baseFilePool) deferMappingLocked(data []byte) { + size := int64(len(data)) + if size <= 0 { + return + } + p.deferredMappings[&deferredBaseMapping{data: data, size: size}] = struct{}{} + p.deferredBytes += size +} + +// baseFilePoolBeforeCloseHandle is test-only synchronization for the failure +// accounting tests. Production leaves it nil; the hook runs before any mmap or +// FD is released so a second acquisition can observe the reservation. +var baseFilePoolBeforeCloseHandle func() + +func (p *baseFilePool) closeHandle(handle *baseFileHandle, size int64) { + if handle == nil { + return + } + if hook := baseFilePoolBeforeCloseHandle; hook != nil { + hook() + } + if handle.validationData != nil { + if err := munmap(handle.validationData); err != nil { + p.mu.Lock() + p.deferMappingLocked(handle.validationData) + p.mu.Unlock() + } + handle.validationData = nil + } + if handle.file != nil { + _ = handle.file.Close() + } + p.mu.Lock() + p.removeFileLocked(handle.path, size) + p.mu.Unlock() +} + +// removeFileLocked transfers failed unlink to retry ownership before the caller +// releases its byte reservation. The descriptor has already been closed. +func (p *baseFilePool) removeFileLocked(path string, size int64) { + if path == "" { + return + } + if err := os.Remove(path); err != nil && !os.IsNotExist(err) { + if _, exists := p.deferredFiles[path]; !exists { + p.deferredFiles[path] = size + p.deferredFileBytes += size + } + } +} + +func (p *baseFilePool) pinFallback() { + p.mu.Lock() + p.liveFallbacks++ + p.mu.Unlock() +} + +func (p *baseFilePool) releaseFallback() { + p.mu.Lock() + p.liveFallbacks-- + p.mu.Unlock() +} + +// retryDeferred retries mappings, Segment frees and linked paths outside the +// pool mutex. Failed cleanup remains owned for the next Destroy/Close boundary. +func (p *baseFilePool) retryDeferred() { + if p == nil { + return + } + p.retryMu.Lock() + defer p.retryMu.Unlock() + p.mu.Lock() + segments := make([]*Segment, 0, len(p.deferredSegments)) + for s := range p.deferredSegments { + segments = append(segments, s) + } + mappings := make([]*deferredBaseMapping, 0, len(p.deferredMappings)) + for m := range p.deferredMappings { + mappings = append(mappings, m) + } + files := make(map[string]int64, len(p.deferredFiles)) + for path, size := range p.deferredFiles { + files[path] = size + } + p.mu.Unlock() + + for _, s := range segments { + s.Free() + } + for _, m := range mappings { + if err := munmap(m.data); err == nil { + p.mu.Lock() + if _, ok := p.deferredMappings[m]; ok { + delete(p.deferredMappings, m) + p.deferredBytes -= m.size + if p.deferredBytes < 0 { + p.deferredBytes = 0 + } + } + p.mu.Unlock() + } + } + for path, size := range files { + if err := os.Remove(path); err == nil || os.IsNotExist(err) { + p.mu.Lock() + if _, exists := p.deferredFiles[path]; exists { + delete(p.deferredFiles, path) + p.deferredFileBytes -= size + } + p.mu.Unlock() + } + } +} + +func (p *baseFilePool) deferredCount() int { + p.mu.Lock() + defer p.mu.Unlock() + return len(p.deferredSegments) + len(p.deferredMappings) + len(p.deferredFiles) +} + +func (l *baseFileLease) MapReadOnly() ([]byte, error) { + if l == nil || l.pool == nil || l.entry == nil { + return nil, fmt.Errorf("fulltext2 nil base file lease") + } + l.mu.Lock() + defer l.mu.Unlock() + if l.released { + return nil, fmt.Errorf("fulltext2 base file lease already released") + } + if l.mapped { + return l.data, l.err + } + if err := l.pool.mappingAdmissionError(); err != nil { + return nil, err + } + l.mapped = true + l.entry.mapMu.Lock() + if l.entry.handle.validated { + // The fill path already checksummed and decoded this immutable file. Consume + // that proof for one first mapping; later READY hits re-check the complete + // file before mapping so a damaged cached file is retired. + l.entry.handle.validated = false + l.data, l.err = mmapReadOnly(l.entry.handle.file) + } else { + checksum, checksumErr := checksumFile(l.entry.handle.file, l.entry.key.size) + if checksumErr != nil { + l.err = wrapBaseFilePoolError(errBaseFilePoolReadyCorrupt, checksumErr) + } else if checksum != l.entry.key.checksum { + l.err = wrapBaseFilePoolError(errBaseFilePoolReadyCorrupt, + fmt.Errorf("fulltext2 base file %s checksum mismatch", l.entry.key.id)) + } else { + l.data, l.err = mmapReadOnly(l.entry.handle.file) + } + } + l.entry.mapMu.Unlock() + if l.err != nil { + l.pool.retire(l.entry) + } + return l.data, l.err +} + +func checksumFile(f *os.File, size int64) (string, error) { + if f == nil || size < 0 { + return "", fmt.Errorf("fulltext2 invalid base file for checksum") + } + info, err := f.Stat() + if err != nil { + return "", err + } + if info.Size() != size { + return "", fmt.Errorf("fulltext2 base file size mismatch: got %d want %d", info.Size(), size) + } + h := md5.New() + if _, err := io.CopyN(h, io.NewSectionReader(f, 0, size), size); err != nil { + return "", err + } + return hex.EncodeToString(h.Sum(nil)), nil +} + +func (l *baseFileLease) Invalidate() { + if l == nil { + return + } + l.pool.retire(l.entry) +} + +func (l *baseFileLease) Release() { + if l == nil { + return + } + l.once.Do(func() { + l.mu.Lock() + l.released = true + l.mu.Unlock() + p := l.pool + p.mu.Lock() + if l.entry.users > 0 { + l.entry.users-- + } + if l.entry.users == 0 && (p.closed || l.entry.retired) { + if p.entries[l.entry.key] == l.entry { + delete(p.entries, l.entry.key) + } + p.closeEntryLocked(l.entry) + } + p.mu.Unlock() + }) +} + +// Close rejects new acquisitions and closes all unpinned files. Pinned files +// stay alive until their Segment releases its lease. +func (p *baseFilePool) Close() { + p.retryDeferred() + p.mu.Lock() + if p.closed { + p.mu.Unlock() + return + } + p.closed = true + for key, e := range p.entries { + if e.state == baseFileFilling && e.cancel != nil { + e.cancel() + } + if e.state == baseFileReady && e.users == 0 { + delete(p.entries, key) + e.retired = true + p.closeEntryLocked(e) + } else if e.state == baseFileReady { + e.retired = true + } + } + p.mu.Unlock() +} diff --git a/pkg/fulltext2/base_file_pool_test.go b/pkg/fulltext2/base_file_pool_test.go new file mode 100644 index 0000000000000..ba9f1f3601dc6 --- /dev/null +++ b/pkg/fulltext2/base_file_pool_test.go @@ -0,0 +1,476 @@ +// Copyright 2026 Matrix Origin +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +package fulltext2 + +import ( + "bytes" + "context" + "crypto/md5" + "encoding/hex" + "errors" + "fmt" + "os" + "runtime" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/stretchr/testify/require" +) + +func testPoolKey(id string, size int64, value string) baseFileKey { + sum := md5.Sum([]byte(value)) + return baseFileKey{db: "db", index: "idx", metadata: "meta", id: id, checksum: hex.EncodeToString(sum[:]), size: size} +} + +func testPoolFill(t *testing.T, fills *atomic.Int32, value string) func(context.Context) (*baseFileHandle, error) { + t.Helper() + dir := t.TempDir() + return func(context.Context) (*baseFileHandle, error) { + fills.Add(1) + f, err := os.CreateTemp(dir, "ft2-pool") + if err != nil { + return nil, err + } + if _, err = f.WriteString(value); err != nil { + _ = f.Close() + return nil, err + } + if err = f.Sync(); err != nil { + _ = f.Close() + return nil, err + } + if _, err = f.Seek(0, 0); err != nil { + _ = f.Close() + return nil, err + } + return &baseFileHandle{file: f, path: f.Name()}, nil + } +} + +func TestBaseFilePoolConcurrentFillAndIndependentMaps(t *testing.T) { + pool := newBaseFilePool(64, 2) + defer pool.Close() + var fills atomic.Int32 + key := testPoolKey("s0", 8, "abcdefgh") + fill := testPoolFill(t, &fills, "abcdefgh") + + const n = 8 + start := make(chan struct{}) + results := make(chan error, n) + var wg sync.WaitGroup + for i := 0; i < n; i++ { + wg.Add(1) + go func() { + defer wg.Done() + <-start + lease, err := pool.acquire(context.Background(), key, fill) + if err != nil { + results <- err + return + } + data, err := lease.MapReadOnly() + // Use the real failed-munmap ownership contract even when a worker + // fails: Free retains the Segment in the pool until cleanup succeeds. + seg := &Segment{mmapData: data, mmapRelease: lease.Release, mmapRetryPool: pool} + defer seg.Free() + if err == nil && !bytes.Equal([]byte("abcdefgh"), data) { + err = fmt.Errorf("unexpected mapping contents: %q", data) + } + results <- err + }() + } + close(start) + wg.Wait() + close(results) + for err := range results { + require.NoError(t, err) + } + require.Zero(t, pool.deferredCount()) + require.Equal(t, int32(1), fills.Load(), "one FILLING owner must publish one immutable file") + + lease, err := pool.acquire(context.Background(), key, func(context.Context) (*baseFileHandle, error) { + t.Fatal("a READY hit must not refill") + return nil, nil + }) + require.NoError(t, err) + data, err := lease.MapReadOnly() + require.NoError(t, err) + require.Equal(t, []byte("abcdefgh"), data) + require.NoError(t, munmap(data)) + lease.Release() +} + +func TestBaseFilePoolCapacityEvictsOnlyIdleFiles(t *testing.T) { + pool := newBaseFilePool(8, 1) + defer pool.Close() + var fills atomic.Int32 + + first, err := pool.acquire(context.Background(), testPoolKey("s0", 8, "12345678"), testPoolFill(t, &fills, "12345678")) + require.NoError(t, err) + firstData, err := first.MapReadOnly() + require.NoError(t, err) + require.NoError(t, munmap(firstData)) + firstPath := first.entry.handle.path + first.Release() + require.FileExists(t, firstPath) + + second, err := pool.acquire(context.Background(), testPoolKey("s1", 8, "abcdefgh"), testPoolFill(t, &fills, "abcdefgh")) + require.NoError(t, err) + require.NoError(t, munmap(mustMapLease(t, second))) + require.NoFileExists(t, firstPath, "only an idle READY file may be evicted") + second.Release() + require.Equal(t, int32(2), fills.Load()) +} + +func TestBaseFilePoolPinnedCapacityIsAnOptimizationMiss(t *testing.T) { + pool := newBaseFilePool(16, 1) + defer pool.Close() + var fills atomic.Int32 + pinned, err := pool.acquire(context.Background(), testPoolKey("pinned", 8, "12345678"), testPoolFill(t, &fills, "12345678")) + require.NoError(t, err) + defer pinned.Release() + + _, err = pool.acquire(context.Background(), testPoolKey("blocked", 8, "abcdefgh"), testPoolFill(t, &fills, "abcdefgh")) + require.ErrorIs(t, err, errBaseFilePoolCapacity) + require.Equal(t, int32(1), fills.Load(), "a pinned READY file must not be evicted to admit another pooled file") +} + +func TestBaseFilePoolRetiredPinnedFileStillConsumesFDQuota(t *testing.T) { + pool := newBaseFilePool(16, 1) + defer pool.Close() + var fills atomic.Int32 + key := testPoolKey("retired", 8, "12345678") + first, err := pool.acquire(context.Background(), key, testPoolFill(t, &fills, "12345678")) + require.NoError(t, err) + data, err := first.MapReadOnly() + require.NoError(t, err) + pooledFile := first.entry.handle.file + _, err = pooledFile.WriteAt([]byte{'X'}, 0) + require.NoError(t, err) + + var unexpectedRefill atomic.Bool + second, err := pool.acquire(context.Background(), key, func(context.Context) (*baseFileHandle, error) { + unexpectedRefill.Store(true) + return nil, errors.New("unexpected corrupt-hit refill") + }) + require.NoError(t, err) + _, err = second.MapReadOnly() + require.ErrorContains(t, err, "checksum mismatch") + second.Release() + require.False(t, unexpectedRefill.Load()) + + _, err = pool.acquire(context.Background(), testPoolKey("next", 8, "abcdefgh"), testPoolFill(t, &fills, "abcdefgh")) + require.ErrorIs(t, err, errBaseFilePoolCapacity, "the retired file remains an open FD while first lease is pinned") + require.Equal(t, int32(1), fills.Load()) + + require.NoError(t, munmap(data)) + first.Release() + next, err := pool.acquire(context.Background(), testPoolKey("next", 8, "abcdefgh"), testPoolFill(t, &fills, "abcdefgh")) + require.NoError(t, err, "new FD admission becomes possible after the retired lease closes") + require.NoError(t, munmap(mustMapLease(t, next))) + next.Release() + require.Equal(t, int32(2), fills.Load()) +} + +func TestBaseFilePoolFailedFillKeepsFDReservationUntilClose(t *testing.T) { + pool := newBaseFilePool(16, 1) + defer pool.Close() + + closeEntered := make(chan struct{}) + releaseClose := make(chan struct{}) + var releaseOnce sync.Once + var workers sync.WaitGroup + var closeOnce sync.Once + previousHook := baseFilePoolBeforeCloseHandle + baseFilePoolBeforeCloseHandle = func() { + closeOnce.Do(func() { close(closeEntered) }) + <-releaseClose + } + t.Cleanup(func() { baseFilePoolBeforeCloseHandle = previousHook }) + // Independent rescue must run before the global hook is restored, including + // when the capacity assertion or a bounded wait fails. + t.Cleanup(func() { + releaseOnce.Do(func() { close(releaseClose) }) + workers.Wait() + }) + + fillDir := t.TempDir() + failedFill := func(context.Context) (*baseFileHandle, error) { + f, err := os.CreateTemp(fillDir, "ft2-failed-fill") + if err != nil { + return nil, err + } + if _, err = f.WriteString("12345678"); err != nil { + _ = f.Close() + return nil, err + } + if _, err = f.Seek(0, 0); err != nil { + _ = f.Close() + return nil, err + } + return &baseFileHandle{file: f, path: f.Name()}, errors.New("synthetic fill failure") + } + + firstResult := make(chan error, 1) + workers.Add(1) + go func() { + defer workers.Done() + _, err := pool.acquire(context.Background(), testPoolKey("failed", 8, "12345678"), failedFill) + firstResult <- err + }() + select { + case <-closeEntered: + case <-time.After(2 * time.Second): + t.Fatal("failed fill did not enter handle cleanup") + } + + // The failed fill has left the directory, but its FD is still inside + // closeHandle. A second admission must see the reserved slot. + secondResult := make(chan error, 1) + fill := testPoolFill(t, new(atomic.Int32), "abcdefgh") + workers.Add(1) + go func() { + defer workers.Done() + lease, err := pool.acquire(context.Background(), testPoolKey("blocked", 8, "abcdefgh"), fill) + if lease != nil { + lease.Release() + } + secondResult <- err + }() + select { + case err := <-secondResult: + require.ErrorIs(t, err, errBaseFilePoolCapacity) + case <-time.After(2 * time.Second): + t.Fatal("second admission blocked behind failed handle cleanup") + } + + releaseOnce.Do(func() { close(releaseClose) }) + select { + case err := <-firstResult: + require.Error(t, err) + case <-time.After(2 * time.Second): + t.Fatal("failed fill did not finish after cleanup was released") + } + + lease, err := pool.acquire(context.Background(), testPoolKey("after", 8, "abcdefgh"), testPoolFill(t, new(atomic.Int32), "abcdefgh")) + require.NoError(t, err, "the FD slot is reusable after the failed handle is actually closed") + data, err := lease.MapReadOnly() + require.NoError(t, err) + require.NoError(t, munmap(data)) + lease.Release() +} + +func TestBaseFilePoolDeferredValidationMappingConsumesByteQuota(t *testing.T) { + const size = int64(8) + pool := newBaseFilePool(size, 2) + defer pool.Close() + key := testPoolKey("deferred", size, "abcdefgh") + var fills atomic.Int32 + original := munmapFn + t.Cleanup(func() { munmapFn = original }) + munmapFn = func([]byte) error { + return errors.New("synthetic deferred munmap failure") + } + fill := func(context.Context) (*baseFileHandle, error) { + fills.Add(1) + f, err := os.CreateTemp(t.TempDir(), "ft2-deferred") + require.NoError(t, err) + _, err = f.WriteString("abcdefgh") + require.NoError(t, err) + data, err := mmapReadOnly(f) + require.NoError(t, err) + return &baseFileHandle{file: f, path: f.Name(), validationData: data}, errors.New("synthetic validation failure") + } + + _, err := pool.acquire(context.Background(), key, fill) + require.Error(t, err) + require.Equal(t, size, pool.deferredBytes, "a failed validation mapping transfers fill reservation into deferred bytes") + require.Equal(t, int32(1), fills.Load()) + + _, err = pool.acquire(context.Background(), testPoolKey("next", size, "12345678"), func(context.Context) (*baseFileHandle, error) { + t.Fatal("a deferred mapping must consume the entire byte budget") + return nil, nil + }) + require.ErrorIs(t, err, errBaseFilePoolCapacity) + require.Equal(t, int32(1), fills.Load()) + + munmapFn = original + pool.retryDeferred() + require.Zero(t, pool.deferredBytes) + require.Zero(t, pool.deferredCount()) + lease, err := pool.acquire(context.Background(), testPoolKey("after-cleanup", size, "abcdefgh"), testPoolFill(t, &fills, "abcdefgh")) + require.NoError(t, err, "capacity admission must recover after deferred mapping cleanup") + data, err := lease.MapReadOnly() + require.NoError(t, err) + require.NoError(t, munmap(data)) + lease.Release() + require.Equal(t, int32(2), fills.Load()) +} + +func mustMapLease(t *testing.T, lease *baseFileLease) []byte { + t.Helper() + data, err := lease.MapReadOnly() + require.NoError(t, err) + return data +} + +func TestBaseFilePoolPinnedFileSurvivesCloseUntilRelease(t *testing.T) { + pool := newBaseFilePool(8, 1) + var fills atomic.Int32 + lease, err := pool.acquire(context.Background(), testPoolKey("s0", 8, "12345678"), testPoolFill(t, &fills, "12345678")) + require.NoError(t, err) + data, err := lease.MapReadOnly() + require.NoError(t, err) + path := lease.entry.handle.path + pool.Close() + require.FileExists(t, path, "Close must not remove a file pinned by a live Segment") + require.NoError(t, munmap(data)) + lease.Release() + require.Eventually(t, func() bool { + _, statErr := os.Stat(path) + return os.IsNotExist(statErr) + }, time.Second, time.Millisecond) +} + +func TestBaseFilePoolClassifiesLeaderCancellationForHealthyWaiter(t *testing.T) { + pool := newBaseFilePool(64, 1) + defer pool.Close() + key := testPoolKey("cancel", 8, "abcdefgh") + started := make(chan struct{}) + fill := func(ctx context.Context) (*baseFileHandle, error) { + close(started) + <-ctx.Done() + return nil, ctx.Err() + } + leaderCtx, cancelLeader := context.WithCancel(context.Background()) + leaderResult := make(chan error, 1) + go func() { + _, err := pool.acquire(leaderCtx, key, fill) + leaderResult <- err + }() + <-started + waiterResult := make(chan error, 1) + go func() { + _, err := pool.acquire(context.Background(), key, fill) + waiterResult <- err + }() + for { + pool.mu.Lock() + entry := pool.entries[key] + ready := entry != nil && entry.waiters == 1 + pool.mu.Unlock() + if ready { + break + } + runtime.Gosched() + } + cancelLeader() + require.Error(t, <-leaderResult) + require.ErrorIs(t, <-waiterResult, errBaseFilePoolLeaderCanceled) +} + +func TestBaseFilePoolCloseClassifiesCanceledFill(t *testing.T) { + pool := newBaseFilePool(64, 1) + key := testPoolKey("close", 8, "abcdefgh") + started := make(chan struct{}) + fill := func(ctx context.Context) (*baseFileHandle, error) { + close(started) + <-ctx.Done() + return nil, ctx.Err() + } + result := make(chan error, 1) + go func() { + _, err := pool.acquire(context.Background(), key, fill) + result <- err + }() + <-started + pool.Close() + require.ErrorIs(t, <-result, errBaseFilePoolClosed) +} + +func TestBaseFilePoolContextCancellationWhileFilling(t *testing.T) { + pool := newBaseFilePool(8, 1) + defer pool.Close() + key := testPoolKey("s0", 8, "12345678") + started := make(chan struct{}) + finish := make(chan struct{}) + fill := func(context.Context) (*baseFileHandle, error) { + close(started) + <-finish + f, err := os.CreateTemp(t.TempDir(), "ft2-pool") + if err != nil { + return nil, err + } + _, _ = f.WriteString("12345678") + _, _ = f.Seek(0, 0) + return &baseFileHandle{file: f, path: f.Name()}, nil + } + ownerDone := make(chan *baseFileLease, 1) + go func() { + lease, _ := pool.acquire(context.Background(), key, fill) + ownerDone <- lease + }() + <-started + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond) + defer cancel() + _, err := pool.acquire(ctx, key, func(context.Context) (*baseFileHandle, error) { + t.Fatal("the waiter must not become a second filler") + return nil, nil + }) + require.ErrorIs(t, err, context.DeadlineExceeded) + close(finish) + owner := <-ownerDone + require.NotNil(t, owner) + owner.Release() +} + +func TestBaseFilePoolFillFailurePropagatesToExistingWaiters(t *testing.T) { + pool := newBaseFilePool(8, 1) + defer pool.Close() + key := testPoolKey("failed", 8, "12345678") + started := make(chan struct{}) + finish := make(chan struct{}) + fillErr := errors.New("synthetic immutable base fill failure") + fill := func(context.Context) (*baseFileHandle, error) { + close(started) + <-finish + return nil, fillErr + } + ownerDone := make(chan error, 1) + go func() { + _, err := pool.acquire(context.Background(), key, fill) + ownerDone <- err + }() + <-started + + var unexpectedFill atomic.Bool + waiterDone := make(chan error, 1) + go func() { + _, err := pool.acquire(context.Background(), key, func(context.Context) (*baseFileHandle, error) { + unexpectedFill.Store(true) + return nil, errors.New("unexpected second fill") + }) + waiterDone <- err + }() + require.Eventually(t, func() bool { + pool.mu.Lock() + defer pool.mu.Unlock() + entry := pool.entries[key] + return entry != nil && entry.waiters == 1 + }, time.Second, time.Millisecond) + close(finish) + require.ErrorIs(t, <-ownerDone, fillErr) + require.ErrorIs(t, <-waiterDone, fillErr) + require.False(t, unexpectedFill.Load(), "an existing waiter must not become a second filler") + + var fills atomic.Int32 + retry, err := pool.acquire(context.Background(), key, testPoolFill(t, &fills, "12345678")) + require.NoError(t, err, "a later independent request may retry after the failed generation") + retry.Release() + require.Equal(t, int32(1), fills.Load()) +} diff --git a/pkg/fulltext2/base_file_reuse_init.go b/pkg/fulltext2/base_file_reuse_init.go new file mode 100644 index 0000000000000..79e2e53abae6e --- /dev/null +++ b/pkg/fulltext2/base_file_reuse_init.go @@ -0,0 +1,327 @@ +//go:build fulltext2_base_file_reuse + +package fulltext2 + +import ( + "errors" + "sync" + "sync/atomic" + + veccache "github.com/matrixorigin/matrixone/pkg/vectorindex/cache" +) + +const experimentalBaseFileReuseEnabled = true + +const ( + experimentalBaseFileReuseMaxBytes = int64(4 << 30) + experimentalBaseFileReuseMaxFiles = 64 +) + +type experimentalOwnerEntry struct { + closeMu sync.Mutex + owner *baseFileOwner + token *BaseFileReuseOwnerToken + generation uint64 + closed bool + shutdownInProgress bool +} + +// BaseFileReuseOwnerToken is an opaque service-generation capability. CN +// keeps the token returned at Start and presents that same token at Close; +// an old service object therefore cannot close a newer owner that reused the +// same UUID. +type BaseFileReuseOwnerToken struct { + service string + generation uint64 + owner *baseFileOwner +} + +var experimentalOwnerState struct { + sync.Mutex + owners map[string]*experimentalOwnerEntry + globalClosed bool + shutdownInProgress bool +} + +var experimentalOwnerHookOnce sync.Once +var experimentalOwnerHookInstallCount uint32 + +// Close phase barriers are nil in production. Tests hold exact lifecycle +// windows without delaying unrelated services or changing ownership selection. +var experimentalOwnerCurrentBarrier func(*experimentalOwnerEntry) +var experimentalOwnerFinalizeBarrier func(*experimentalOwnerEntry) + +var ( + errExperimentalOwnerServiceMismatch = errors.New("fulltext2 experimental owner service mismatch") + errExperimentalOwnerShutdown = errors.New("fulltext2 experimental owner is shutting down") +) + +type experimentalOwnerSelection struct { + owner *baseFileOwner + closed bool +} + +func init() { + // The lifecycle hook must exist before the first VectorIndexCache can take a + // shutdown snapshot. Lazy registration from the first SQL query has a race: + // shutdown may copy the old hook list, then the query can publish an owner + // that will never receive the shutdown callback. + installExperimentalOwnerHook() +} + +// installExperimentalOwnerHook installs one stable dispatcher for the process. +// RegisterLifecycleHook has no unregister operation, so registering a closure for +// every owner generation would retain every old owner forever. The dispatcher +// instead looks up the current service owner when shutdown is delivered. +func installExperimentalOwnerHook() { + experimentalOwnerHookOnce.Do(func() { + atomic.AddUint32(&experimentalOwnerHookInstallCount, 1) + veccache.RegisterPreShutdownHook(experimentalOwnerBeforeShutdown) + veccache.RegisterLifecycleHook(experimentalOwnerLifecycleHook) + }) +} + +// experimentalOwnerBeforeShutdown publishes the registry barrier before +// invoking cancellations, outside the registry lock. No entry close mutex, +// cache drain, pool retry, or OS cleanup may run in this phase. +func experimentalOwnerBeforeShutdown() { + owners := markExperimentalOwnersClosing() + for _, entry := range owners { + entry.owner.cancelOperations() + } +} + +func markExperimentalOwnersClosing() map[string]*experimentalOwnerEntry { + experimentalOwnerState.Lock() + experimentalOwnerState.globalClosed = true + experimentalOwnerState.shutdownInProgress = true + owners := make(map[string]*experimentalOwnerEntry, len(experimentalOwnerState.owners)) + for service, entry := range experimentalOwnerState.owners { + if entry == nil { + continue + } + entry.closed = true + entry.shutdownInProgress = true + owners[service] = entry + } + experimentalOwnerState.Unlock() + return owners +} + +func experimentalOwnerLifecycleHook(shutdown bool) { + if !shutdown { + return + } + // Direct service/test callers retain the complete shutdown contract. In + // Cache.Destroy the prephase has already sealed the registry, so replacement + // generations cannot be admitted between this snapshot and finalization. + experimentalOwnerBeforeShutdown() + experimentalOwnerState.Lock() + owners := make(map[string]*experimentalOwnerEntry, len(experimentalOwnerState.owners)) + for service, entry := range experimentalOwnerState.owners { + if entry != nil { + owners[service] = entry + } + } + experimentalOwnerState.Unlock() + for service, entry := range owners { + _ = closeExperimentalOwnerEntry(service, entry) + } + experimentalOwnerState.Lock() + experimentalOwnerState.shutdownInProgress = false + experimentalOwnerState.Unlock() +} + +// initializeExperimentalOwnerService is the explicit lifecycle boundary for a +// new experiment service. The first service may be initialized while the +// process is open; after shutdown, a query cannot reopen the experiment merely +// by passing a different service string. +func initializeExperimentalOwnerService(service string) error { + if service == "" { + return errExperimentalOwnerServiceMismatch + } + experimentalOwnerState.Lock() + defer experimentalOwnerState.Unlock() + if experimentalOwnerState.shutdownInProgress { + return errExperimentalOwnerShutdown + } + if experimentalOwnerState.owners == nil { + experimentalOwnerState.owners = make(map[string]*experimentalOwnerEntry) + } + entry := experimentalOwnerState.owners[service] + if entry != nil && entry.shutdownInProgress { + return errExperimentalOwnerShutdown + } + if entry != nil && entry.closed { + if entry.owner != nil && entry.owner.hasResources() { + return errBaseFileOwnerPending + } + // Never mutate a retired entry into a new generation: an already queued + // close/dispatcher still holds that entry pointer. + entry = nil + } + if entry == nil { + entry = &experimentalOwnerEntry{} + experimentalOwnerState.owners[service] = entry + } + if entry.owner == nil { + entry.owner = newBaseFileOwner(experimentalBaseFileReuseMaxBytes, experimentalBaseFileReuseMaxFiles) + entry.generation++ + entry.token = &BaseFileReuseOwnerToken{service: service, generation: entry.generation, owner: entry.owner} + } + if entry.token == nil { + entry.generation++ + entry.token = &BaseFileReuseOwnerToken{service: service, generation: entry.generation, owner: entry.owner} + } + entry.shutdownInProgress = false + // An explicit service-start boundary may open a new generation after the + // process-wide vector-cache dispatcher observed shutdown. Pending owners + // remain reachable until their final resources have been released. + experimentalOwnerState.globalClosed = false + return nil +} + +// InitializeBaseFileReuseOwner is the explicit service-start boundary used by +// the tagged CN integration. Keeping this boundary separate from the SQL +// factory prevents a query from resurrecting a closed service owner. +func InitializeBaseFileReuseOwner(service string) (*BaseFileReuseOwnerToken, error) { + if err := initializeExperimentalOwnerService(service); err != nil { + return nil, err + } + experimentalOwnerState.Lock() + defer experimentalOwnerState.Unlock() + entry := experimentalOwnerState.owners[service] + if entry == nil || entry.token == nil { + return nil, errExperimentalOwnerServiceMismatch + } + return entry.token, nil +} + +// CloseBaseFileReuseOwner closes the owner named by one service-generation +// token. It is deliberately idempotent and may return errBaseFileOwnerPending +// while a pinned mapping or deferred munmap still has a live owner. The owner +// remains reachable so a later service-close retry can finish the cleanup. +func CloseBaseFileReuseOwner(token *BaseFileReuseOwnerToken) error { + if token == nil || token.service == "" { + return nil + } + experimentalOwnerState.Lock() + entry := experimentalOwnerState.owners[token.service] + // A stale service object is allowed to close idempotently, but it must never + // operate on the current generation that reused the same UUID. + if entry == nil || entry.token != token || entry.generation != token.generation { + experimentalOwnerState.Unlock() + return nil + } + entry.closed = true + entry.shutdownInProgress = true + experimentalOwnerState.Unlock() + + return closeExperimentalOwnerEntry(token.service, entry) +} + +func closeExperimentalOwnerEntry(service string, entry *experimentalOwnerEntry) error { + if entry == nil { + return nil + } + entry.closeMu.Lock() + defer entry.closeMu.Unlock() + experimentalOwnerState.Lock() + current := experimentalOwnerState.owners[service] == entry + experimentalOwnerState.Unlock() + if !current { + // A late close that was queued for a removed generation must not drain + // the cache of the new service instance that reused the same UUID. + return nil + } + if experimentalOwnerCurrentBarrier != nil { + experimentalOwnerCurrentBarrier(entry) + } + var err error + if entry.owner != nil { + entry.owner.beginClose() + // Drain the actual VectorIndexCache entries before closing the owner. + // The cache owns Search wrappers; the owner only owns files. + veccache.Cache.DestroyByService(service) + err = entry.owner.close() + } + if experimentalOwnerFinalizeBarrier != nil { + experimentalOwnerFinalizeBarrier(entry) + } + // Keep entry.closeMu through registry retirement. Otherwise a queued old + // closer can pass the current-entry check before this deletion, then scan + // service-only cache entries after Start publishes a replacement owner. + // Never hold the registry lock through cache, owner or OS cleanup waits. + experimentalOwnerState.Lock() + if experimentalOwnerState.owners[service] == entry { + entry.closed = true + entry.shutdownInProgress = false + if err == nil { + delete(experimentalOwnerState.owners, service) + } + } + experimentalOwnerState.Unlock() + return err +} + +func experimentalOwnerForSQL(service string) experimentalOwnerSelection { + if service == "" { + return experimentalOwnerSelection{} + } + experimentalOwnerState.Lock() + entry := experimentalOwnerState.owners[service] + if entry != nil && entry.closed { + selection := experimentalOwnerSelection{ + owner: entry.owner, + closed: true, + } + experimentalOwnerState.Unlock() + return selection + } + // Only CN Start creates an owner. In particular an empty registry after + // successful teardown must not let a late SQL factory resurrect a service. + // This allows fully cleaned entries to be removed instead of retaining an + // unbounded tombstone for every historical service UUID. + if entry == nil { + experimentalOwnerState.Unlock() + return experimentalOwnerSelection{closed: true} + } + owner := entry.owner + if owner != nil { + experimentalOwnerState.Unlock() + if _, err := owner.poolForSearch(); err == nil { + return experimentalOwnerSelection{owner: owner} + } + // A closed owner is deliberately returned to the factory. The caller + // retains it as a closed Search handle, so Load cannot fall back to the + // ordinary path after the service has shut down. + experimentalOwnerState.Lock() + if current := experimentalOwnerState.owners[service]; current == entry { + current.closed = true + } + experimentalOwnerState.Unlock() + return experimentalOwnerSelection{owner: owner, closed: true} + } + // This branch is defensive: a well-formed entry always carries an owner. + experimentalOwnerState.Unlock() + return experimentalOwnerSelection{closed: true} +} + +// newExperimentalFulltext2Search is the only build-tagged activation point for +// the local Base-file reuse prototype. It is intentionally absent from normal +// SQL construction; callers must opt into the tag and supply an explicit +// budget. +func newExperimentalFulltext2Search(cfg TableConfig, maxBytes int64, maxFiles int) *Fulltext2Search { + return newFulltext2SearchWithBaseOwner(cfg, newBaseFileOwner(maxBytes, maxFiles)) +} + +// NewFulltext2SearchForExecution is the sole SQL-construction switch for the +// local experiment. The regular build returns the ordinary search object; the +// tagged build borrows the CN-process owner above. +func NewFulltext2SearchForExecution(cfg TableConfig, service string) *Fulltext2Search { + selection := experimentalOwnerForSQL(service) + if selection.closed && selection.owner == nil { + return &Fulltext2Search{cfg: cfg, baseOwnerClosed: true} + } + return newFulltext2SearchWithBaseOwnerForService(cfg, selection.owner, service) +} diff --git a/pkg/fulltext2/base_file_reuse_init_default.go b/pkg/fulltext2/base_file_reuse_init_default.go new file mode 100644 index 0000000000000..21196d5f826b9 --- /dev/null +++ b/pkg/fulltext2/base_file_reuse_init_default.go @@ -0,0 +1,26 @@ +//go:build !fulltext2_base_file_reuse + +package fulltext2 + +const experimentalBaseFileReuseEnabled = false + +// The normal build has no optional Base-file owner. These no-op lifecycle +// entry points let CN service code keep one call site while preserving the +// production default loader and its ownership model. +type BaseFileReuseOwnerToken struct{} + +func InitializeBaseFileReuseOwner(_ string) (*BaseFileReuseOwnerToken, error) { return nil, nil } + +func CloseBaseFileReuseOwner(_ *BaseFileReuseOwnerToken) error { return nil } + +// newExperimentalFulltext2Search deliberately falls back to the ordinary +// loader unless the private fulltext2_base_file_reuse build tag is enabled. +func newExperimentalFulltext2Search(cfg TableConfig, _ int64, _ int) *Fulltext2Search { + return NewFulltext2Search(cfg) +} + +// NewFulltext2SearchForExecution keeps the normal SQL path unchanged unless +// the private fulltext2_base_file_reuse experiment is explicitly built. +func NewFulltext2SearchForExecution(cfg TableConfig, _ string) *Fulltext2Search { + return NewFulltext2Search(cfg) +} diff --git a/pkg/fulltext2/base_file_reuse_init_test.go b/pkg/fulltext2/base_file_reuse_init_test.go new file mode 100644 index 0000000000000..e8c1103832cc6 --- /dev/null +++ b/pkg/fulltext2/base_file_reuse_init_test.go @@ -0,0 +1,766 @@ +//go:build fulltext2_base_file_reuse + +package fulltext2 + +import ( + "context" + "fmt" + "runtime" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/matrixorigin/matrixone/pkg/vectorindex" + veccache "github.com/matrixorigin/matrixone/pkg/vectorindex/cache" + "github.com/matrixorigin/matrixone/pkg/vectorindex/sqlexec" + "github.com/stretchr/testify/require" +) + +// withExperimentalOwnerState isolates tests from the service-scoped owners +// used by the tagged SQL factory. The real lifecycle dispatcher remains +// installed; only the test's synthetic owner state is restored after the test. +func withExperimentalOwnerState(t *testing.T, owner *baseFileOwner, service string, closed bool) { + t.Helper() + experimentalOwnerState.Lock() + previousOwners := experimentalOwnerState.owners + previousGlobalClosed := experimentalOwnerState.globalClosed + previousShutdownInProgress := experimentalOwnerState.shutdownInProgress + owners := make(map[string]*experimentalOwnerEntry) + if service != "" { + owners[service] = &experimentalOwnerEntry{ + owner: owner, + closed: closed, + } + } + experimentalOwnerState.owners = owners + experimentalOwnerState.globalClosed = false + experimentalOwnerState.shutdownInProgress = false + if owner == nil && closed { + experimentalOwnerState.globalClosed = true + } + experimentalOwnerState.Unlock() + t.Cleanup(func() { + experimentalOwnerState.Lock() + experimentalOwnerState.owners = previousOwners + experimentalOwnerState.globalClosed = previousGlobalClosed + experimentalOwnerState.shutdownInProgress = previousShutdownInProgress + experimentalOwnerState.Unlock() + }) +} + +// fillingOwnerSearch is a real VectorIndexSearch wrapper whose Load owns a +// base-file-pool fill. It lets the shutdown test exercise the same lock order +// as the production cache drain without requiring a SQL cluster. +type fillingOwnerSearch struct { + owner *baseFileOwner + service string + started chan struct{} + once sync.Once +} + +func (s *fillingOwnerSearch) CacheServiceID() string { return s.service } + +func (s *fillingOwnerSearch) Search(*sqlexec.SqlProcess, any, vectorindex.RuntimeConfig) (any, []float64, error) { + return nil, nil, nil +} + +func (s *fillingOwnerSearch) SearchFloat32(*sqlexec.SqlProcess, any, vectorindex.RuntimeConfig, []int64, []float32) error { + return nil +} + +func (s *fillingOwnerSearch) SearchInto(*sqlexec.SqlProcess, any, vectorindex.RuntimeConfig, *vectorindex.SearchOutput) error { + return nil +} + +func (s *fillingOwnerSearch) Preload(*sqlexec.SqlProcess) error { return nil } + +func (s *fillingOwnerSearch) Load(*sqlexec.SqlProcess) error { + pool, err := s.owner.poolForSearch() + if err != nil { + return err + } + _, err = s.owner.run(context.Background(), func(context.Context) (*Segment, error) { + lease, acquireErr := pool.acquire(context.Background(), testPoolKey("close-order", 8, "abcdefgh"), func(ctx context.Context) (*baseFileHandle, error) { + s.once.Do(func() { close(s.started) }) + <-ctx.Done() + return nil, ctx.Err() + }) + if lease != nil { + lease.Release() + } + return nil, acquireErr + }) + return err +} + +func (s *fillingOwnerSearch) GetIndexSize() (int64, int64) { return 0, 0 } +func (s *fillingOwnerSearch) BuildTS() int64 { return 0 } +func (s *fillingOwnerSearch) Destroy() {} + +func TestExperimentalOwnerCloseCancelsInFlightFillingSearch(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + const service = "cn-filling-close" + token, err := InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + owner := experimentalOwnerForSQL(service).owner + + algo := &fillingOwnerSearch{owner: owner, service: service, started: make(chan struct{})} + entry := &veccache.VectorIndexSearch{Algo: algo} + entry.Cond = sync.NewCond(entry.Mutex.RLocker()) + const key = "filling-close-entry" + veccache.Cache.IndexMap.Store(key, entry) + t.Cleanup(func() { + veccache.Cache.IndexMap.Delete(key) + _ = owner.close() + }) + + loadDone := make(chan error, 1) + go func() { loadDone <- entry.Load(nil) }() + select { + case <-algo.started: + case <-time.After(2 * time.Second): + t.Fatal("the test Search did not enter a pool fill") + } + + closeDone := make(chan error, 1) + go func() { closeDone <- CloseBaseFileReuseOwner(token) }() + select { + case err := <-closeDone: + require.NoError(t, err) + case <-time.After(2 * time.Second): + // Keep the failed test from leaving the fill and cache-drain goroutines + // behind; the production path must have issued this cancellation first. + owner.pool.Close() + t.Fatal("owner shutdown waited for a filling Search without canceling the fill") + } + require.ErrorIs(t, <-loadDone, errBaseFilePoolClosed) + _, present := veccache.Cache.IndexMap.Load(key) + require.False(t, present) +} + +func TestExperimentalSQLFactoryRejectsClosedOwner(t *testing.T) { + owner := newBaseFileOwner(1024, 2) + withExperimentalOwnerState(t, owner, "cn-closed", false) + require.NoError(t, owner.close()) + + search := NewFulltext2SearchForExecution(TableConfig{IndexTable: "idx"}, "cn-closed") + require.Same(t, owner, search.baseOwner, "a closed owner must remain attached to the SQL handle") + require.Nil(t, search.basePool) + require.ErrorIs(t, search.Preload(nil), errBaseFileOwnerClosed, + "preload must also stop at the closed owner boundary") + require.ErrorIs(t, search.Load(nil), errBaseFileOwnerClosed, + "the factory must not turn a closed service owner into an ordinary load") + search.Destroy() +} + +func TestExperimentalOwnerShutdownDispatcherClosesCurrentOwner(t *testing.T) { + owner := newBaseFileOwner(1024, 2) + withExperimentalOwnerState(t, owner, "cn-shutdown", false) + + experimentalOwnerLifecycleHook(true) + _, err := owner.poolForSearch() + require.ErrorIs(t, err, errBaseFileOwnerClosed) + search := NewFulltext2SearchForExecution(TableConfig{IndexTable: "idx"}, "cn-shutdown") + require.Nil(t, search.baseOwner, "fully cleaned owners must leave the registry") + require.ErrorIs(t, search.Load(nil), errBaseFileOwnerClosed) + search.Destroy() +} + +func TestExperimentalOwnerAllowsOnlyAnExplicitNewServiceAfterClose(t *testing.T) { + oldOwner := newBaseFileOwner(1024, 2) + withExperimentalOwnerState(t, oldOwner, "cn-old", true) + require.NoError(t, oldOwner.close()) + + closedSelection := experimentalOwnerForSQL("cn-new") + require.True(t, closedSelection.closed) + require.NoError(t, initializeExperimentalOwnerService("cn-new")) + newOwner := experimentalOwnerForSQL("cn-new").owner + require.NotNil(t, newOwner) + require.NotSame(t, oldOwner, newOwner, + "a new service identity is the explicit lifecycle boundary for a new owner") + require.NoError(t, newOwner.close()) +} + +func TestExperimentalOwnerServiceLifecycleEntryPoints(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + token, err := InitializeBaseFileReuseOwner("cn-entrypoint") + require.NoError(t, err) + selection := experimentalOwnerForSQL("cn-entrypoint") + require.NotNil(t, selection.owner) + + require.NoError(t, CloseBaseFileReuseOwner(token)) + closed := experimentalOwnerForSQL("cn-entrypoint") + require.True(t, closed.closed) + require.Nil(t, closed.owner) + _, err = selection.owner.poolForSearch() + require.ErrorIs(t, err, errBaseFileOwnerClosed) + + // A later service generation must use the explicit initialization boundary; + // changing the query's service string is not sufficient to reopen the old + // owner. + other := experimentalOwnerForSQL("cn-entrypoint-new") + require.True(t, other.closed) + newToken, err := InitializeBaseFileReuseOwner("cn-entrypoint-new") + require.NoError(t, err) + newSelection := experimentalOwnerForSQL("cn-entrypoint-new") + require.NotNil(t, newSelection.owner) + require.NotSame(t, selection.owner, newSelection.owner) + require.NoError(t, CloseBaseFileReuseOwner(newToken)) +} + +func TestExperimentalOwnerIsolatedPerService(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + aToken, err := InitializeBaseFileReuseOwner("cn-a") + require.NoError(t, err) + bToken, err := InitializeBaseFileReuseOwner("cn-b") + require.NoError(t, err) + a := experimentalOwnerForSQL("cn-a") + b := experimentalOwnerForSQL("cn-b") + require.NotNil(t, a.owner) + require.NotNil(t, b.owner) + require.NotSame(t, a.owner, b.owner, + "service generations must not share a mutable owner or pool") + require.NoError(t, CloseBaseFileReuseOwner(aToken)) + require.NotNil(t, experimentalOwnerForSQL("cn-b").owner, + "closing one service must not close another service owner") + require.NoError(t, CloseBaseFileReuseOwner(bToken)) +} + +func TestExperimentalOwnerStaleTokenCannotCloseNewGeneration(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + oldToken, err := InitializeBaseFileReuseOwner("cn-reused") + require.NoError(t, err) + oldOwner := experimentalOwnerForSQL("cn-reused").owner + require.NoError(t, CloseBaseFileReuseOwner(oldToken)) + + newToken, err := InitializeBaseFileReuseOwner("cn-reused") + require.NoError(t, err) + newOwner := experimentalOwnerForSQL("cn-reused").owner + require.NotSame(t, oldOwner, newOwner) + + // A second Close from the old CN object is a stale no-op. It must not + // reach the newly initialized owner that reused the same service UUID. + require.NoError(t, CloseBaseFileReuseOwner(oldToken)) + _, err = newOwner.poolForSearch() + require.NoError(t, err) + require.NoError(t, CloseBaseFileReuseOwner(newToken)) +} + +func TestExperimentalOwnerStaleCloseEntryCannotDrainNewGeneration(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + const service = "cn-delayed-close" + oldToken, err := InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + experimentalOwnerState.Lock() + oldEntry := experimentalOwnerState.owners[service] + experimentalOwnerState.Unlock() + require.NoError(t, CloseBaseFileReuseOwner(oldToken)) + newToken, err := InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + search := newFulltext2SearchWithBaseOwnerForService(TableConfig{IndexTable: "delayed-close"}, newToken.owner, service) + entry := &veccache.VectorIndexSearch{Algo: search} + entry.Cond = sync.NewCond(entry.Mutex.RLocker()) + entry.Status.Store(veccache.STATUS_LOADED) + const key = "delayed-close-entry" + veccache.Cache.IndexMap.Store(key, entry) + t.Cleanup(func() { _ = CloseBaseFileReuseOwner(newToken); veccache.Cache.IndexMap.Delete(key) }) + // Model a close/dispatcher that already captured the old entry before + // another closer reclaimed it. UUID equality alone is insufficient here. + require.NoError(t, closeExperimentalOwnerEntry(service, oldEntry)) + resident, ok := veccache.Cache.IndexMap.Load(key) + require.True(t, ok) + require.Same(t, entry, resident) + _, err = newToken.owner.poolForSearch() + require.NoError(t, err) + require.NoError(t, CloseBaseFileReuseOwner(newToken)) +} + +func TestExperimentalOwnerCloseDrainsServiceCacheSearch(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + const service = "cn-cache-drain" + token, err := InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + owner := experimentalOwnerForSQL(service).owner + + source := loadedSearch(t) + search := newFulltext2SearchWithBaseOwnerForService(TableConfig{IndexTable: "cache-drain"}, owner, service) + search.idx = source.idx + search.loaded = true + source.idx = nil + source.Destroy() + + entry := &veccache.VectorIndexSearch{Algo: search} + entry.Cond = sync.NewCond(entry.Mutex.RLocker()) + entry.Status.Store(veccache.STATUS_LOADED) + const key = "cache-drain-entry" + veccache.Cache.IndexMap.Store(key, entry) + t.Cleanup(func() { veccache.Cache.IndexMap.Delete(key) }) + + require.NoError(t, CloseBaseFileReuseOwner(token)) + _, present := veccache.Cache.IndexMap.Load(key) + require.False(t, present, "service close must evict the resident cache wrapper before closing its owner") + require.Nil(t, search.idx, "the resident Search must release its Segment before owner.Close") + require.True(t, search.baseOwnerClosed, "a retry racing service close must fail closed") + require.ErrorIs(t, search.Preload(nil), errBaseFileOwnerClosed, + "a cache retry of the same Search must fail closed after owner shutdown") +} + +func TestExperimentalOwnerRegistersOneStableLifecycleDispatcher(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + before := atomic.LoadUint32(&experimentalOwnerHookInstallCount) + + require.NoError(t, initializeExperimentalOwnerService("cn-hook")) + first := experimentalOwnerForSQL("cn-hook") + second := experimentalOwnerForSQL("cn-hook") + require.Same(t, first.owner, second.owner) + hookToken, err := InitializeBaseFileReuseOwner("cn-hook") + require.NoError(t, err) + require.NoError(t, CloseBaseFileReuseOwner(hookToken)) + thirdSelection := experimentalOwnerForSQL("cn-hook-new-service") + require.True(t, thirdSelection.closed) + require.NoError(t, initializeExperimentalOwnerService("cn-hook-new-service")) + third := experimentalOwnerForSQL("cn-hook-new-service").owner + require.NotSame(t, first.owner, third) + require.NoError(t, third.close()) + + after := atomic.LoadUint32(&experimentalOwnerHookInstallCount) + require.LessOrEqual(t, after-before, uint32(1), + "owner generations must not register one permanent hook each") +} + +func TestExperimentalFactoryShutdownRaceCannotLeaveOpenOwner(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + require.NoError(t, initializeExperimentalOwnerService("cn-race")) + start := make(chan struct{}) + selectionCh := make(chan experimentalOwnerSelection, 1) + factoryDone := make(chan struct{}) + shutdownDone := make(chan struct{}) + go func() { + defer close(factoryDone) + <-start + selectionCh <- experimentalOwnerForSQL("cn-race") + }() + go func() { + defer close(shutdownDone) + <-start + experimentalOwnerLifecycleHook(true) + }() + close(start) + selection := <-selectionCh + <-factoryDone + <-shutdownDone + + experimentalOwnerState.Lock() + entry := experimentalOwnerState.owners["cn-race"] + var owner *baseFileOwner + var ownerEntryClosed bool + if entry != nil { + owner = entry.owner + ownerEntryClosed = entry.closed + // Only a pending owner may still occupy the registry after shutdown. + } + experimentalOwnerState.Unlock() + if owner != nil { + require.True(t, ownerEntryClosed) + _, err := owner.poolForSearch() + require.ErrorIs(t, err, errBaseFileOwnerClosed) + } + if selection.owner != nil { + _, err := selection.owner.poolForSearch() + require.ErrorIs(t, err, errBaseFileOwnerClosed) + } else { + require.True(t, selection.closed) + } +} + +func TestExperimentalOwnerRejectsNewServiceDuringShutdown(t *testing.T) { + owner := newBaseFileOwner(1024, 2) + withExperimentalOwnerState(t, owner, "cn-existing", false) + + operationEntered := make(chan struct{}) + releaseOperation := make(chan struct{}) + operationDone := make(chan struct{}) + shutdownDone := make(chan struct{}) + var releaseOnce sync.Once + var workers sync.WaitGroup + t.Cleanup(func() { + // Rescue does not depend on owner cancellation, the behavior under test. + releaseOnce.Do(func() { close(releaseOperation) }) + workers.Wait() + }) + workers.Add(1) + go func() { + defer workers.Done() + defer close(operationDone) + _, _ = owner.run(context.Background(), func(context.Context) (*Segment, error) { + close(operationEntered) + <-releaseOperation + return nil, nil + }) + }() + select { + case <-operationEntered: + case <-time.After(2 * time.Second): + t.Fatal("owner operation did not start") + } + + workers.Add(1) + go func() { + defer workers.Done() + defer close(shutdownDone) + experimentalOwnerLifecycleHook(true) + }() + + // The admitted operation keeps owner.close blocked after the dispatcher + // publishes the shutdown state. Spin only on the protected state; no sleep + // or timing guess is used to establish the interleaving. + deadline := time.NewTimer(2 * time.Second) + defer deadline.Stop() + for { + experimentalOwnerState.Lock() + inProgress := experimentalOwnerState.shutdownInProgress + experimentalOwnerState.Unlock() + if inProgress { + break + } + select { + case <-deadline.C: + t.Fatal("dispatcher did not publish its shutdown admission barrier") + default: + } + runtime.Gosched() + } + require.ErrorIs(t, initializeExperimentalOwnerService("cn-new"), errExperimentalOwnerShutdown) + + releaseOnce.Do(func() { close(releaseOperation) }) + for _, done := range []<-chan struct{}{operationDone, shutdownDone} { + select { + case <-done: + case <-time.After(2 * time.Second): + t.Fatal("shutdown workers did not finish after rescue") + } + } + selection := experimentalOwnerForSQL("cn-new") + require.True(t, selection.closed) +} + +func TestExperimentalFactoryCompletesBeforeShutdown(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + require.NoError(t, initializeExperimentalOwnerService("cn-factory-first")) + selection := experimentalOwnerForSQL("cn-factory-first") + require.NotNil(t, selection.owner) + + experimentalOwnerLifecycleHook(true) + experimentalOwnerState.Lock() + entry := experimentalOwnerState.owners["cn-factory-first"] + experimentalOwnerState.Unlock() + require.Nil(t, entry, "successful dispatcher cleanup must release its registry reference") + _, err := selection.owner.poolForSearch() + require.ErrorIs(t, err, errBaseFileOwnerClosed) +} + +func TestExperimentalOwnerRegistryReclaimsClosedServices(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + for i := 0; i < 100; i++ { + service := fmt.Sprintf("cn-history-%d", i) + token, err := InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + require.NoError(t, CloseBaseFileReuseOwner(token)) + experimentalOwnerState.Lock() + count := len(experimentalOwnerState.owners) + experimentalOwnerState.Unlock() + require.Zero(t, count, "closed UUIDs must not accumulate owner/token references") + search := NewFulltext2SearchForExecution(TableConfig{IndexTable: "idx"}, service) + search.Destroy() + require.ErrorIs(t, search.Preload(nil), errBaseFileOwnerClosed) + require.ErrorIs(t, search.Load(nil), errBaseFileOwnerClosed) + } + // Even an empty, initially open registry cannot bootstrap from a query. + require.True(t, experimentalOwnerForSQL("cn-never-started").closed) +} + +func TestExperimentalOwnerRegistryRetainsPendingResources(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + const service = "cn-pending" + token, err := InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + pool, err := token.owner.poolForSearch() + require.NoError(t, err) + lease, err := pool.acquire(context.Background(), testPoolKey("pinned", 8, "abcdefgh"), testPoolFill(t, new(atomic.Int32), "abcdefgh")) + require.NoError(t, err) + t.Cleanup(func() { lease.Release(); _ = CloseBaseFileReuseOwner(token) }) + require.ErrorIs(t, CloseBaseFileReuseOwner(token), errBaseFileOwnerPending) + experimentalOwnerState.Lock() + entry := experimentalOwnerState.owners[service] + experimentalOwnerState.Unlock() + require.NotNil(t, entry) + require.Same(t, token.owner, entry.owner) + require.ErrorIs(t, initializeExperimentalOwnerService(service), errBaseFileOwnerPending) + require.True(t, experimentalOwnerForSQL(service).closed) + lease.Release() + require.NoError(t, CloseBaseFileReuseOwner(token)) + experimentalOwnerState.Lock() + entry = experimentalOwnerState.owners[service] + experimentalOwnerState.Unlock() + require.Nil(t, entry) +} + +func TestExperimentalDispatcherRetainsPendingAndReclaimsCleanOwners(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + previousCache := veccache.Cache + cache := veccache.NewVectorIndexCache() + veccache.Cache = cache + t.Cleanup(func() { veccache.Cache = previousCache }) + pending, err := InitializeBaseFileReuseOwner("cn-dispatch-pending") + require.NoError(t, err) + clean, err := InitializeBaseFileReuseOwner("cn-dispatch-clean") + require.NoError(t, err) + pool, err := pending.owner.poolForSearch() + require.NoError(t, err) + lease, err := pool.acquire(context.Background(), testPoolKey("dispatch-pinned", 8, "abcdefgh"), testPoolFill(t, new(atomic.Int32), "abcdefgh")) + require.NoError(t, err) + t.Cleanup(func() { lease.Release(); _ = CloseBaseFileReuseOwner(pending) }) + cache.Destroy() + experimentalOwnerState.Lock() + pendingEntry := experimentalOwnerState.owners["cn-dispatch-pending"] + cleanEntry := experimentalOwnerState.owners["cn-dispatch-clean"] + experimentalOwnerState.Unlock() + require.NotNil(t, pendingEntry) + require.Nil(t, cleanEntry) + _, err = clean.owner.poolForSearch() + require.ErrorIs(t, err, errBaseFileOwnerClosed) + lease.Release() + cache.Destroy() + experimentalOwnerState.Lock() + count := len(experimentalOwnerState.owners) + experimentalOwnerState.Unlock() + require.Zero(t, count) +} + +func TestExperimentalFactoryRejectsFirstUseAfterShutdown(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + experimentalOwnerLifecycleHook(true) + + selection := experimentalOwnerForSQL("cn-late") + require.True(t, selection.closed) + require.Nil(t, selection.owner) + search := NewFulltext2SearchForExecution(TableConfig{IndexTable: "idx"}, "cn-late") + require.True(t, search.baseOwnerClosed) + require.ErrorIs(t, search.Load(nil), errBaseFileOwnerClosed) + search.Destroy() + + // Reopening is an explicit service-start action, not a query-side service + // string change. + require.NoError(t, initializeExperimentalOwnerService("cn-late")) + owner := experimentalOwnerForSQL("cn-late").owner + require.NotNil(t, owner) + require.NoError(t, owner.close()) +} + +func TestExperimentalClosedOwnerlessSearchStaysClosedAfterDestroy(t *testing.T) { + withExperimentalOwnerState(t, nil, "", true) + search := NewFulltext2SearchForExecution(TableConfig{IndexTable: "idx"}, "cn-ownerless-closed") + require.True(t, search.baseOwnerClosed) + require.Nil(t, search.baseOwner) + + search.Destroy() + search.Destroy() + require.True(t, search.baseOwnerClosed, + "Destroy must not erase the closed marker when no owner pointer was available") + require.ErrorIs(t, search.Preload(nil), errBaseFileOwnerClosed) + require.ErrorIs(t, search.Load(nil), errBaseFileOwnerClosed) +} + +// operationOwnerSearch follows production Load's complete-operation context. +// Cancellation is observed while the real cache entry write lock is held; +// release is an independent rescue and also holds teardown open for assertions. +type operationOwnerSearch struct { + fillingOwnerSearch + source context.Context + canceled chan struct{} + release chan struct{} + destroyed atomic.Bool +} + +func (s *operationOwnerSearch) Load(*sqlexec.SqlProcess) error { + return s.owner.runOperation(s.source, func(opCtx context.Context) error { + close(s.started) + select { + case <-opCtx.Done(): + close(s.canceled) + case <-s.release: + return opCtx.Err() + } + <-s.release + return opCtx.Err() + }) +} + +func (s *operationOwnerSearch) Destroy() { s.destroyed.Store(true) } + +func TestExperimentalGlobalDestroyCancelsBeforeEntryDrain(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + const service = "cn-global-operation-close" + token, err := InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + owner := experimentalOwnerForSQL(service).owner + previousCache := veccache.Cache + cache := veccache.NewVectorIndexCache() + veccache.Cache = cache + t.Cleanup(func() { veccache.Cache = previousCache }) + + source, cancelSource := context.WithCancel(context.Background()) + algo := &operationOwnerSearch{ + fillingOwnerSearch: fillingOwnerSearch{owner: owner, service: service, started: make(chan struct{})}, + source: source, canceled: make(chan struct{}), release: make(chan struct{}), + } + entry := &veccache.VectorIndexSearch{Algo: algo} + entry.Cond = sync.NewCond(entry.Mutex.RLocker()) + const key = "global-operation-close-entry" + cache.IndexMap.Store(key, entry) + var workers sync.WaitGroup + var releaseOnce sync.Once + release := func() { releaseOnce.Do(func() { close(algo.release) }) } + // Registered before either worker: failed assertions cannot strand workers + // or restore process globals while a worker still uses them. + t.Cleanup(func() { + release() + cancelSource() + workers.Wait() + cache.Destroy() + _ = owner.close() + }) + loadDone := make(chan error, 1) + workers.Add(1) + go func() { defer workers.Done(); loadDone <- entry.Load(nil) }() + select { + case <-algo.started: + case <-time.After(5 * time.Second): + t.Fatal("Load did not enter the complete owner operation") + } + destroyDone := make(chan struct{}) + workers.Add(1) + go func() { defer workers.Done(); cache.Destroy(); close(destroyDone) }() + select { + case <-algo.canceled: + case <-time.After(5 * time.Second): + t.Fatal("global Destroy waited for entry Load before canceling its owner operation") + } + require.NoError(t, source.Err(), "source cancellation must not provide the shutdown oracle") + require.False(t, algo.destroyed.Load(), "Load must still hold the entry lock") + require.True(t, experimentalOwnerForSQL(service).closed) + require.ErrorIs(t, initializeExperimentalOwnerService("cn-during-global-close"), errExperimentalOwnerShutdown) + require.ErrorIs(t, owner.runOperation(context.Background(), func(context.Context) error { + t.Error("shutdown admitted another complete operation") + return nil + }), errBaseFileOwnerClosed) + release() + select { + case <-destroyDone: + case <-time.After(5 * time.Second): + t.Fatal("global Destroy did not complete after the load released its lock") + } + require.ErrorIs(t, <-loadDone, context.Canceled) + require.True(t, algo.destroyed.Load()) + _, present := cache.IndexMap.Load(key) + require.False(t, present) + require.Equal(t, int32(veccache.STATUS_DESTROYED), entry.Status.Load()) + require.ErrorIs(t, owner.runOperation(context.Background(), func(context.Context) error { return nil }), errBaseFileOwnerClosed) + require.NoError(t, CloseBaseFileReuseOwner(token), "dedicated close remains idempotent after global shutdown") + replacement, err := InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + require.NotSame(t, token.owner, replacement.owner) + require.NoError(t, CloseBaseFileReuseOwner(token), "old token must not close the replacement") + require.False(t, replacement.owner.isClosing()) + require.NoError(t, CloseBaseFileReuseOwner(replacement)) +} + +func TestExperimentalGlobalDestroyCancelsAllOwnersBeforePoolCleanup(t *testing.T) { + withExperimentalOwnerState(t, nil, "", false) + previousCache := veccache.Cache + cache := veccache.NewVectorIndexCache() + veccache.Cache = cache + t.Cleanup(func() { veccache.Cache = previousCache }) + var workers sync.WaitGroup + unblock := make([]func(), 0, 2) + var unlockOnce sync.Once + unlock := func() { + unlockOnce.Do(func() { + for _, fn := range unblock { + fn() + } + }) + } + source, cancelSource := context.WithCancel(context.Background()) + tokens := make([]*BaseFileReuseOwnerToken, 0, 2) + canceled := make([]chan struct{}, 0, 2) + t.Cleanup(func() { + unlock() + cancelSource() + workers.Wait() + for _, token := range tokens { + _ = CloseBaseFileReuseOwner(token) + } + }) + for _, service := range []string{"cn-global-all-a", "cn-global-all-b"} { + token, err := InitializeBaseFileReuseOwner(service) + require.NoError(t, err) + tokens = append(tokens, token) + pool, err := token.owner.poolForSearch() + require.NoError(t, err) + // Either registry iteration order must cancel both owners before the + // first pool retry/OS cleanup can block. + pool.retryMu.Lock() + unblock = append(unblock, pool.retryMu.Unlock) + entered, canceledOwner := make(chan struct{}), make(chan struct{}) + canceled = append(canceled, canceledOwner) + workers.Add(1) + go func() { + defer workers.Done() + _ = token.owner.runOperation(source, func(opCtx context.Context) error { + close(entered) + <-opCtx.Done() + close(canceledOwner) + return opCtx.Err() + }) + }() + select { + case <-entered: + case <-time.After(5 * time.Second): + t.Fatal("owner operation did not enter") + } + } + done := make(chan struct{}) + workers.Add(1) + go func() { defer workers.Done(); cache.Destroy(); close(done) }() + for _, canceledOwner := range canceled { + select { + case <-canceledOwner: + case <-time.After(5 * time.Second): + t.Fatal("pool cleanup blocked cancellation of another owner") + } + } + require.NoError(t, source.Err()) + for _, token := range tokens { + require.True(t, token.owner.isClosing()) + require.True(t, experimentalOwnerForSQL(token.service).closed) + } + // A competing dedicated close waits only in cleanup; it must not reopen + // admission or remove a replacement generation. + closeDone := make(chan error, 1) + workers.Add(1) + go func() { defer workers.Done(); closeDone <- CloseBaseFileReuseOwner(tokens[0]) }() + require.ErrorIs(t, initializeExperimentalOwnerService("cn-global-all-new"), errExperimentalOwnerShutdown) + unlock() + select { + case <-done: + case <-time.After(5 * time.Second): + t.Fatal("global cleanup did not finish after pool retry release") + } + require.NoError(t, <-closeDone) + experimentalOwnerState.Lock() + count := len(experimentalOwnerState.owners) + experimentalOwnerState.Unlock() + require.Zero(t, count) +} diff --git a/pkg/fulltext2/mmap_unix.go b/pkg/fulltext2/mmap_unix.go index 92be18430a04f..7bb3d7ce9439d 100644 --- a/pkg/fulltext2/mmap_unix.go +++ b/pkg/fulltext2/mmap_unix.go @@ -21,12 +21,13 @@ import ( "syscall" ) -// mmapReadOnly maps the whole file into a shared, read-only region. One mapping is -// created per cached base segment at load and shared by all concurrent queries — -// reads are plain memory loads (no lock), the kernel serializes page faults, and -// the OS page cache (reclaimable, unlike our off-heap C-malloc) is the residency -// manager. Unmapped by munmap under the cache's eviction write-lock (no reader in -// flight). Returns nil for a zero-length file. +var munmapFn = syscall.Munmap + +// mmapReadOnly maps the whole file into a shared, read-only region. Normal +// loading creates one mapping per Segment; the experimental file pool creates +// one independent mapping per lease while the kernel still shares the backing +// OS page cache. Reads are plain memory loads and the cache's Segment lifetime +// controls munmap. Returns nil for a zero-length file. func mmapReadOnly(f *os.File) ([]byte, error) { fi, err := f.Stat() if err != nil { @@ -43,5 +44,5 @@ func munmap(b []byte) error { if len(b) == 0 { return nil } - return syscall.Munmap(b) + return munmapFn(b) } diff --git a/pkg/fulltext2/search_cache.go b/pkg/fulltext2/search_cache.go index a62500b9fe65c..3cc6525549ad8 100644 --- a/pkg/fulltext2/search_cache.go +++ b/pkg/fulltext2/search_cache.go @@ -73,6 +73,18 @@ type Fulltext2Search struct { idx *Index loaded bool + // basePool is nil for every normal production search. The package-private + // constructor below enables the bounded immutable-file experiment without + // adding a SQL variable, public configuration, or changing cache ownership. + basePool *baseFilePool + basePoolOwned bool + baseOwner *baseFileOwner + baseOwnerClosed bool + // baseOwnerService identifies the CN owner for the optional cache drain. It + // is deliberately only an identity string; the owner/token remain the + // source of lifecycle authority. + baseOwnerService string + // Generation captured at Load (same txn snapshot as the loaded data) for the cache's // cross-CN freshness check (IsStale, called off the housekeeping goroutine): loadedTs = // MAX(metadata.timestamp) (REBUILD/MERGE), loadedTail = MAX(tag=1 CdcTail chunk_id) (CDC @@ -119,6 +131,40 @@ func NewFulltext2Search(cfg TableConfig) *Fulltext2Search { return &Fulltext2Search{cfg: cfg} } +// newFulltext2SearchWithBasePool is intentionally package-private and used by +// the local Base-file reuse prototype only. A pooled file never supplies a +// shared Segment; Load still creates an independent mmap, Index and liveness +// state for every search object. The caller owns the pool and must close it at +// the CN/cache shutdown boundary; Destroy only releases this handle's leases. +func newFulltext2SearchWithBasePool(cfg TableConfig, pool *baseFilePool) *Fulltext2Search { + return &Fulltext2Search{cfg: cfg, basePool: pool} +} + +func newFulltext2SearchWithBaseOwner(cfg TableConfig, owner *baseFileOwner) *Fulltext2Search { + return newFulltext2SearchWithBaseOwnerForService(cfg, owner, "") +} + +func newFulltext2SearchWithBaseOwnerForService(cfg TableConfig, owner *baseFileOwner, service string) *Fulltext2Search { + if owner == nil { + return &Fulltext2Search{cfg: cfg} + } + pool, err := owner.poolForSearch() + if err != nil { + return &Fulltext2Search{cfg: cfg, baseOwner: owner, baseOwnerService: service, baseOwnerClosed: true} + } + return &Fulltext2Search{cfg: cfg, basePool: pool, baseOwner: owner, baseOwnerService: service} +} + +// CacheServiceID is consumed by the optional VectorIndexCache shutdown drain. +// Ordinary searches return an empty identity and are never selected by that +// service-scoped operation. +func (s *Fulltext2Search) CacheServiceID() string { + if s == nil { + return "" + } + return s.baseOwnerService +} + // Load reads the index from the chunk store: the tag=0 base sub-indexes plus the // tag=1 CdcTail delta frames (+ delete set), assembled into a queryable Index with // global stats and per-pk liveness. An index created on an empty table has no tag=0 @@ -127,6 +173,25 @@ func NewFulltext2Search(cfg TableConfig) *Fulltext2Search { // from -- without loading or mapping any of them, so the cache can reclaim room for this index // before Load claims it. func (s *Fulltext2Search) Preload(sqlproc *sqlexec.SqlProcess) error { + if s.baseOwnerClosed { + return errBaseFileOwnerClosed + } + if s.baseOwner != nil { + return s.baseOwner.runOperation(sqlproc.GetTopContext(), func(ctx context.Context) error { + err := s.preload(sqlproc.WithContext(ctx)) + if err == nil { + err = context.Cause(ctx) + } + if err != nil { + s.preloaded = false + } + return err + }) + } + return s.preload(sqlproc) +} + +func (s *Fulltext2Search) preload(sqlproc *sqlexec.SqlProcess) error { ndoc, bytes, err := baseDocCountAndBytes(sqlproc, s.cfg) if err != nil { return err @@ -145,6 +210,31 @@ func (s *Fulltext2Search) Preload(sqlproc *sqlexec.SqlProcess) error { } func (s *Fulltext2Search) Load(sqlproc *sqlexec.SqlProcess) error { + // A service-owned experiment handle must fail closed after its owner has + // shut down. Do this before the ordinary budget queries: a closed owner is + // not permission to silently fall back to a normal Search or to create a new + // owner behind the cache lifecycle boundary. + if s.baseOwnerClosed { + return errBaseFileOwnerClosed + } + if s.baseOwner != nil { + return s.baseOwner.runOperation(sqlproc.GetTopContext(), func(ctx context.Context) error { + err := s.load(sqlproc.WithContext(ctx)) + // Generation capture intentionally tolerates SQL errors on the normal + // path. It must not turn owner cancellation into a successful load. + if err == nil { + err = context.Cause(ctx) + } + if err != nil { + s.Destroy() + } + return err + }) + } + return s.load(sqlproc) +} + +func (s *Fulltext2Search) load(sqlproc *sqlexec.SqlProcess) error { // Fail fast on the QUERY path if the bases' per-doc metadata won't fit the heap // budget, rather than OOM-killing the CN (which takes down every query on the node). // This guard is on Load, NOT LoadAllBases, so CompactSegments (MERGE) — the remedy — @@ -158,7 +248,15 @@ func (s *Fulltext2Search) Load(sqlproc *sqlexec.SqlProcess) error { } else if err := checkBaseLoadBudget(sqlproc, s.cfg); err != nil { return err } - bases, err := LoadAllBases(sqlproc, s.cfg) + var bases []*Segment + var err error + if s.baseOwner != nil { + bases, err = loadAllBasesWithOwner(sqlproc, s.cfg, s.baseOwner) + } else if s.basePool != nil { + bases, err = loadAllBasesWithPool(sqlproc, s.cfg, s.basePool) + } else { + bases, err = LoadAllBases(sqlproc, s.cfg) + } if err != nil { return err } @@ -167,7 +265,8 @@ func (s *Fulltext2Search) Load(sqlproc *sqlexec.SqlProcess) error { freeSegs(bases) // munmap the base segments on a tail-load error (don't leak the mappings) return err } - segs := append(bases, tails...) + segs := bases + segs = append(segs, tails...) s.idx = NewIndex(segs, deletes) s.loaded = true s.baseSegs = len(bases) @@ -209,11 +308,13 @@ func (s *Fulltext2Search) EmptyGeneration() bool { // resident, so the device figure is 0. // GetIndexSize charges the doc heap PLUS the file this entry maps. // -// The mapping is not shared between cache entries: LoadFromStorage spills to a fresh LOCAL file -// per load and mmaps it whole, so a second named-snapshot key of the same index maps its own -// copy. Reporting only the heap made a multi-megabyte mapping look like a few hundred bytes, and -// N generations could pin N files while the governor saw almost nothing. Same shape as hnsw: -// rows x per-row heap, plus the mapped file. +// The normal mapping is not shared between cache entries: LoadFromStorage spills to a fresh +// LOCAL file per load and mmaps it whole, so a second named-snapshot key of the same index maps +// its own copy. The experimental Base-file pool may share the immutable file backing those +// mappings, but still charges each independent mapping here. Reporting only the heap made a +// multi-megabyte mapping look like a few hundred bytes, and N generations could pin N files +// while the governor saw almost nothing. Same shape as hnsw: rows x per-row heap, plus mapped +// bytes. func (s *Fulltext2Search) GetIndexSize() (hostBytes, deviceBytes int64) { if !s.loaded || s.idx == nil { // Between Preload and Load: report what Load is about to cost, mapping included. @@ -495,11 +596,48 @@ func (s *Fulltext2Search) SearchFloat32(proc *sqlexec.SqlProcess, query any, rt // are views into those mappings), then drops it. The cache holds the write lock // around this, so no search is in flight. func (s *Fulltext2Search) Destroy() { + pool := s.basePool + poolOwned := s.basePoolOwned + owner := s.baseOwner if s.idx != nil { s.idx.Free() } + if pool != nil { + // A failed munmap leaves the Segment and its lease in the pool's + // deferred-owner queue; retry once at the same lifecycle boundary before + // dropping the search handle. + pool.retryDeferred() + } s.idx = nil s.loaded = false + // A service-owned experimental pool is closed by its owner lifecycle hook, + // not by this Search handle. A caller that injects a shared CN-level pool + // likewise retains ownership. The ordinary constructor leaves basePool nil. + if pool != nil && poolOwned { + pool.Close() + } + // A legacy private-owned pool remains attached while an OS mapping still + // needs a retry. Service-owned pools are held by baseOwner and are detached + // here; the service owner remains reachable for a later shutdown retry. + if pool != nil && poolOwned && pool.deferredCount() != 0 { + s.basePool = pool + s.basePoolOwned = true + s.baseOwner = owner + if owner != nil { + s.baseOwnerClosed = owner.isClosing() + } + return + } + s.basePool = nil + s.basePoolOwned = false + // The VectorIndexCache may retry the same newalgo after it destroys an + // entry. Keep its service owner binding so a normal retry can reuse the + // pool, and a retry after CN shutdown fails closed instead of silently + // falling back to LoadAllBases. + s.baseOwner = owner + if owner != nil { + s.baseOwnerClosed = owner.isClosing() + } } // SetReservedAhead receives what the governor has already promised to loads ahead of this one, diff --git a/pkg/fulltext2/search_cache_unit_test.go b/pkg/fulltext2/search_cache_unit_test.go index fedddedf501b4..1364a7fe01b34 100644 --- a/pkg/fulltext2/search_cache_unit_test.go +++ b/pkg/fulltext2/search_cache_unit_test.go @@ -15,8 +15,10 @@ package fulltext2 import ( + "context" "math" "strings" + "sync/atomic" "testing" "github.com/matrixorigin/matrixone/pkg/common/moerr" @@ -69,6 +71,36 @@ func TestFulltext2SearchNewAndUnloaded(t *testing.T) { require.False(t, s.loaded) } +func TestBaseFileReusePrototypeIsDefaultOff(t *testing.T) { + plain := NewFulltext2Search(TableConfig{IndexTable: "idx"}) + require.Nil(t, plain.basePool, "normal search construction must not enable the prototype") + token, err := InitializeBaseFileReuseOwner("cn-default-off-test") + require.NoError(t, err) + t.Cleanup(func() { require.NoError(t, CloseBaseFileReuseOwner(token)) }) + execution := NewFulltext2SearchForExecution(TableConfig{IndexTable: "idx"}, "cn-default-off-test") + t.Cleanup(execution.Destroy) + defaultFactory := newExperimentalFulltext2Search(TableConfig{IndexTable: "idx"}, 1024, 2) + t.Cleanup(defaultFactory.Destroy) + if experimentalBaseFileReuseEnabled { + require.NotNil(t, defaultFactory.basePool, "the private build tag must enable the experimental factory") + require.NotNil(t, defaultFactory.baseOwner, "the tagged factory must borrow a service owner") + require.NotNil(t, execution.basePool, "tagged SQL construction must borrow the experiment owner") + } else { + require.Nil(t, defaultFactory.basePool, "the experimental factory is inert without its private build tag") + require.Nil(t, defaultFactory.baseOwner) + require.Nil(t, execution.basePool, "ordinary SQL construction must remain pool-free") + } + pool := newBaseFilePool(1024, 2) + defer pool.Close() + experimental := newFulltext2SearchWithBasePool(TableConfig{IndexTable: "idx"}, pool) + require.Same(t, pool, experimental.basePool) + experimental.Destroy() + require.Nil(t, experimental.basePool, "destroy must detach a caller-owned experiment pool") + lease, err := pool.acquire(context.Background(), testPoolKey("owner", 1, "x"), testPoolFill(t, new(atomic.Int32), "x")) + require.NoError(t, err, "an injected pool remains owned by its caller") + lease.Release() +} + func TestFulltext2SearchLoad(t *testing.T) { sp, mp := mockSqlProc(t) cfg := testStorageCfg() diff --git a/pkg/fulltext2/search_owner_cancellation_test.go b/pkg/fulltext2/search_owner_cancellation_test.go new file mode 100644 index 0000000000000..a64b608cc9c7b --- /dev/null +++ b/pkg/fulltext2/search_owner_cancellation_test.go @@ -0,0 +1,109 @@ +// Copyright 2026 Matrix Origin +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +package fulltext2 + +import ( + "context" + "fmt" + "testing" + "time" + + "github.com/matrixorigin/matrixone/pkg/util/executor" + "github.com/matrixorigin/matrixone/pkg/vectorindex/sqlexec" + "github.com/stretchr/testify/require" +) + +// Exercise the real Preload/Load call sequence, including SQL whose errors are +// deliberately tolerated (build timestamp and generation capture). Empty Base +// and Tail results isolate cancellation at every metadata stage; storage tests +// separately cover nonempty Base streaming and fallback. +func TestSearchOwnerCancelsEveryLoadSQLStage(t *testing.T) { + for _, preload := range []bool{true, false} { + name := "Load" + if preload { + name = "Preload" + } + t.Run(name, func(t *testing.T) { + var statements []string + t.Run("trace", func(t *testing.T) { + sp, _ := mockSqlProcWithIdentity(t, "complete-load") + owner := newBaseFileOwner(1024, 1) + s := newFulltext2SearchWithBaseOwner(testStorageCfg(), owner) + t.Cleanup(func() { s.Destroy(); require.NoError(t, owner.close()) }) + swapRunSql(t, func(_ *sqlexec.SqlProcess, sql string) (executor.Result, error) { + statements = append(statements, sql) + return executor.Result{}, nil + }) + if preload { + require.NoError(t, s.Preload(sp)) + } else { + require.NoError(t, s.Load(sp)) + } + }) + require.NotEmpty(t, statements) + for target, sql := range statements { + t.Run(fmt.Sprintf("read-%d", target), func(t *testing.T) { + t.Log(sql) + sp, _ := mockSqlProcWithIdentity(t, "complete-load") + ctx, cancel := context.WithCancel(sp.GetTopContext()) + sp = sp.WithContext(ctx) + owner := newBaseFileOwner(1024, 1) + s := newFulltext2SearchWithBaseOwner(testStorageCfg(), owner) + entered, exited, rescue := make(chan struct{}), make(chan struct{}), make(chan struct{}) + done := make(chan error, 1) + n := 0 + swapRunSql(t, func(got *sqlexec.SqlProcess, _ string) (executor.Result, error) { + i := n + n++ + if i != target { + return executor.Result{}, nil + } + close(entered) + select { + case <-got.GetTopContext().Done(): + return executor.Result{}, got.GetTopContext().Err() + case <-rescue: + return executor.Result{}, context.Canceled + } + }) + // Rescue is independent of both owner and propagated context. + // Join before restoring package SQL indirections. + t.Cleanup(func() { + cancel() + close(rescue) + <-exited + s.Destroy() + require.NoError(t, owner.close()) + }) + go func() { + defer close(exited) + if preload { + done <- s.Preload(sp) + } else { + done <- s.Load(sp) + } + }() + select { + case <-entered: + case <-time.After(5 * time.Second): + t.Fatal("target SQL not reached") + } + owner.beginClose() + select { + case err := <-done: + require.ErrorIs(t, err, context.Canceled) + case <-time.After(5 * time.Second): + t.Fatal("shutdown did not cancel complete load") + } + <-exited + require.False(t, s.loaded) + require.False(t, s.preloaded) + require.Nil(t, s.idx) + require.NoError(t, owner.close()) + }) + } + }) + } +} diff --git a/pkg/fulltext2/segment.go b/pkg/fulltext2/segment.go index 0b1af3fe6b0b5..1b77f5a4149e5 100644 --- a/pkg/fulltext2/segment.go +++ b/pkg/fulltext2/segment.go @@ -395,6 +395,16 @@ type Segment struct { // in-memory (tail) segment, whose bytes are GC-managed Go slices. mmapData []byte mmapPath string + // mmapRelease is set only for the experimental Base-file pool path. It + // releases the file lease after this Segment's private mapping is gone. + // Ordinary storage loads leave it nil and keep their existing ownership. + mmapRelease func() + // mmapRetryPool retains this Segment when munmap/unlink fails during Destroy or a + // decode-failure cleanup. The pool is the explicit owner for a later retry; + // it is nil on default ordinary and build-side segments. + mmapRetryPool *baseFilePool + mmapFallbackPinned bool + mmapFileSize int64 } // numDocs is the segment's document count, valid for both a build-side segment (== len(pks)) @@ -684,12 +694,38 @@ func cbAppendNull(k *vectorindex.ColumnBuffer, fixedW int, fixed bool) { // munmap reclaims them — there is no off-heap buffer to deallocate. func (s *Segment) Free() { if s.mmapData != nil { - _ = munmap(s.mmapData) + if err := munmap(s.mmapData); err != nil { + // Keep the mapping, backing path, and lease until a later Free can + // retry. Releasing the pool lease before munmap succeeds could let + // the pool close or reuse the file while this mapping is live. + if s.mmapRetryPool != nil { + s.mmapRetryPool.deferSegment(s) + } + return + } s.mmapData = nil } if s.mmapPath != "" { - _ = os.Remove(s.mmapPath) + if err := os.Remove(s.mmapPath); err != nil && !os.IsNotExist(err) && s.mmapRetryPool != nil { + s.mmapRetryPool.deferSegment(s) + return + } s.mmapPath = "" + s.mmapFileSize = 0 + } + if s.mmapRelease != nil { + release := s.mmapRelease + s.mmapRelease = nil + release() + } + if s.mmapRetryPool != nil { + pool := s.mmapRetryPool + if s.mmapFallbackPinned { + s.mmapFallbackPinned = false + pool.releaseFallback() + } + s.mmapRetryPool = nil + pool.undeferSegment(s) } s.ranking, s.blocks, s.positions = nil, nil, nil } diff --git a/pkg/fulltext2/segment_mmap_test.go b/pkg/fulltext2/segment_mmap_test.go new file mode 100644 index 0000000000000..98d78eca05a62 --- /dev/null +++ b/pkg/fulltext2/segment_mmap_test.go @@ -0,0 +1,69 @@ +// Copyright 2026 Matrix Origin +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +//go:build darwin || linux + +package fulltext2 + +import ( + "errors" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestSegmentFreeRetainsLeaseWhenMunmapFails(t *testing.T) { + original := munmapFn + t.Cleanup(func() { munmapFn = original }) + munmapErr := errors.New("synthetic munmap failure") + munmapFn = func([]byte) error { return munmapErr } + + releases := 0 + s := &Segment{ + mmapData: []byte("mapped"), + mmapRelease: func() { releases++ }, + } + s.Free() + require.NotNil(t, s.mmapData) + require.NotNil(t, s.mmapRelease) + require.Zero(t, releases) + + munmapFn = func([]byte) error { return nil } + s.Free() + require.Nil(t, s.mmapData) + require.Nil(t, s.mmapRelease) + require.Equal(t, 1, releases) +} + +func TestFulltext2SearchDestroyDefersFailedMunmapToPoolOwner(t *testing.T) { + original := munmapFn + t.Cleanup(func() { munmapFn = original }) + munmapErr := errors.New("synthetic destroy munmap failure") + munmapFn = func([]byte) error { return munmapErr } + + pool := newBaseFilePool(1, 1) + releases := 0 + seg := &Segment{ + mmapData: []byte("mapped"), + mmapRelease: func() { releases++ }, + mmapRetryPool: pool, + } + search := newFulltext2SearchWithBasePool(TableConfig{}, pool) + search.basePoolOwned = true + search.idx = NewIndex([]*Segment{seg}, nil) + search.loaded = true + search.Destroy() + require.Nil(t, search.idx) + require.Same(t, pool, search.basePool, "an owned pool remains the retry owner while munmap is failing") + require.True(t, search.basePoolOwned) + require.Equal(t, 1, pool.deferredCount(), "pool owner must retain a failed mapping after Destroy") + require.Zero(t, releases) + + munmapFn = func([]byte) error { return nil } + search.Destroy() + require.Zero(t, pool.deferredCount()) + require.Equal(t, 1, releases) + require.Nil(t, search.basePool) +} diff --git a/pkg/fulltext2/storage.go b/pkg/fulltext2/storage.go index f81c854056a89..4e4e93d31a93a 100644 --- a/pkg/fulltext2/storage.go +++ b/pkg/fulltext2/storage.go @@ -17,6 +17,7 @@ package fulltext2 import ( "bytes" "context" + "errors" "fmt" "math" "os" @@ -118,8 +119,9 @@ type TableConfig struct { // compaction). Production points them at the real executor; tests swap them for fakes // that return synthetic result batches, optionally dispatching by SQL text. var ( - runSql = sqlexec.RunSql - runStreamingSql = sqlexec.RunStreamingSql + runSql = sqlexec.RunSql + runSqlWithContext = sqlexec.RunSqlWithContext + runStreamingSql = sqlexec.RunStreamingSql ) // SubIndexId is the index_id for the i-th tag=0 base sub-index of a build @@ -418,12 +420,24 @@ func DeleteTailSqls(cfg TableConfig) []string { // readMetadata fetches an index id's metadata (checksum + filesize + recency). func readMetadata(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string) (checksum string, filesize, recency int64, found bool, err error) { + return readMetadataWithRunner(sqlproc, cfg, id, func(sql string) (executor.Result, error) { + return runSql(sqlproc, sql) + }) +} + +func readMetadataWithContext(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string, ctx context.Context) (checksum string, filesize, recency int64, found bool, err error) { + return readMetadataWithRunner(sqlproc, cfg, id, func(sql string) (executor.Result, error) { + return runSqlWithContext(ctx, sqlproc, sql) + }) +} + +func readMetadataWithRunner(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string, run func(string) (executor.Result, error)) (checksum string, filesize, recency int64, found bool, err error) { sql := fmt.Sprintf("SELECT %s, %s, %s FROM %s WHERE %s = %s", catalog.FullText2Index_TblCol_Metadata_Checksum, catalog.FullText2Index_TblCol_Metadata_Filesize, catalog.FullText2Index_TblCol_Metadata_Recency, sqlquote.QualifiedIdent(cfg.DbName, cfg.MetadataTable), catalog.FullText2Index_TblCol_Metadata_Index_Id, sqlquote.String(id)) - res, err := runSql(sqlproc, sql) + res, err := run(sql) if err != nil { return "", 0, 0, false, err } @@ -562,11 +576,81 @@ func LoadAllBases(sqlproc *sqlexec.SqlProcess, cfg TableConfig) ([]*Segment, err return bases, nil } +// loadAllBasesWithPool is the opt-in prototype path used only by the internal +// Base-file reuse experiment. It shares immutable files, never decoded +// Segments, with the normal loader. Compaction and direct storage callers stay +// on LoadAllBases. +func loadAllBasesWithPool(sqlproc *sqlexec.SqlProcess, cfg TableConfig, pool *baseFilePool) ([]*Segment, error) { + return loadAllBasesWithPoolOwner(sqlproc, cfg, pool, nil) +} + +func loadAllBasesWithOwner(sqlproc *sqlexec.SqlProcess, cfg TableConfig, owner *baseFileOwner) ([]*Segment, error) { + if owner == nil { + return nil, errBaseFileOwnerClosed + } + pool, err := owner.poolForSearch() + if err != nil { + return nil, err + } + return loadAllBasesWithPoolOwner(sqlproc, cfg, pool, owner) +} + +func loadAllBasesWithPoolOwner(sqlproc *sqlexec.SqlProcess, cfg TableConfig, pool *baseFilePool, owner *baseFileOwner) ([]*Segment, error) { + idSQL := fmt.Sprintf("SELECT %s FROM %s WHERE %s", + catalog.FullText2Index_TblCol_Metadata_Index_Id, + sqlquote.QualifiedIdent(cfg.DbName, cfg.MetadataTable), notTailFrame()) + res, err := runSql(sqlproc, idSQL) + if err != nil { + return nil, err + } + var ids []string + for _, bat := range res.Batches { + if bat == nil { + continue + } + for i := 0; i < bat.RowCount(); i++ { + ids = append(ids, bat.Vecs[0].GetStringAt(i)) + } + } + res.Close() + + bases := make([]*Segment, 0, len(ids)) + for _, id := range ids { + var m *Segment + var lerr error + if owner == nil { + m, lerr = loadFromStorageWithPool(sqlproc, cfg, id, pool) + } else { + m, lerr = loadFromStorageWithOwner(sqlproc, cfg, id, owner, pool) + } + if lerr != nil { + freeSegs(bases) + return nil, lerr + } + bases = append(bases, m) + } + return bases, nil +} + // LoadFromStorage reads an index id's metadata + chunks, verifies the checksum, // and deserializes it. Chunks stream by chunk_id offset into a temp file so the // mpool never holds the whole index. func LoadFromStorage(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string) (*Segment, error) { - checksum, filesize, recency, found, err := readMetadata(sqlproc, cfg, id) + return loadFromStorageWithContext(sqlproc, cfg, id, sqlproc.GetTopContext()) +} + +func loadFromStorageWithContext(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string, ctx context.Context) (*Segment, error) { + return loadFromStorageWithCleanupPool(sqlproc, cfg, id, ctx, nil) +} + +// Only experimental fallback supplies a retry owner. Ordinary default loads +// retain their existing admission and lifecycle semantics. +func loadFromStorageWithCleanupPool(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string, ctx context.Context, retryPool *baseFilePool) (*Segment, error) { + if err := retryPool.mappingAdmissionError(); err != nil { + return nil, err + } + + checksum, filesize, recency, found, err := readMetadataWithContext(sqlproc, cfg, id, ctx) if err != nil { return nil, err } @@ -586,16 +670,20 @@ func LoadFromStorage(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string) (* return nil, err } cleanup := func() { - fp.Close() - if path != "" { - os.Remove(path) + if retryPool != nil { + retryPool.closeHandle(&baseFileHandle{file: fp, path: path}, filesize) + } else { + fp.Close() + if path != "" { + os.Remove(path) + } } } if err = fp.Truncate(filesize); err != nil { cleanup() return nil, err } - if err = streamChunksToFile(sqlproc, cfg, id, filesize, fp); err != nil { + if err = streamChunksToFileContext(ctx, sqlproc, cfg, id, filesize, fp); err != nil { cleanup() return nil, err } @@ -604,26 +692,176 @@ func LoadFromStorage(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string) (* data, err := mmapReadOnly(fp) fp.Close() if err != nil { - if path != "" { + if retryPool != nil { + retryPool.mu.Lock() + retryPool.removeFileLocked(path, filesize) + retryPool.mu.Unlock() + } else if path != "" { os.Remove(path) } return nil, err } + // Install the cleanup owner before validation can fail. A failed unmap + // must remain reachable even when no Segment is returned to Search. + m := &Segment{Id: id, mmapData: data, mmapPath: path, mmapFileSize: filesize, mmapRetryPool: retryPool} + if retryPool != nil { + retryPool.pinFallback() + m.mmapFallbackPinned = true + } // Checksum the mapped bytes (the anonymous SSD file has no path to CheckSum). if vectorindex.CheckSumFromBuffer(data) != checksum { - _ = munmap(data) - if path != "" { - os.Remove(path) + if retryPool == nil { + _ = munmap(data) + if path != "" { + _ = os.Remove(path) + } + } else { + m.Free() } return nil, moerr.NewInternalError(sqlproc.GetContext(), fmt.Sprintf("fulltext2 index %s checksum mismatch", id)) } // The Segment OWNS the mapping (+ path for the /tmp fallback): Free() munmaps and, // if linked, deletes it. - m := &Segment{Id: id, mmapData: data, mmapPath: path} if err := m.decodeSegment(data); err != nil { m.Free() return nil, err } + if retryPool != nil && ctx != nil && ctx.Err() != nil { + m.Free() + return nil, context.Cause(ctx) + } + m.Recency = recency + return m, nil +} + +func loadFromStorageWithPool(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string, pool *baseFilePool) (*Segment, error) { + return loadFromStorageWithPoolOwnerContext(sqlproc, cfg, id, pool, nil, sqlproc.GetTopContext()) +} + +func loadFromStorageWithOwner(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string, fileOwner *baseFileOwner, pool *baseFilePool) (*Segment, error) { + return fileOwner.run(sqlproc.GetTopContext(), func(ctx context.Context) (*Segment, error) { + return loadFromStorageWithPoolOwnerContext(sqlproc, cfg, id, pool, fileOwner, ctx) + }) +} + +func loadFromStorageWithPoolOwnerContext(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string, pool *baseFilePool, fileOwner *baseFileOwner, ctx context.Context) (*Segment, error) { + checksum, filesize, recency, found, err := readMetadataWithContext(sqlproc, cfg, id, ctx) + if err != nil { + return nil, err + } + if !found { + return nil, moerr.NewInternalError(sqlproc.GetContext(), fmt.Sprintf("fulltext2 index %s metadata not found", id)) + } + if filesize <= 0 { + return nil, moerr.NewInternalError(sqlproc.GetContext(), fmt.Sprintf("fulltext2 index %s has empty filesize", id)) + } + owner := sqlproc.GetService() + // Match the tenant used by the SQL executor, including trusted publisher + // and historical-snapshot overrides, not merely the caller's session. + account, accountErr := sqlproc.EffectiveAccountID() + if owner == "" || accountErr != nil || id == "" || checksum == "" { + // A missing durable identity must never turn into account 0 or an empty + // segment key. The optimization is optional, so use the ordinary loader + // when isolation cannot be proved. + if fileOwner != nil && fileOwner.isClosing() { + return nil, errBaseFileOwnerClosed + } + return loadFromStorageWithCleanupPool(sqlproc, cfg, id, ctx, pool) + } + key := baseFileKey{ + owner: owner, account: account, db: cfg.DbName, src: cfg.SrcTable, + index: cfg.IndexTable, metadata: cfg.MetadataTable, pkey: cfg.PKey, + id: id, checksum: checksum, size: filesize, + } + lease, err := pool.acquire(ctx, key, func(fillCtx context.Context) (*baseFileHandle, error) { + fp, path, ferr := createLocalTempFile(sqlproc, "ft2idx_pool") + if ferr != nil { + return nil, ferr + } + if ferr = fp.Truncate(filesize); ferr != nil { + return &baseFileHandle{file: fp, path: path}, ferr + } + if ferr = streamChunksToFileContext(fillCtx, sqlproc, cfg, id, filesize, fp); ferr != nil { + return &baseFileHandle{file: fp, path: path}, ferr + } + // Validate checksum and decodability before publishing READY. The + // validation mapping is short-lived; every successful lease maps the + // immutable file independently. + data, ferr := mmapReadOnly(fp) + if ferr != nil { + return &baseFileHandle{file: fp, path: path}, ferr + } + if vectorindex.CheckSumFromBuffer(data) != checksum { + unmapErr := munmap(data) + if unmapErr != nil { + return &baseFileHandle{file: fp, path: path, validationData: data}, fmt.Errorf("fulltext2 base validation mmap release: %w", unmapErr) + } + return &baseFileHandle{file: fp, path: path}, moerr.NewInternalError(sqlproc.GetContext(), fmt.Sprintf("fulltext2 index %s checksum mismatch", id)) + } + probe := &Segment{Id: id} + ferr = probe.decodeSegment(data) + probe.ranking, probe.blocks, probe.positions = nil, nil, nil + if unmapErr := munmap(data); unmapErr != nil { + // Do not publish a READY file while its validation mapping is still + // owned by this fill attempt. The fill therefore fails closed; the + // ordinary cleanup below only handles the file descriptor/path. + if ferr == nil { + ferr = fmt.Errorf("fulltext2 base validation mmap release: %w", unmapErr) + } + return &baseFileHandle{file: fp, path: path, validationData: data}, ferr + } + if ferr != nil { + return &baseFileHandle{file: fp, path: path}, ferr + } + return &baseFileHandle{file: fp, path: path, validated: true}, nil + }) + if err != nil { + if errors.Is(err, errBaseFilePoolCapacity) || errors.Is(err, errBaseFilePoolReadyCorrupt) { + // Pool admission and a published READY-file failure are optimization + // concerns. Retry the ordinary loader exactly once; it owns its own + // temporary file and never re-enters the pool. + if fileOwner != nil && fileOwner.isClosing() { + return nil, errBaseFileOwnerClosed + } + return loadFromStorageWithCleanupPool(sqlproc, cfg, id, ctx, pool) + } + if errors.Is(err, errBaseFilePoolLeaderCanceled) { + // The leader must observe its own cancellation. A healthy waiter may + // take over once with its own transaction/context, but a canceled + // leader must not resurrect work after its request ended. + if ctx != nil && ctx.Err() != nil { + return nil, context.Cause(ctx) + } + if fileOwner != nil && fileOwner.isClosing() { + return nil, errBaseFileOwnerClosed + } + return loadFromStorageWithCleanupPool(sqlproc, cfg, id, ctx, pool) + } + return nil, err + } + data, err := lease.MapReadOnly() + if err != nil { + lease.Release() + if errors.Is(err, errBaseFilePoolReadyCorrupt) { + if fileOwner != nil && fileOwner.isClosing() { + return nil, errBaseFileOwnerClosed + } + return loadFromStorageWithCleanupPool(sqlproc, cfg, id, ctx, pool) + } + return nil, err + } + m := &Segment{Id: id, mmapData: data, mmapRelease: lease.Release, mmapRetryPool: pool} + if err = m.decodeSegment(data); err != nil { + lease.Invalidate() + m.Free() + // A READY file that cannot be decoded is treated like a damaged cache + // artifact. Retire it and make one ordinary-loader attempt; the source + // error is returned if that attempt also fails. + if fileOwner != nil && fileOwner.isClosing() { + return nil, errBaseFileOwnerClosed + } + return loadFromStorageWithCleanupPool(sqlproc, cfg, id, ctx, pool) + } m.Recency = recency return m, nil } @@ -678,7 +916,8 @@ func createLocalSpillFile(sqlproc *sqlexec.SqlProcess, prefix string) (*os.File, // CreateAndRemoveFile the JOIN spill uses — an ANONYMOUS file (unlinked; the fd + // mapping keep the inode alive, Free just munmaps, no os.Remove). Falls back to // os.CreateTemp (/tmp, linked → Free deletes by path) when no process/fileservice -// is attached (tests / one-shot tools). Returns (file, path, err); path=="" for the +// is attached or LOCAL lookup/directory/file creation fails. Returns +// (file, path, err); path=="" for the // anonymous SSD file. name is the caller-chosen file-name prefix. func createLocalTempFile(sqlproc *sqlexec.SqlProcess, name string) (*os.File, string, error) { if sqlproc != nil && sqlproc.Proc != nil { @@ -942,11 +1181,15 @@ func LoadTailSegmentsWithin(sqlproc *sqlexec.SqlProcess, cfg TableConfig, reserv // streamChunksToFile streams a tag=0 index's chunk rows, writing each at // chunk_id*MaxChunkSize into fp; the assembled bytes must fill filesize exactly. func streamChunksToFile(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string, filesize int64, fp *os.File) error { + return streamChunksToFileContext(sqlproc.GetTopContext(), sqlproc, cfg, id, filesize, fp) +} + +func streamChunksToFileContext(ctx context.Context, sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string, filesize int64, fp *os.File) error { sql := fmt.Sprintf("SELECT %s, %s FROM %s WHERE %s = %s", catalog.FullText2Index_TblCol_Storage_Chunk_Id, catalog.FullText2Index_TblCol_Storage_Data, sqlquote.QualifiedIdent(cfg.DbName, cfg.IndexTable), catalog.FullText2Index_TblCol_Storage_Index_Id, sqlquote.String(id)) - written, _, err := streamChunkRowsToFile(sqlproc, sql, 0, filesize, fp) + written, _, err := streamChunkRowsToFileContext(ctx, sqlproc, sql, 0, filesize, fp) if err != nil { return err } @@ -961,9 +1204,13 @@ func streamChunksToFile(sqlproc *sqlexec.SqlProcess, cfg TableConfig, id string, // at (chunk_id-baseChunk)*MaxChunkSize into fp, bounding the mpool to the stream // buffer. Returns bytes written + chunk-row count. func streamChunkRowsToFile(sqlproc *sqlexec.SqlProcess, sql string, baseChunk, bound int64, fp *os.File) (written, nchunks int64, err error) { + return streamChunkRowsToFileContext(sqlproc.GetTopContext(), sqlproc, sql, baseChunk, bound, fp) +} + +func streamChunkRowsToFileContext(ctx context.Context, sqlproc *sqlexec.SqlProcess, sql string, baseChunk, bound int64, fp *os.File) (written, nchunks int64, err error) { streamCh := make(chan executor.Result, 2) errorCh := make(chan error, 2) - ctx, cancel := context.WithCancelCause(sqlproc.GetTopContext()) + ctx, cancel := context.WithCancelCause(ctx) defer cancel(nil) var wg sync.WaitGroup diff --git a/pkg/fulltext2/storage_mock_test.go b/pkg/fulltext2/storage_mock_test.go index 6ce6adbb5b288..f5d16d9cfe5ff 100644 --- a/pkg/fulltext2/storage_mock_test.go +++ b/pkg/fulltext2/storage_mock_test.go @@ -16,8 +16,12 @@ package fulltext2 import ( "context" + "errors" + "os" "strings" + "sync/atomic" "testing" + "time" "github.com/matrixorigin/matrixone/pkg/common/moerr" "github.com/matrixorigin/matrixone/pkg/common/mpool" @@ -37,8 +41,13 @@ import ( // are mockable. func swapRunSql(t *testing.T, fn func(*sqlexec.SqlProcess, string) (executor.Result, error)) { prev := runSql + prevWithContext := runSqlWithContext runSql = fn + runSqlWithContext = func(_ context.Context, sqlproc *sqlexec.SqlProcess, sql string) (executor.Result, error) { + return fn(sqlproc, sql) + } t.Cleanup(func() { runSql = prev }) + t.Cleanup(func() { runSqlWithContext = prevWithContext }) } func swapRunStreamingSql(t *testing.T, fn func(context.Context, *sqlexec.SqlProcess, string, chan executor.Result, chan error) (executor.Result, error)) { @@ -98,6 +107,17 @@ func mockSqlProc(t *testing.T) (*sqlexec.SqlProcess, *mpool.MPool) { return sqlexec.NewSqlProcess(proc), proc.Mp() } +// mockSqlProcWithIdentity uses the background SQL context so the optional +// Base-file pool can exercise its durable owner/account key. The ordinary +// process helper intentionally has no lock service and therefore represents +// the fail-closed identity-missing path. +func mockSqlProcWithIdentity(t *testing.T, service string) (*sqlexec.SqlProcess, *mpool.MPool) { + _, mp := mockSqlProc(t) + sp := sqlexec.NewSqlProcessWithContext( + sqlexec.NewSqlContext(context.Background(), service, nil, 0, nil)) + return sp, mp +} + func TestReadMetadata(t *testing.T) { sp, mp := mockSqlProc(t) cfg := testStorageCfg() @@ -129,6 +149,181 @@ func TestReadMetadata(t *testing.T) { require.Error(t, err) } +func TestBaseFileOwnerCloseCancelsOrdinaryFallback(t *testing.T) { + requestCtx, requestCancel := context.WithCancel(context.Background()) + _, mp := mockSqlProc(t) + sp := sqlexec.NewSqlProcessWithContext( + sqlexec.NewSqlContext(requestCtx, "fallback-close", nil, 0, nil)) + cfg := testStorageCfg() + const filesize = int64(8) + + // Make pool admission fail before any pooled fill starts. The loader must + // then use the ordinary path, but that path remains an admitted owner + // operation and must observe owner shutdown while streaming source bytes. + swapRunSql(t, func(_ *sqlexec.SqlProcess, _ string) (executor.Result, error) { + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, "fallback-checksum", filesize, 7)}}, nil + }) + streamStarted := make(chan struct{}) + streamCanceled := make(chan struct{}) + swapRunStreamingSql(t, func(ctx context.Context, _ *sqlexec.SqlProcess, _ string, _ chan executor.Result, _ chan error) (executor.Result, error) { + close(streamStarted) + <-ctx.Done() + close(streamCanceled) + return executor.Result{}, ctx.Err() + }) + + owner := newBaseFileOwner(1, 1) + loadDone := make(chan error, 1) + closeDone := make(chan error, 1) + loadExited := make(chan struct{}) + closeExited := make(chan struct{}) + var closeStarted atomic.Bool + t.Cleanup(func() { + // This is an independent rescue path: it must release the request even + // when the owner cancellation under test is broken. Wait for every + // goroutine before the mock restorers registered above run. + requestCancel() + wait := func(name string, done <-chan struct{}) bool { + select { + case <-done: + return true + case <-time.After(5 * time.Second): + t.Errorf("%s goroutine did not terminate during cleanup", name) + return false + } + } + loadStopped := wait("ordinary fallback", loadExited) + if closeStarted.Load() { + wait("owner close", closeExited) + } else if loadStopped { + if err := owner.close(); err != nil && !errors.Is(err, errBaseFileOwnerPending) { + t.Errorf("cleanup owner close: %v", err) + } + } + }) + go func() { + defer close(loadExited) + _, err := loadFromStorageWithOwner(sp, cfg, "seg-fallback", owner, owner.pool) + loadDone <- err + }() + select { + case <-streamStarted: + case <-time.After(2 * time.Second): + t.Fatal("ordinary fallback did not reach source streaming") + } + + closeStarted.Store(true) + go func() { + defer close(closeExited) + closeDone <- owner.close() + }() + select { + case err := <-closeDone: + require.NoError(t, err) + case <-time.After(2 * time.Second): + t.Fatal("owner close waited for an ordinary fallback that ignored cancellation") + } + select { + case <-streamCanceled: + case <-time.After(2 * time.Second): + t.Fatal("owner close did not cancel the ordinary loader stream") + } + require.ErrorIs(t, <-loadDone, context.Canceled) +} + +func TestBaseFileOwnerCloseCancelsMetadataReadAndPreservesIdentity(t *testing.T) { + requestCtx, requestCancel := context.WithCancel(context.Background()) + _, mp := mockSqlProc(t) + sp := sqlexec.NewSqlProcessWithContext( + sqlexec.NewSqlContext(requestCtx, "metadata-close", nil, 7, nil)) + cfg := testStorageCfg() + metadataStarted := make(chan struct{}) + metadataCanceled := make(chan struct{}) + identity := make(chan *sqlexec.SqlProcess, 1) + prevWithContext := runSqlWithContext + runSqlWithContext = func(ctx context.Context, got *sqlexec.SqlProcess, _ string) (executor.Result, error) { + identity <- got + close(metadataStarted) + <-ctx.Done() + close(metadataCanceled) + return executor.Result{Mp: mp}, ctx.Err() + } + t.Cleanup(func() { runSqlWithContext = prevWithContext }) + + owner := newBaseFileOwner(1, 1) + loadDone := make(chan error, 1) + closeDone := make(chan error, 1) + loadExited := make(chan struct{}) + closeExited := make(chan struct{}) + var closeStarted atomic.Bool + t.Cleanup(func() { + // Request cancellation is independent of owner.close and is the rescue + // path if the metadata propagation under test fails before shutdown can + // start. + requestCancel() + loadStopped := false + select { + case <-loadExited: + loadStopped = true + case <-time.After(5 * time.Second): + t.Errorf("metadata fallback did not terminate during cleanup") + } + if !loadStopped { + return + } + if closeStarted.Load() { + select { + case <-closeExited: + case <-time.After(5 * time.Second): + t.Errorf("owner close did not terminate during cleanup") + } + } else { + if err := owner.close(); err != nil && !errors.Is(err, errBaseFileOwnerPending) { + t.Errorf("cleanup owner close: %v", err) + } + } + }) + go func() { + defer close(loadExited) + _, err := loadFromStorageWithOwner(sp, cfg, "seg-metadata-fallback", owner, owner.pool) + loadDone <- err + }() + select { + case <-metadataStarted: + case <-time.After(2 * time.Second): + t.Fatal("metadata read did not start") + } + got := <-identity + require.Same(t, sp, got, "metadata executor must receive the original SqlProcess") + require.Equal(t, "metadata-close", got.GetService()) + account, err := got.GetAccountID() + require.NoError(t, err) + require.Equal(t, uint32(7), account) + + closeStarted.Store(true) + go func() { + defer close(closeExited) + closeDone <- owner.close() + }() + select { + case err := <-closeDone: + require.NoError(t, err) + case <-time.After(2 * time.Second): + t.Fatal("owner close did not finish") + } + select { + case err := <-loadDone: + require.ErrorIs(t, err, context.Canceled) + case <-time.After(2 * time.Second): + t.Fatal("metadata load did not finish after owner close") + } + select { + case <-metadataCanceled: + case <-time.After(2 * time.Second): + t.Fatal("owner close did not cancel metadata executor context") + } +} + func TestScanHelpers(t *testing.T) { sp, mp := mockSqlProc(t) cfg := testStorageCfg() @@ -382,6 +577,340 @@ func TestLoadFromStorageRoundTrip(t *testing.T) { require.ErrorContains(t, err, "metadata not found") } +// TestLoadFromStoragePoolReusesOnlyTheImmutableBaseFile proves the prototype's +// ownership boundary: the second load skips chunk materialization, but receives +// a distinct mmap/Segment and therefore cannot share decoded state or liveness. +func TestLoadFromStoragePoolUsesExecutionTenant(t *testing.T) { + sp, mp := mockSqlProcWithIdentity(t, "tenant-pool") + b := NewBuilder("seg0", int32(types.T_int64)) + feed(t, b, int64(1), "hello") + seg, err := b.Finish() + require.NoError(t, err) + t.Cleanup(seg.Free) + buf, err := seg.Serialize() + require.NoError(t, err) + size := int64(len(buf)) + pool := newBaseFilePool(4*size, 4) + t.Cleanup(pool.Close) + var reads int + swapRunSql(t, func(_ *sqlexec.SqlProcess, _ string) (executor.Result, error) { + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, vectorindex.CheckSumFromBuffer(buf), size, 1)}}, nil + }) + swapRunStreamingSql(t, func(_ context.Context, _ *sqlexec.SqlProcess, _ string, sc chan executor.Result, _ chan error) (executor.Result, error) { + reads++ + sc <- executor.Result{Mp: mp, Batches: []*batch.Batch{chunkBatch(mp, buf)}} + return executor.Result{}, nil + }) + // One calling session reads two publishers. Even identical physical names + // and bytes must not reuse a file across the effective tenant boundary. + for _, account := range []uint32{42, 43, 42} { + func() { + loaded, err := loadFromStorageWithPool(sp.WithExecutionIdentity(account, "publisher"), testStorageCfg(), "seg0", pool) + require.NoError(t, err) + defer loaded.Free() + require.Equal(t, int64(1), loaded.N) + }() + } + require.Equal(t, 2, reads) +} + +func TestLoadFromStoragePoolReusesOnlyTheImmutableBaseFile(t *testing.T) { + if !experimentalBaseFileReuseEnabled { + t.Skip("Base-file pool is only enabled in the experimental build") + } + sp, mp := mockSqlProcWithIdentity(t, "test-service") + cfg := testStorageCfg() + b := NewBuilder("seg0", int32(types.T_int64)) + feed(t, b, int64(1), "hello", "world") + feed(t, b, int64(2), "hello", "matrix") + seg, err := b.Finish() + require.NoError(t, err) + seg.Id = "seg0" + buf, err := seg.Serialize() + require.NoError(t, err) + checksum := vectorindex.CheckSumFromBuffer(buf) + filesize := int64(len(buf)) + var streamCalls atomic.Int32 + swapRunSql(t, func(_ *sqlexec.SqlProcess, _ string) (executor.Result, error) { + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, checksum, filesize, 7)}}, nil + }) + swapRunStreamingSql(t, func(_ context.Context, _ *sqlexec.SqlProcess, _ string, sc chan executor.Result, _ chan error) (executor.Result, error) { + streamCalls.Add(1) + sc <- executor.Result{Mp: mp, Batches: []*batch.Batch{chunkBatch(mp, buf)}} + return executor.Result{}, nil + }) + + pool := newBaseFilePool(filesize*2, 2) + first, err := loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err) + second, err := loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err) + require.Equal(t, int32(1), streamCalls.Load(), "the immutable Base file is materialized once") + require.NotSame(t, first, second) + require.NotEmpty(t, first.mmapData) + require.NotEmpty(t, second.mmapData) + require.False(t, &first.mmapData[0] == &second.mmapData[0], "each Segment gets an independent mapping") + firstPosting, ok := first.LookupLoaded("hello") + require.True(t, ok) + secondPosting, ok := second.LookupLoaded("hello") + require.True(t, ok) + require.Equal(t, firstPosting.materializeDocIDs(), secondPosting.materializeDocIDs()) + first.Free() + second.Free() + fallbackPool := newBaseFilePool(filesize-1, 1) + streamCalls.Store(0) + fallback, err := loadFromStorageWithPool(sp, cfg, "seg0", fallbackPool) + require.NoError(t, err, "pool admission must fall back to ordinary loading") + require.Nil(t, fallback.mmapRelease, "a capacity fallback must not retain a pool lease") + fallback.Free() + fallbackPool.Close() + require.Equal(t, int32(1), streamCalls.Load(), "ordinary fallback still materializes the source once") + pool.Close() +} + +func TestBaseFileReuseSurvivesSearchHandleDestroy(t *testing.T) { + if !experimentalBaseFileReuseEnabled { + t.Skip("Base-file pool is only enabled in the experimental build") + } + sp, mp := mockSqlProcWithIdentity(t, "test-service") + cfg := testStorageCfg() + b := NewBuilder("seg0", int32(types.T_int64)) + feed(t, b, int64(1), "hello", "world") + feed(t, b, int64(2), "hello", "matrix") + seg, err := b.Finish() + require.NoError(t, err) + seg.Id = "seg0" + buf, err := seg.Serialize() + require.NoError(t, err) + checksum := vectorindex.CheckSumFromBuffer(buf) + filesize := int64(len(buf)) + var streamCalls atomic.Int32 + swapRunSql(t, func(_ *sqlexec.SqlProcess, _ string) (executor.Result, error) { + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, checksum, filesize, 7)}}, nil + }) + swapRunStreamingSql(t, func(_ context.Context, _ *sqlexec.SqlProcess, _ string, sc chan executor.Result, _ chan error) (executor.Result, error) { + streamCalls.Add(1) + sc <- executor.Result{Mp: mp, Batches: []*batch.Batch{chunkBatch(mp, buf)}} + return executor.Result{}, nil + }) + + pool := newBaseFilePool(filesize*2, 2) + firstSearch := newFulltext2SearchWithBasePool(cfg, pool) + firstSeg, err := loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err) + firstSearch.idx = NewIndex([]*Segment{firstSeg}, nil) + firstSearch.loaded = true + firstSearch.Destroy() + require.Nil(t, firstSearch.basePool) + + secondSearch := newFulltext2SearchWithBasePool(cfg, pool) + secondSeg, err := loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err) + secondSearch.idx = NewIndex([]*Segment{secondSeg}, nil) + secondSearch.loaded = true + require.Equal(t, int32(1), streamCalls.Load(), "a caller-owned pool survives search-handle destruction") + secondSearch.Destroy() + pool.Close() +} + +func TestBaseFileOwnerSurvivesSearchHandleDestroy(t *testing.T) { + if !experimentalBaseFileReuseEnabled { + t.Skip("Base-file owner is only enabled in the experimental build") + } + sp, mp := mockSqlProcWithIdentity(t, "test-service") + cfg := testStorageCfg() + b := NewBuilder("seg0", int32(types.T_int64)) + feed(t, b, int64(1), "hello", "world") + feed(t, b, int64(2), "hello", "matrix") + seg, err := b.Finish() + require.NoError(t, err) + seg.Id = "seg0" + buf, err := seg.Serialize() + require.NoError(t, err) + checksum := vectorindex.CheckSumFromBuffer(buf) + filesize := int64(len(buf)) + var streamCalls atomic.Int32 + swapRunSql(t, func(_ *sqlexec.SqlProcess, _ string) (executor.Result, error) { + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, checksum, filesize, 7)}}, nil + }) + swapRunStreamingSql(t, func(_ context.Context, _ *sqlexec.SqlProcess, _ string, sc chan executor.Result, _ chan error) (executor.Result, error) { + streamCalls.Add(1) + sc <- executor.Result{Mp: mp, Batches: []*batch.Batch{chunkBatch(mp, buf)}} + return executor.Result{}, nil + }) + + owner := newBaseFileOwner(filesize*2, 2) + pool, err := owner.poolForSearch() + require.NoError(t, err) + firstSearch := newFulltext2SearchWithBaseOwner(cfg, owner) + firstSeg, err := loadFromStorageWithOwner(sp, cfg, "seg0", owner, pool) + require.NoError(t, err) + firstSearch.idx = NewIndex([]*Segment{firstSeg}, nil) + firstSearch.loaded = true + firstSearch.Destroy() + require.Same(t, owner, firstSearch.baseOwner, "a Search retry must retain its service owner") + require.False(t, firstSearch.baseOwnerClosed, "an open owner remains usable after ordinary Destroy") + + secondSearch := newFulltext2SearchWithBaseOwner(cfg, owner) + secondSeg, err := loadFromStorageWithOwner(sp, cfg, "seg0", owner, pool) + require.NoError(t, err) + secondSearch.idx = NewIndex([]*Segment{secondSeg}, nil) + secondSearch.loaded = true + require.Equal(t, int32(1), streamCalls.Load(), "the service owner survives the first Search destruction") + secondSearch.Destroy() + require.NoError(t, owner.close()) +} + +func TestLoadFromStoragePoolInvalidatesCorruptReadyFile(t *testing.T) { + if !experimentalBaseFileReuseEnabled { + t.Skip("Base-file pool is only enabled in the experimental build") + } + sp, mp := mockSqlProcWithIdentity(t, "test-service") + cfg := testStorageCfg() + b := NewBuilder("seg0", int32(types.T_int64)) + feed(t, b, int64(1), "hello", "world") + feed(t, b, int64(2), "hello", "matrix") + seg, err := b.Finish() + require.NoError(t, err) + seg.Id = "seg0" + buf, err := seg.Serialize() + require.NoError(t, err) + checksum := vectorindex.CheckSumFromBuffer(buf) + filesize := int64(len(buf)) + var streamCalls atomic.Int32 + swapRunSql(t, func(_ *sqlexec.SqlProcess, _ string) (executor.Result, error) { + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, checksum, filesize, 7)}}, nil + }) + swapRunStreamingSql(t, func(_ context.Context, _ *sqlexec.SqlProcess, _ string, sc chan executor.Result, _ chan error) (executor.Result, error) { + streamCalls.Add(1) + sc <- executor.Result{Mp: mp, Batches: []*batch.Batch{chunkBatch(mp, buf)}} + return executor.Result{}, nil + }) + + pool := newBaseFilePool(filesize*2, 2) + loaded, err := loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err) + loaded.Free() + pool.mu.Lock() + var pooledFile *os.File + for _, entry := range pool.entries { + pooledFile = entry.handle.file + break + } + pool.mu.Unlock() + require.NotNil(t, pooledFile) + _, err = pooledFile.WriteAt([]byte{buf[0] ^ 0xff}, 0) + require.NoError(t, err) + + reloaded, err := loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err, "a corrupt READY hit falls back to one ordinary source load") + reloaded.Free() + require.Equal(t, int32(2), streamCalls.Load(), "the corrupt READY hit must retire before one fallback fill") + // The ordinary fallback deliberately does not publish a READY entry. The + // following request therefore misses the pool and refills it. + reloaded, err = loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err) + reloaded.Free() + require.Equal(t, int32(3), streamCalls.Load()) + + pool.mu.Lock() + pooledFile = nil + for _, entry := range pool.entries { + pooledFile = entry.handle.file + break + } + pool.mu.Unlock() + require.NotNil(t, pooledFile) + require.NoError(t, pooledFile.Truncate(filesize+1)) + reloaded, err = loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err, "a size-corrupt READY hit falls back to one ordinary source load") + reloaded.Free() + require.Equal(t, int32(4), streamCalls.Load()) + reloaded, err = loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err) + reloaded.Free() + require.Equal(t, int32(5), streamCalls.Load()) + pool.Close() +} + +func TestLoadFromStoragePoolValidationMunmapFailureRetainsOwner(t *testing.T) { + if !experimentalBaseFileReuseEnabled { + t.Skip("Base-file pool is only enabled in the experimental build") + } + sp, mp := mockSqlProcWithIdentity(t, "test-service") + cfg := testStorageCfg() + b := NewBuilder("seg0", int32(types.T_int64)) + feed(t, b, int64(1), "hello", "world") + seg, err := b.Finish() + require.NoError(t, err) + seg.Id = "seg0" + buf, err := seg.Serialize() + require.NoError(t, err) + checksum := vectorindex.CheckSumFromBuffer(buf) + filesize := int64(len(buf)) + var streamCalls atomic.Int32 + swapRunSql(t, func(_ *sqlexec.SqlProcess, _ string) (executor.Result, error) { + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, checksum, filesize, 7)}}, nil + }) + swapRunStreamingSql(t, func(_ context.Context, _ *sqlexec.SqlProcess, _ string, sc chan executor.Result, _ chan error) (executor.Result, error) { + streamCalls.Add(1) + sc <- executor.Result{Mp: mp, Batches: []*batch.Batch{chunkBatch(mp, buf)}} + return executor.Result{}, nil + }) + + pool := newBaseFilePool(filesize*2, 2) + original := munmapFn + t.Cleanup(func() { munmapFn = original }) + munmapFn = func(data []byte) error { + return errors.New("synthetic validation munmap failure") + } + _, err = loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.ErrorContains(t, err, "validation mmap release") + require.Equal(t, 1, pool.deferredCount(), "the failed validation mapping must remain owned by the pool") + + munmapFn = original + pool.retryDeferred() + require.Zero(t, pool.deferredCount()) + reloaded, err := loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.NoError(t, err) + reloaded.Free() + require.Equal(t, int32(2), streamCalls.Load()) + pool.Close() +} + +func TestLoadFromStoragePoolDecodeFailureRetainsValidationOwner(t *testing.T) { + if !experimentalBaseFileReuseEnabled { + t.Skip("Base-file pool is only enabled in the experimental build") + } + sp, mp := mockSqlProcWithIdentity(t, "test-service") + cfg := testStorageCfg() + bad := []byte("not a serialized fulltext2 segment") + checksum := vectorindex.CheckSumFromBuffer(bad) + filesize := int64(len(bad)) + swapRunSql(t, func(_ *sqlexec.SqlProcess, _ string) (executor.Result, error) { + return executor.Result{Mp: mp, Batches: []*batch.Batch{metaBatch(mp, checksum, filesize, 7)}}, nil + }) + swapRunStreamingSql(t, func(_ context.Context, _ *sqlexec.SqlProcess, _ string, sc chan executor.Result, _ chan error) (executor.Result, error) { + sc <- executor.Result{Mp: mp, Batches: []*batch.Batch{chunkBatch(mp, bad)}} + return executor.Result{}, nil + }) + + pool := newBaseFilePool(filesize*2, 1) + original := munmapFn + t.Cleanup(func() { munmapFn = original }) + munmapFn = func([]byte) error { + return errors.New("synthetic validation munmap failure") + } + _, err := loadFromStorageWithPool(sp, cfg, "seg0", pool) + require.Error(t, err, "the malformed validation blob must fail before READY publication") + require.Equal(t, 1, pool.deferredCount(), "decode failure must retain its validation mapping for the pool owner") + + munmapFn = original + pool.retryDeferred() + require.Zero(t, pool.deferredCount()) + pool.Close() +} + // tailChunkBatch renders forged (chunk_id, data) rows as the tail-data result batch. func tailChunkBatch(mp *mpool.MPool, chunks []TailChunk) *batch.Batch { b := batch.NewWithSize(2) diff --git a/pkg/sql/colexec/table_function/fulltext2_search.go b/pkg/sql/colexec/table_function/fulltext2_search.go index 6065cbc94cfbe..0cb7d67d37e33 100644 --- a/pkg/sql/colexec/table_function/fulltext2_search.go +++ b/pkg/sql/colexec/table_function/fulltext2_search.go @@ -740,7 +740,7 @@ func (u *fulltext2SearchState) start(tf *TableFunction, proc *process.Process, n } } - newsearch := fulltext2.NewFulltext2Search(u.tblcfg) + newsearch := fulltext2.NewFulltext2SearchForExecution(u.tblcfg, proc.GetService()) q := fulltext2.Fulltext2Query{ ScoreRange: scoreRange, Pattern: []byte(pattern), diff --git a/pkg/tests/fulltextreuse/lifecycle_test.go b/pkg/tests/fulltextreuse/lifecycle_test.go new file mode 100644 index 0000000000000..fe54d21f60a1f --- /dev/null +++ b/pkg/tests/fulltextreuse/lifecycle_test.go @@ -0,0 +1,311 @@ +//go:build fulltext2_base_file_reuse + +// Copyright 2026 Matrix Origin +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. + +package fulltextreuse + +import ( + "context" + "database/sql" + "fmt" + "math" + "strings" + "testing" + "time" + + _ "github.com/go-sql-driver/mysql" + "github.com/matrixorigin/matrixone/pkg/clusterservice" + "github.com/matrixorigin/matrixone/pkg/embed" + "github.com/matrixorigin/matrixone/pkg/fulltext2" + "github.com/matrixorigin/matrixone/pkg/iscp" + "github.com/matrixorigin/matrixone/pkg/pb/metadata" + "github.com/matrixorigin/matrixone/pkg/tests/testutils" + veccache "github.com/matrixorigin/matrixone/pkg/vectorindex/cache" + "github.com/stretchr/testify/require" +) + +// This is a real embedded SQL/CN lifecycle test, not an owner helper test. +// CloseComplete on the actual CN includes the experimental owner's empty-resource +// check. TN/LogService remain alive while the CN closes and restarts. +func TestSQLWarmSearchCNCloseRestart(t *testing.T) { + c, err := embed.NewCluster(embed.WithTesting(), embed.WithCNCount(1), embed.WithPreStart(func(op embed.ServiceOperator) { + if op.ServiceType() == metadata.ServiceType_CN { + op.Adjust(func(cfg *embed.ServiceConfig) { cfg.CN.AutomaticUpgrade = true }) + } + })) + require.NoError(t, err) + t.Cleanup(func() { require.NoError(t, c.Close()) }) + require.NoError(t, c.Start()) + cn, err := c.GetCNService(0) + require.NoError(t, err) + waitDiscovery := func() { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + cluster, err := clusterservice.GetMOClusterWithContext(ctx, cn.ServiceID()) + require.NoError(t, err) + refresh, ok := cluster.(clusterservice.AuthoritativeRefresher) + require.True(t, ok) + tick := time.NewTicker(100 * time.Millisecond) + defer tick.Stop() + for { + require.NoError(t, refresh.Refresh(ctx)) + ready := false + err := clusterservice.GetCNServiceWithoutWorkingStateWithContext(ctx, cluster, clusterservice.NewServiceIDSelector(cn.ServiceID()), func(cn metadata.CNService) bool { + ready = cn.QueryAddress != "" + return false + }) + require.NoError(t, err) + if ready { + return + } + select { + case <-ctx.Done(): + t.Fatal("CN query address did not become discoverable") + case <-tick.C: + } + } + } + waitDiscovery() + open := func() *sql.DB { + db, err := sql.Open("mysql", fmt.Sprintf("dump:111@tcp(127.0.0.1:%d)/?timeout=10s&readTimeout=60s&writeTimeout=60s", cn.GetServiceConfig().CN.Frontend.Port)) + require.NoError(t, err) + db.SetMaxOpenConns(1) + t.Cleanup(func() { _ = db.Close() }) + return db + } + db := open() + readyCtx, readyCancel := context.WithTimeout(context.Background(), time.Minute) + err = testutils.WaitSystemBootstrap(readyCtx, db) + readyCancel() + require.NoError(t, err) + waitISCP := func() { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + tick := time.NewTicker(100 * time.Millisecond) + defer tick.Stop() + for { + if executor, ok := iscp.GetExecutorRuntime(cn.ServiceID()); ok && executor != nil { + return + } + select { + case <-ctx.Done(): + t.Fatal("CN ISCP executor did not become ready") + case <-tick.C: + } + } + } + waitISCP() + exec := func(statement string) { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + _, err := db.ExecContext(ctx, statement) + require.NoError(t, err, statement) + } + exec("set experimental_fulltext2_index = 1") + exec("create database ft2_owner_lifecycle") + exec("use ft2_owner_lifecycle") + exec("create table docs(id bigint primary key, body text)") + exec("insert into docs values (1,'alpha beta gamma'),(2,'beta gamma delta'),(3,'gamma delta epsilon'),(4,'alpha zeta')") + exec("create fulltext2 index ft on docs(body) max_index_capacity 2") + relation := "docs" + query := func() map[int64]uint32 { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + rows, err := db.QueryContext(ctx, "select id, match(body) against('alpha' in bm25 mode) from "+relation+" where match(body) against('alpha' in bm25 mode) order by id") + require.NoError(t, err) + defer rows.Close() + result := make(map[int64]uint32) + for rows.Next() { + var pk int64 + var score float64 + require.NoError(t, rows.Scan(&pk, &score)) + result[pk] = math.Float32bits(float32(score)) + } + require.NoError(t, rows.Err()) + require.Len(t, result, 2) + return result + } + resident := func() map[string]*fulltext2.Fulltext2Search { + found := make(map[string]*fulltext2.Fulltext2Search) + veccache.Cache.IndexMap.Range(func(key, value any) bool { + entry, ok := value.(*veccache.VectorIndexSearch) + if !ok { + return true + } + entry.Mutex.RLock() + defer entry.Mutex.RUnlock() + if search, ok := entry.Algo.(*fulltext2.Fulltext2Search); ok && search.CacheServiceID() == cn.ServiceID() && entry.Status.Load() == veccache.STATUS_LOADED { + found[key.(string)] = search + } + return true + }) + return found + } + want := query() + require.Contains(t, want, int64(1)) + require.Contains(t, want, int64(4)) + first := resident() + require.NotEmpty(t, first, "SQL must leave a real experimental Search resident") + require.Equal(t, want, query()) + require.Equal(t, first, resident(), "warm query must reuse Search handles") + // Drive a real CDC Tail, not a no-op MERGE of an unchanged source. Keep + // alpha TF and document length unchanged so its cross-phase score oracle + // remains valid after the update is made live. + require.Len(t, first, 1) + var storage string + for key := range first { + storage = key + } + quote := func(name string) string { return "`" + strings.ReplaceAll(name, "`", "``") + "`" } + var metadataTable string + func() { + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + err := db.QueryRowContext(ctx, "select distinct index_table_name from mo_catalog.mo_indexes where name='ft' and algo_table_type='ftv2_meta' and table_id in (select rel_id from mo_catalog.mo_tables where reldatabase=database() and relname='docs')").Scan(&metadataTable) + require.NoError(t, err) + }() + baseIDs := func() []string { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + rows, err := db.QueryContext(ctx, "select index_id from "+quote(metadataTable)+" where not prefix_eq(index_id, 'cdc_tail:') order by index_id") + require.NoError(t, err) + defer rows.Close() + var ids []string + for rows.Next() { + var id string + require.NoError(t, rows.Scan(&id)) + ids = append(ids, id) + } + require.NoError(t, rows.Err()) + require.NotEmpty(t, ids) + return ids + } + beforeMerge := baseIDs() + exec("update docs set body='alpha beta changed' where id=1") + func() { + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + ticker := time.NewTicker(100 * time.Millisecond) + defer ticker.Stop() + for { + var n int64 + err := db.QueryRowContext(ctx, "select count(*) from "+quote(storage)+" where tag=1").Scan(&n) + require.NoError(t, err) + if n > 0 { + return + } + select { + case <-ctx.Done(): + t.Fatal("CDC Tail was not persisted") + case <-ticker.C: + } + } + }() + require.Equal(t, first, resident(), "CDC flush must not evict the warm Search") + require.Equal(t, beforeMerge, baseIDs(), "Tail update must keep the Base build identity") + termIDs := func(term string) []int64 { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + rows, err := db.QueryContext(ctx, "select id from docs where match(body) against('"+term+"' in bm25 mode) order by id") + require.NoError(t, err) + defer rows.Close() + ids := []int64{} + for rows.Next() { + var id int64 + require.NoError(t, rows.Scan(&id)) + ids = append(ids, id) + } + require.NoError(t, rows.Err()) + return ids + } + require.Empty(t, termIDs("changed"), "resident Search keeps its original Tail") + require.Equal(t, []int64{1, 2, 3}, termIDs("gamma")) + // Release only this service's observed entries, not the process-wide cache. + for key := range first { + veccache.Cache.Remove(key) + } + require.Equal(t, want, query()) + for key, search := range resident() { + require.NotSame(t, first[key], search) + } + require.Equal(t, beforeMerge, baseIDs(), "reload must reuse the same published Base identity") + require.Equal(t, []int64{1}, termIDs("changed"), "new Search sees the persisted Tail term") + require.Equal(t, []int64{2, 3}, termIDs("gamma"), "new Search suppresses the replaced old term") + // File-hit and capacity mechanisms are asserted separately by real-materializer + // component tests; SQL here asserts Search replacement and Tail visibility. + // Rebuild and merge must select newly published metadata, not a prior Base. + exec("alter table docs alter reindex ft fulltext2 merge force_sync") + require.NotEqual(t, beforeMerge, baseIDs(), "MERGE must publish a new Base build") + require.Equal(t, want, query()) + exec("alter table docs alter reindex ft fulltext2 force_sync") + require.Equal(t, want, query()) + exec("create snapshot ft2_owner_snapshot for table ft2_owner_lifecycle docs") + require.NotEmpty(t, resident()) + require.NoError(t, db.Close()) + old := cn.RawService().(interface { + Close() error + CloseComplete() bool + }) + require.NoError(t, cn.Close()) + require.True(t, old.CloseComplete(), "normal warm residence must not leave a pinned owner") + require.Empty(t, resident()) + require.NoError(t, cn.Start()) + waitDiscovery() + waitISCP() + require.NoError(t, old.Close(), "old service repeated close must not close the new generation") + db = open() + exec("set experimental_fulltext2_index = 1") + exec("use ft2_owner_lifecycle") + require.Equal(t, want, query()) + require.NotEmpty(t, resident()) + // Same SQL table/index names with different contents are a new incarnation. + exec("drop table docs") + exec("create table docs(id bigint primary key, body text)") + exec("insert into docs values (11,'alpha beta'),(12,'alpha gamma'),(13,'delta')") + exec("create fulltext2 index ft on docs(body) max_index_capacity 2") + got := query() + require.Contains(t, got, int64(11)) + require.Contains(t, got, int64(12)) + require.NotContains(t, got, int64(1)) + relation = "docs{snapshot='ft2_owner_snapshot'}" + require.Equal(t, want, query(), "historical index must not use the recreated table's Base") + relation = "docs" + exec("restore table ft2_owner_lifecycle.docs{snapshot='ft2_owner_snapshot'}") + got = query() + require.Equal(t, want, got, "restored table must use its restored index metadata") + exec("drop snapshot ft2_owner_snapshot") + // A second real account uses the same logical names but a disjoint corpus. + // The sessions stay distinct; switching db below only selects the client + // used by the test helpers, never an execution identity override. + exec("create account ft2_owner_tenant admin_name 'root' identified by '111'") + sysDB := db + tenantDB, err := sql.Open("mysql", fmt.Sprintf("ft2_owner_tenant#root#accountadmin:111@tcp(127.0.0.1:%d)/?timeout=10s&readTimeout=60s&writeTimeout=60s", cn.GetServiceConfig().CN.Frontend.Port)) + require.NoError(t, err) + t.Cleanup(func() { _ = tenantDB.Close() }) + tenantDB.SetMaxOpenConns(1) + db = tenantDB + exec("set experimental_fulltext2_index = 1") + exec("create database ft2_owner_lifecycle") + exec("use ft2_owner_lifecycle") + exec("create table docs(id bigint primary key, body text)") + exec("insert into docs values (21,'alpha beta'),(22,'alpha gamma'),(23,'delta')") + exec("create fulltext2 index ft on docs(body) max_index_capacity 2") + tenantResult := query() + require.Contains(t, tenantResult, int64(21)) + require.Contains(t, tenantResult, int64(22)) + require.NotContains(t, tenantResult, int64(11)) + db = sysDB + require.Equal(t, got, query()) + require.NoError(t, tenantDB.Close()) + require.NoError(t, db.Close()) + require.NoError(t, cn.Close()) + require.Empty(t, resident()) +} diff --git a/pkg/vectorindex/cache/cache.go b/pkg/vectorindex/cache/cache.go index 9a86f1b31bc5f..0e1131a4fa6fe 100644 --- a/pkg/vectorindex/cache/cache.go +++ b/pkg/vectorindex/cache/cache.go @@ -131,7 +131,8 @@ func IsRetryableLoadError(err error) bool { var lifecycleHooks struct { sync.RWMutex - hooks []func(shutdown bool) + hooks []func(shutdown bool) + beforeShutdown []func(shutdown bool) } // RegisterLifecycleHook lets an algorithm-owned pool attach cleanup to the @@ -146,10 +147,33 @@ func RegisterLifecycleHook(hook func(shutdown bool)) { lifecycleHooks.Unlock() } +// RegisterPreShutdownHook registers admission/cancellation work that must run +// before Destroy waits for its serve loop or any cache entry. It is shutdown-only; +// callbacks must not drain entries or wait for the work they cancel. +func RegisterPreShutdownHook(hook func()) { + if hook == nil { + return + } + lifecycleHooks.Lock() + lifecycleHooks.beforeShutdown = append(lifecycleHooks.beforeShutdown, func(bool) { hook() }) + lifecycleHooks.Unlock() +} + +func runPreShutdownHooks() { + lifecycleHooks.RLock() + hooks := append([]func(bool){}, lifecycleHooks.beforeShutdown...) + lifecycleHooks.RUnlock() + runLifecycleCallbacks(hooks, true) +} + func runLifecycleHooks(shutdown bool) { lifecycleHooks.RLock() hooks := append([]func(bool){}, lifecycleHooks.hooks...) lifecycleHooks.RUnlock() + runLifecycleCallbacks(hooks, shutdown) +} + +func runLifecycleCallbacks(hooks []func(bool), shutdown bool) { for _, hook := range hooks { func() { defer func() { @@ -999,6 +1023,11 @@ var afterIdleClaim func(key string) // the override still reaches the live ticker. var serveStartBarrier func() +// afterServiceScan runs after DestroyByService has identified the entries it +// intends to destroy and before it claims any of them. Nil in production; a +// test sets it to place a replacement-generation race in that window. +var afterServiceScan func() + // evictIdleEntry evicts key ONLY if no search is in flight on it, and holds that claim through // removal and destroy so the entry cannot become busy in between. Returns false if a search // holds it, or if it lost the same races evictEntry can lose. @@ -1216,6 +1245,7 @@ func (c *VectorIndexCache) checkStale() { // destroy the cache func (c *VectorIndexCache) Destroy() { + runPreShutdownHooks() if c.started.Load() { //c.ticker.Stop() if !c.exited.Load() { @@ -1573,3 +1603,51 @@ func (c *VectorIndexCache) RemovePrefix(prefix string) { c.Remove(k) } } + +// DestroyByService tears down only entries whose algorithm explicitly belongs +// to service. This is used by service-scoped optional caches during CN +// shutdown: the service must release resident Search objects before its owner +// closes the files those objects pin. Algorithms that do not implement the +// optional identity method are left untouched. +// +// The map entry is removed before Destroy, just like ordinary eviction, so a +// concurrent cache miss cannot observe a half-destroyed object. Destroy may +// wait for an in-flight search; callers use this only after service request +// admission has been drained. This scan does not join evictions already +// removed from IndexMap. Resource owners must retain consumer lifetime pins +// through successful release before reporting complete cleanup. +func (c *VectorIndexCache) DestroyByService(service string) int { + if c == nil || service == "" { + return 0 + } + type candidate struct { + key string + entry *VectorIndexSearch + } + candidates := make([]candidate, 0, 4) + c.IndexMap.Range(func(key, value any) bool { + k, ok := key.(string) + if !ok { + return true + } + entry, ok := value.(*VectorIndexSearch) + if !ok || entry.Algo == nil { + return true + } + owned, ok := entry.Algo.(interface{ CacheServiceID() string }) + if ok && owned.CacheServiceID() == service { + candidates = append(candidates, candidate{key: k, entry: entry}) + } + return true + }) + if afterServiceScan != nil { + afterServiceScan() + } + destroyed := 0 + for _, item := range candidates { + if c.evictEntry(item.key, item.entry, "service_shutdown") { + destroyed++ + } + } + return destroyed +} diff --git a/pkg/vectorindex/cache/cache_observer_test.go b/pkg/vectorindex/cache/cache_observer_test.go index ef4941045fd4a..548892385f3a5 100644 --- a/pkg/vectorindex/cache/cache_observer_test.go +++ b/pkg/vectorindex/cache/cache_observer_test.go @@ -15,6 +15,7 @@ package cache import ( + "fmt" "sync" "sync/atomic" "testing" @@ -421,3 +422,108 @@ func TestVectorIndexCacheEvictEntrySkipsRenewedTTLAfterHousekeepingCheck(t *test // BuildTS stubs (fulltext2 async-freshness interface method). func (m *observerMock) BuildTS() int64 { return 0 } + +// The stop receiver waits for the prephase, proving that cancellation can run +// before the unbuffered done send. No real ticker or scheduling delay is needed. +func TestVectorIndexCachePreShutdownOrdering(t *testing.T) { + for _, started := range []bool{false, true} { + t.Run(fmt.Sprintf("started=%v", started), func(t *testing.T) { + lifecycleHooks.Lock() + previousPre, previousPost := lifecycleHooks.beforeShutdown, lifecycleHooks.hooks + lifecycleHooks.beforeShutdown, lifecycleHooks.hooks = nil, nil + lifecycleHooks.Unlock() + t.Cleanup(func() { + lifecycleHooks.Lock() + lifecycleHooks.beforeShutdown, lifecycleHooks.hooks = previousPre, previousPost + lifecycleHooks.Unlock() + }) + pre := make(chan struct{}) + release := make(chan struct{}) + var workers sync.WaitGroup + var post, preCalled atomic.Bool + RegisterPreShutdownHook(func() { + preCalled.Store(true) + close(pre) + }) + RegisterLifecycleHook(func(shutdown bool) { + if shutdown { + post.Store(true) + } + }) + c := NewVectorIndexCache() + if started { + c.started.Store(true) + c.done = make(chan bool) + workers.Add(1) + go func() { + defer workers.Done() + select { + case <-pre: + case <-release: + } + <-c.done + c.exited.Store(true) + }() + } + // Rescue and join precede hook restoration even on assertion failure. + t.Cleanup(func() { close(release); workers.Wait() }) + done := make(chan struct{}) + workers.Add(1) + go func() { defer workers.Done(); c.Destroy(); close(done) }() + select { + case <-pre: + case <-time.After(5 * time.Second): + t.Fatal("pre-shutdown hook did not run before the stop-loop wait") + } + select { + case <-done: + case <-time.After(5 * time.Second): + t.Fatal("shutdown phases did not complete") + } + require.True(t, preCalled.Load(), "empty-map shutdown must run the prephase") + require.True(t, post.Load(), "empty-map shutdown must run the postphase") + }) + } +} + +func TestVectorIndexCachePreShutdownPanicAndHousekeeping(t *testing.T) { + lifecycleHooks.Lock() + previousPre, previousPost := lifecycleHooks.beforeShutdown, lifecycleHooks.hooks + lifecycleHooks.beforeShutdown, lifecycleHooks.hooks = nil, nil + lifecycleHooks.Unlock() + t.Cleanup(func() { + lifecycleHooks.Lock() + lifecycleHooks.beforeShutdown, lifecycleHooks.hooks = previousPre, previousPost + lifecycleHooks.Unlock() + }) + var events []string + RegisterPreShutdownHook(nil) + RegisterPreShutdownHook(func() { panic("synthetic pre-shutdown panic") }) + RegisterPreShutdownHook(func() { + // Registration within a callback proves registry locks are not held. + RegisterPreShutdownHook(func() {}) + events = append(events, "pre") + }) + RegisterLifecycleHook(func(shutdown bool) { + if shutdown { + events = append(events, "post") + } else { + events = append(events, "housekeeping") + } + }) + c := NewVectorIndexCache() + houseKeepingSync(t, c) + require.Equal(t, []string{"housekeeping"}, events, "housekeeping must never pre-close admission") + entry := &VectorIndexSearch{Algo: &shutdownOrderSearch{MockSearch: MockSearch{}, onDestroy: func() { events = append(events, "entry") }}} + entry.Cond = sync.NewCond(entry.Mutex.RLocker()) + c.IndexMap.Store("shutdown-order", entry) + require.NotPanics(t, c.Destroy) + require.Equal(t, []string{"housekeeping", "pre", "entry", "post"}, events) +} + +type shutdownOrderSearch struct { + MockSearch + onDestroy func() +} + +func (s *shutdownOrderSearch) Destroy() { s.onDestroy() } diff --git a/pkg/vectorindex/cache/cache_test.go b/pkg/vectorindex/cache/cache_test.go index d7ccb28eb48d4..8336d7244fc40 100644 --- a/pkg/vectorindex/cache/cache_test.go +++ b/pkg/vectorindex/cache/cache_test.go @@ -36,6 +36,93 @@ type MockSearch struct { BuildTs int64 } +type serviceOwnedMockSearch struct { + MockSearch + service string + destroyed bool +} + +func (m *serviceOwnedMockSearch) CacheServiceID() string { return m.service } + +func (m *serviceOwnedMockSearch) Destroy() { m.destroyed = true } + +func TestVectorIndexCacheDestroyByService(t *testing.T) { + c := NewVectorIndexCache() + match := &serviceOwnedMockSearch{service: "cn-a"} + other := &serviceOwnedMockSearch{service: "cn-b"} + ordinary := &MockSearch{} + c.IndexMap.Store("match", newVectorIndexSearch(match)) + c.IndexMap.Store("other", newVectorIndexSearch(other)) + c.IndexMap.Store("ordinary", newVectorIndexSearch(ordinary)) + + require.Equal(t, 1, c.DestroyByService("cn-a")) + require.True(t, match.destroyed) + require.False(t, other.destroyed) + _, ok := c.IndexMap.Load("match") + require.False(t, ok) + _, ok = c.IndexMap.Load("other") + require.True(t, ok) + _, ok = c.IndexMap.Load("ordinary") + require.True(t, ok) +} + +func TestVectorIndexCacheDestroyByServiceDoesNotDeleteReplacement(t *testing.T) { + c := NewVectorIndexCache() + const key = "same-generation-key" + oldSearch := &serviceOwnedMockSearch{service: "cn-a"} + oldEntry := newVectorIndexSearch(oldSearch) + c.IndexMap.Store(key, oldEntry) + + scanned := make(chan struct{}) + release := make(chan struct{}) + var releaseOnce sync.Once + exited := make(chan struct{}) + previousHook := afterServiceScan + afterServiceScan = func() { + close(scanned) + <-release + } + t.Cleanup(func() { + afterServiceScan = previousHook + c.IndexMap.Delete(key) + }) + t.Cleanup(func() { + // Release the scan independently of the behavior under test, then join + // before restoring the package hook, including on a timed-out wait. + releaseOnce.Do(func() { close(release) }) + <-exited + }) + + destroyed := make(chan int, 1) + go func() { + defer close(exited) + destroyed <- c.DestroyByService("cn-a") + }() + select { + case <-scanned: + case <-time.After(2 * time.Second): + t.Fatal("service drain did not reach its scan barrier") + } + + replacementSearch := &serviceOwnedMockSearch{service: "cn-a"} + replacementEntry := newVectorIndexSearch(replacementSearch) + c.IndexMap.Store(key, replacementEntry) + releaseOnce.Do(func() { close(release) }) + + select { + case n := <-destroyed: + require.Equal(t, 0, n, "the scan must not evict a newer entry at the same key") + case <-time.After(2 * time.Second): + t.Fatal("service drain did not finish after scan release") + } + <-exited + require.False(t, oldSearch.destroyed) + require.False(t, replacementSearch.destroyed) + value, ok := c.IndexMap.Load(key) + require.True(t, ok) + require.Same(t, replacementEntry, value) +} + func (m *MockSearch) Search(sqlproc *sqlexec.SqlProcess, query any, rt vectorindex.RuntimeConfig) (keys any, distances []float64, err error) { //time.Sleep(2 * time.Millisecond) return []int64{1}, []float64{2.0}, nil diff --git a/pkg/vectorindex/sqlexec/sqlexec.go b/pkg/vectorindex/sqlexec/sqlexec.go index b35e6669a2392..b6dd777e426ec 100644 --- a/pkg/vectorindex/sqlexec/sqlexec.go +++ b/pkg/vectorindex/sqlexec/sqlexec.go @@ -74,8 +74,12 @@ func (s *SqlContext) SetResolveVariableFunc(f func(varName string, isSystemVar, // Either process.Process or SqlContext is used in SqlProcess. // We will look for process.Process first before SqlContext type SqlProcess struct { - Proc *process.Process - SqlCtx *SqlContext + // operationContext belongs to a scoped copy, never to the caller's process. + // It changes cancellation only; transaction, snapshot and execution identity + // continue to come from the original Proc/SqlCtx and overrides. + operationContext context.Context + Proc *process.Process + SqlCtx *SqlContext // Optional RuntimeFilterSpec RuntimeFilterSpecs []*plan.RuntimeFilterSpec @@ -306,6 +310,9 @@ func (s *SqlProcess) executionStatementOption() executor.StatementOption { } func (s *SqlProcess) GetContext() context.Context { + if s.operationContext != nil { + return s.operationContext + } if s.Proc != nil { return s.Proc.Ctx } @@ -313,12 +320,23 @@ func (s *SqlProcess) GetContext() context.Context { } func (s *SqlProcess) GetTopContext() context.Context { + if s.operationContext != nil { + return s.operationContext + } if s.Proc != nil { return s.Proc.GetTopContext() } return s.SqlCtx.Ctx } +// WithContext returns a request-local copy for an owner's cancellable operation. +// It does not mutate Proc, SqlCtx, or any execution/snapshot identity. +func (s *SqlProcess) WithContext(ctx context.Context) *SqlProcess { + scoped := *s + scoped.operationContext = ctx + return &scoped +} + func (s *SqlProcess) GetResolveVariableFunc() func(varName string, isSystemVar, isGlobalVar bool) (interface{}, error) { if s.Proc != nil { return s.Proc.GetResolveVariableFunc() @@ -349,7 +367,16 @@ func (s *SqlProcess) GetAccountID() (uint32, error) { // run SQL in batch mode. Result batches will stored in memory and return once all result batches received. func RunSql(sqlproc *SqlProcess, sql string) (executor.Result, error) { - return runSql(sqlproc, sql, "") + return runSqlWithContext(sqlproc.GetTopContext(), sqlproc, sql, "") +} + +// RunSqlWithContext is RunSql with an explicit cancellation context. The +// caller's SqlProcess still supplies the transaction, identity and executor; +// the context controls only this internal statement. This is used by +// owner-scoped optional loaders so service shutdown can cancel an already +// admitted metadata read instead of waiting for the query's context forever. +func RunSqlWithContext(ctx context.Context, sqlproc *SqlProcess, sql string) (executor.Result, error) { + return runSqlWithContext(ctx, sqlproc, sql, "") } // RunSqlWithOptimizerHints is RunSql with a per-statement optimizer_hints string (same @@ -361,6 +388,13 @@ func RunSqlWithOptimizerHints(sqlproc *SqlProcess, sql string, optimizerHints st } func runSql(sqlproc *SqlProcess, sql string, optimizerHints string) (executor.Result, error) { + return runSqlWithContext(sqlproc.GetTopContext(), sqlproc, sql, optimizerHints) +} + +func runSqlWithContext(ctx context.Context, sqlproc *SqlProcess, sql string, optimizerHints string) (executor.Result, error) { + if ctx == nil { + ctx = context.Background() + } stmtOpt := sqlproc.executionStatementOption() if optimizerHints != "" { stmtOpt = stmtOpt.WithOptimizerHints(optimizerHints) @@ -373,7 +407,7 @@ func runSql(sqlproc *SqlProcess, sql string, optimizerHints string) (executor.Re } //------------------------------------------------------- - topContext := sqlproc.executionContext(proc.GetTopContext()) + topContext := sqlproc.executionContext(ctx) // Attach optional membership filter payload to context for internal executor. if len(sqlproc.IvfMembershipFilter) > 0 { topContext = context.WithValue(topContext, defines.IvfMembershipFilter{}, sqlproc.IvfMembershipFilter) @@ -414,7 +448,7 @@ func runSql(sqlproc *SqlProcess, sql string, optimizerHints string) (executor.Re } accountId := sqlproc.executionAccountID(sqlctx.AccountId) - execCtx := sqlproc.executionContext(sqlctx.Ctx) + execCtx := sqlproc.executionContext(ctx) exec := v.(executor.SQLExecutor) // SqlCtx is the background entry point (no frontend session) — diff --git a/pkg/vectorindex/sqlexec/sqlexec_test.go b/pkg/vectorindex/sqlexec/sqlexec_test.go index d76c5dca7ac94..ba6b6193fe480 100644 --- a/pkg/vectorindex/sqlexec/sqlexec_test.go +++ b/pkg/vectorindex/sqlexec/sqlexec_test.go @@ -63,6 +63,90 @@ func (m *identityCapturingSQLExecutor) ExecTxn(context.Context, func(executor.Tx return nil } +type contextBlockingSQLExecutor struct { + entered chan struct{} + rescue <-chan struct{} + ctx context.Context + opts executor.Options +} + +func (m *contextBlockingSQLExecutor) Exec(ctx context.Context, _ string, opts executor.Options) (executor.Result, error) { + m.ctx = ctx + m.opts = opts + close(m.entered) + select { + case <-ctx.Done(): + return executor.Result{}, ctx.Err() + case <-m.rescue: + // The rescue path is used only by cleanup. It must not be the + // successful cancellation signal tested by the body above. + return executor.Result{}, context.Canceled + } +} + +func (m *contextBlockingSQLExecutor) ExecTxn(context.Context, func(executor.TxnExecutor) error, executor.Options) error { + return nil +} + +func TestRunSqlWithContextPreservesTxnIdentityAndCancellation(t *testing.T) { + const uuid = "fulltext-owner-cancel" + ctrl := gomock.NewController(t) + txnOp := mock_frontend.NewMockTxnOperator(ctrl) + rescue := make(chan struct{}) + spy := &contextBlockingSQLExecutor{entered: make(chan struct{}), rescue: rescue} + rt := moruntime.DefaultRuntime() + rt.SetGlobalVariables(moruntime.InternalSQLExecutor, spy) + t.Cleanup(func() { rt.CompareAndDeleteGlobalVariables(moruntime.InternalSQLExecutor, spy) }) + moruntime.SetupServiceBasedRuntime(uuid, rt) + + type contextKey struct{} + key := contextKey{} + baseCtx := context.WithValue(context.Background(), key, "request") + sqlproc := NewSqlProcessWithContext( + NewSqlContext(baseCtx, uuid, txnOp, 7, nil)).WithExecutionIdentity(42, "ft2") + ownerCtx, cancel := context.WithCancel(context.WithValue(context.Background(), key, "owner")) + done := make(chan error, 1) + exited := make(chan struct{}) + t.Cleanup(func() { + // Keep this rescue path independent of the assertions below. If an + // assertion or the entry wait fails, the executor must still be released + // before the runtime global is restored. + cancel() + close(rescue) + select { + case <-exited: + case <-time.After(5 * time.Second): + t.Errorf("RunSqlWithContext goroutine did not terminate during cleanup") + } + }) + go func() { + defer close(exited) + _, err := RunSqlWithContext(ownerCtx, sqlproc, "select 1") + done <- err + }() + + select { + case <-spy.entered: + case <-time.After(2 * time.Second): + t.Fatal("SQL executor did not start") + } + accountID, err := defines.GetAccountId(spy.ctx) + require.NoError(t, err) + require.Equal(t, uint32(42), accountID) + require.Equal(t, "owner", spy.ctx.Value(key)) + require.Same(t, txnOp, spy.opts.Txn(), "explicit cancellation must not replace the caller transaction") + require.Equal(t, uint32(42), spy.opts.AccountID()) + require.Equal(t, "ft2", spy.opts.Database()) + + cancel() + select { + case err := <-done: + require.ErrorIs(t, err, context.Canceled) + case <-time.After(2 * time.Second): + t.Fatal("RunSqlWithContext did not return after cancellation") + } +} + func TestSqlProcessExecutionIdentityOverride(t *testing.T) { uuid := "fulltext-publisher-identity" spy := &identityCapturingSQLExecutor{} @@ -159,6 +243,20 @@ func TestSqlProcessExecutionIdentityOverrideFromProcess(t *testing.T) { require.NoError(t, err) assertExecutionIdentity("subscriber") + ownerCtx, cancelOwner := context.WithCancel(context.WithValue(subscriberCtx, key, "owner")) + t.Cleanup(cancelOwner) + scoped := sqlproc.WithContext(ownerCtx) + require.Same(t, proc, scoped.Proc) + require.Same(t, ownerCtx, scoped.GetContext()) + require.Same(t, ownerCtx, scoped.GetTopContext()) + _, err = RunSql(scoped, "select 1") + require.NoError(t, err) + assertExecutionIdentity("owner") + cancelOwner() + require.ErrorIs(t, scoped.GetTopContext().Err(), context.Canceled) + require.NoError(t, sqlproc.GetTopContext().Err()) + require.Equal(t, "subscriber", sqlproc.GetTopContext().Value(key)) + streamCtx := context.WithValue(context.Background(), key, "stream") streamCtx = defines.AttachAccountId(streamCtx, 7) streamCh := make(chan executor.Result, 1) diff --git a/test/distributed/cases/pessimistic_transaction/fulltext2/fulltext2_snapshot.result b/test/distributed/cases/pessimistic_transaction/fulltext2/fulltext2_snapshot.result index 8e18d76ce2d05..2023db0f4b44b 100644 --- a/test/distributed/cases/pessimistic_transaction/fulltext2/fulltext2_snapshot.result +++ b/test/distributed/cases/pessimistic_transaction/fulltext2/fulltext2_snapshot.result @@ -47,5 +47,29 @@ select id, body from t {snapshot='ft2_snap_case_sp'} where match(body) against(' 1 ¦ historic alpha select id from t {snapshot='ft2_snap_case_sp'} where match(body) against('+beta' in boolean mode); ➤ id[-5,64,0] +drop table t; +create table t(id bigint primary key, body text); +insert into t values (11,'replacement alpha'),(12,'replacement beta'); +create fulltext2 index idx on t(body); +set @ft2_recreated_index = (select index_table_name from mo_catalog.mo_indexes where name = 'idx' and algo = 'fulltext2' and algo_table_type = 'ftv2_index' and table_id in (select rel_id from mo_catalog.mo_tables where reldatabase = database() and relname = 't') limit 1); +set @wait_recreated_sql = concat('select count(*) > 0 as ready from `', database(), '`.`', @ft2_recreated_index, '` where tag = 0'); +prepare wait_recreated from @wait_recreated_sql; +execute wait_recreated; +➤ ready[-7,1,0] 𝄀 +1 +deallocate prepare wait_recreated; +select id from t where match(body) against('+alpha' in boolean mode) order by id; +➤ id[-5,64,0] 𝄀 +11 +select id from t {snapshot='ft2_snap_case_sp'} where match(body) against('+alpha' in boolean mode) order by id; +➤ id[-5,64,0] 𝄀 +1 +alter table t alter reindex idx fulltext2 force_sync; +select id from t where match(body) against('+alpha' in boolean mode) order by id; +➤ id[-5,64,0] 𝄀 +11 +select id from t {snapshot='ft2_snap_case_sp'} where match(body) against('+alpha' in boolean mode) order by id; +➤ id[-5,64,0] 𝄀 +1 drop snapshot ft2_snap_case_sp; drop database ft2_snap_case; diff --git a/test/distributed/cases/pessimistic_transaction/fulltext2/fulltext2_snapshot.sql b/test/distributed/cases/pessimistic_transaction/fulltext2/fulltext2_snapshot.sql index ecf9356183e9d..260e23cdfd0fa 100644 --- a/test/distributed/cases/pessimistic_transaction/fulltext2/fulltext2_snapshot.sql +++ b/test/distributed/cases/pessimistic_transaction/fulltext2/fulltext2_snapshot.sql @@ -67,5 +67,24 @@ select id from t where match(body) against('+beta' in boolean mode) order by id; select id, body from t {snapshot='ft2_snap_case_sp'} where match(body) against('+alpha' in boolean mode); select id from t {snapshot='ft2_snap_case_sp'} where match(body) against('+beta' in boolean mode); +-- Same logical names are not a Base identity. Keep the historical Search warm, +-- recreate the live table/index, and then rebuild it; historical MATCH must +-- continue to select the old metadata, while current MATCH uses the new Base. +drop table t; +create table t(id bigint primary key, body text); +insert into t values (11,'replacement alpha'),(12,'replacement beta'); +create fulltext2 index idx on t(body); +set @ft2_recreated_index = (select index_table_name from mo_catalog.mo_indexes where name = 'idx' and algo = 'fulltext2' and algo_table_type = 'ftv2_index' and table_id in (select rel_id from mo_catalog.mo_tables where reldatabase = database() and relname = 't') limit 1); +set @wait_recreated_sql = concat('select count(*) > 0 as ready from `', database(), '`.`', @ft2_recreated_index, '` where tag = 0'); +prepare wait_recreated from @wait_recreated_sql; +-- @wait_expect(1, 120) +execute wait_recreated; +deallocate prepare wait_recreated; +select id from t where match(body) against('+alpha' in boolean mode) order by id; +select id from t {snapshot='ft2_snap_case_sp'} where match(body) against('+alpha' in boolean mode) order by id; +alter table t alter reindex idx fulltext2 force_sync; +select id from t where match(body) against('+alpha' in boolean mode) order by id; +select id from t {snapshot='ft2_snap_case_sp'} where match(body) against('+alpha' in boolean mode) order by id; + drop snapshot ft2_snap_case_sp; drop database ft2_snap_case;