Skip to content

README.md

Latest commit

 

History

History
131 lines (95 loc) · 5.29 KB

README.md

File metadata and controls

131 lines (95 loc) · 5.29 KB
Raw

FastFind.NET

Ultra-high performance cross-platform file search library for .NET 10

License .NET Build Status

Packages

Package Version Description
FastFind.Core NuGet Core interfaces, SIMD string matching, StringPool
FastFind.Windows NuGet NTFS change-journal enumeration and monitoring
FastFind.Unix NuGet Linux/macOS parallel enumeration, file monitoring
FastFind.SQLite NuGet Disk-backed persistent index

Key Features

  • Cross-Platform SIMD: Vector256/Vector128 auto-dispatch (AVX2, SSE2, NEON) — 1.87M ops/sec
  • NTFS change-journal enumeration (Windows, administrator): 31K+ files/sec NTFS enumeration, 30x faster than standard APIs
  • Parallel BFS Enumeration (Linux/macOS): Channel-based depth-aware parallel traversal
  • Real-Time Monitoring: USN Journal (Windows) / inotify (Linux) / FSEvents (macOS)
  • Disk-Backed Index: Keep the index in SQLite instead of memory — footprint stays flat as the corpus grows
  • Memory Optimized: paths and names are interned, so a repeated directory prefix is stored once
  • Auto Platform Detection: ModuleInitializer auto-registration

Installation

dotnet add package FastFind.Core

# Platform-specific (auto-registered at runtime)
dotnet add package FastFind.Windows    # Windows
dotnet add package FastFind.Unix       # Linux / macOS

dotnet add package FastFind.SQLite     # Optional: persistent index

Quick Start

using FastFind;

// Platform auto-detected — creates Windows, Linux, or macOS engine
using var engine = FastFinder.CreateSearchEngine();

await engine.StartIndexingAsync(new IndexingOptions
{
    SpecificDirectories = [@"D:\Projects"],         // Windows
    // MountPoints = ["/home", "/opt"],             // Linux / macOS
    ExcludedPaths = ["node_modules", ".git", "bin", "obj"],
    CollectFileMetadata = true   // sizes and timestamps on the Windows MFT path
});

while (engine.IsIndexing) await Task.Delay(500);

var results = await engine.SearchAsync(new SearchQuery
{
    BasePath = @"D:\Projects",
    SearchText = "controller",
    ExtensionFilter = ".cs",
    MaxResults = 100
});

await foreach (var file in results.Files)
{
    Console.WriteLine($"{file.Name} ({file.SizeFormatted}) - {file.DirectoryPath}");
}

Keeping the index on disk

By default the index lives in memory. Give the engine a persistence store and it answers queries from disk instead, so its footprint does not grow with the number of indexed files:

using FastFind;
using FastFind.SQLite;

await using var store = SqlitePersistence.Create(@"D:\cache\index.db");
await store.InitializeAsync();

using var engine = FastFinder.CreateSearchEngine(store);

// Index once; the store survives the process.
await engine.StartIndexingAsync(new IndexingOptions { SpecificDirectories = [@"D:\Projects"] });
while (engine.IsIndexing) await Task.Delay(500);

Console.WriteLine($"{engine.Index!.Count} files indexed, {engine.Index.MemoryUsage} bytes held in memory");

Pass PersistenceMode.MirrorInMemory instead to keep the in-memory index for speed and use the store purely for durability — that costs the memory of both, and is the right choice only when the corpus comfortably fits.

The engine does not dispose a store you created; you keep ownership.

A store is safe to use from several threads or tasks at once — searching while indexing needs no coordination from the caller, because each operation takes its own pooled connection. See the concurrency notes for what a transaction covers and the one shape that is not supported.

Performance

Metric Windows Linux macOS
SIMD String Matching 1.87M ops/sec 1.87M ops/sec 1.87M ops/sec
File Enumeration 31K files/sec (MFT) Channel BFS parallel Channel BFS parallel
Search Operations 1.68M ops/sec 1.68M ops/sec 1.68M ops/sec
Memory per Op 439 bytes 439 bytes 439 bytes

Platform Support

Platform Status Package
Windows 10/11, Server 2019+ Production FastFind.Windows
Linux (Ubuntu, RHEL, Alpine) Preview FastFind.Unix
macOS (Ventura+) Preview FastFind.Unix

Documentation

License

MIT License - see LICENSE for details.