From 794bd02e521054528e6cab8f727190267d4db458 Mon Sep 17 00:00:00 2001 From: Imran Akram Date: Mon, 7 Sep 2026 09:55:19 +0200 Subject: [PATCH 1/5] Fix CVE-2025-6965 and remove two dead dependencies Upgrade SQLitePCLRaw.lib.e_sqlite3 2.1.11 -> 2.1.13 to resolve GHSA-2m69-gcr7-jv3q / CVE-2025-6965 (High, CVSS 7.2). The 2.1.11 package embeds SQLite 3.49.1; the flaw (aggregate terms exceeding available columns, leading to memory corruption) is fixed in SQLite 3.50.2. 2.1.13 embeds SQLite 3.53.3. 2.1.13 is a drop-in for net48: the package carries native assets only (no managed assembly), keeps the same buildTransitive/net461 layout as 2.1.11, and needs no binding-redirect changes. The .nuspec previously floored SQLitePCLRaw.bundle_e_sqlite3 at 2.1.10. Because NuGet resolves lowest-applicable, consumers of this package were pulling lib.e_sqlite3 2.1.10 - older than what this repo built against, and vulnerable. Raise that floor to 2.1.13 and add an explicit lib.e_sqlite3 floor so downstream projects cannot resolve an unpatched native binary. Also drop two packages that nothing needs: - Microsoft.IdentityModel 7.0.0 is Windows Identity Foundation 3.5, a lib/net35 assembly superseded by WIF's integration into .NET 4.5. No package in the graph depends on it, no source file uses it, and the built assembly carried zero references to it. Microsoft.Xrm.Sdk references System.IdentityModel (the BCL assembly, already satisfied by the framework reference), not microsoft.identitymodel - likely the original reason it was added. - SQLitePCLRaw.config.e_sqlite3 3.0.2 sat amid an otherwise-2.1.11 stack with nothing depending on it at that version. It shipped a second copy of SQLitePCLRaw.batteries_v2.dll conflicting with the 2.1.11 copy the projects actually reference from bundle_green, and triggered a package-downgrade error under modern resolution. Verified: clean restore, Rebuild in Debug and Release x64, all 43 tests pass, deployed e_sqlite3.dll reports SQLite 3.53.3, and a NuGet audit across all remaining packages reports no vulnerabilities. Co-Authored-By: Claude Opus 5 --- .../Xrm.Persistent.Collections.Tests.csproj | 5 +---- Xrm.Persistent.Collections.Tests/packages.config | 4 +--- Xrm.Persistent.Collections.nuspec | 3 ++- Xrm.Persistent.Collections/Xrm.Persistent.Collections.csproj | 3 --- Xrm.Persistent.Collections/packages.config | 4 +--- 5 files changed, 5 insertions(+), 14 deletions(-) diff --git a/Xrm.Persistent.Collections.Tests/Xrm.Persistent.Collections.Tests.csproj b/Xrm.Persistent.Collections.Tests/Xrm.Persistent.Collections.Tests.csproj index e28ede5..5021990 100644 --- a/Xrm.Persistent.Collections.Tests/Xrm.Persistent.Collections.Tests.csproj +++ b/Xrm.Persistent.Collections.Tests/Xrm.Persistent.Collections.Tests.csproj @@ -79,9 +79,6 @@ ..\packages\Microsoft.CrmSdk.CoreAssemblies.9.0.2.60\lib\net462\Microsoft.Crm.Sdk.Proxy.dll - - ..\packages\Microsoft.IdentityModel.7.0.0\lib\net35\microsoft.identitymodel.dll - ..\packages\Microsoft.TestPlatform.ObjectModel.17.10.0\lib\net462\Microsoft.TestPlatform.CoreUtilities.dll @@ -239,7 +236,7 @@ <_SQLitePclRawAssemblies Include="$(MSBuildProjectDirectory)\..\packages\SQLitePCLRaw.provider.e_sqlite3.2.1.11\lib\netstandard2.0\SQLitePCLRaw.provider.e_sqlite3.dll" /> <_SQLitePclRawAssemblies Include="$(MSBuildProjectDirectory)\..\packages\SQLitePCLRaw.provider.dynamic_cdecl.2.1.11\lib\netstandard2.0\SQLitePCLRaw.provider.dynamic_cdecl.dll" /> <_SQLitePclRawAssemblies Include="$(MSBuildProjectDirectory)\..\packages\SQLitePCLRaw.bundle_green.2.1.11\lib\net461\SQLitePCLRaw.batteries_v2.dll" /> - <_SQLitePclRawNative Include="$(MSBuildProjectDirectory)\..\packages\SQLitePCLRaw.lib.e_sqlite3.2.1.11\runtimes\win-x64\native\e_sqlite3.dll" /> + <_SQLitePclRawNative Include="$(MSBuildProjectDirectory)\..\packages\SQLitePCLRaw.lib.e_sqlite3.2.1.13\runtimes\win-x64\native\e_sqlite3.dll" /> <_AdditionalRuntimeAssemblies Include="$(MSBuildProjectDirectory)\..\packages\System.Runtime.CompilerServices.Unsafe.6.1.2\lib\net462\System.Runtime.CompilerServices.Unsafe.dll" /> diff --git a/Xrm.Persistent.Collections.Tests/packages.config b/Xrm.Persistent.Collections.Tests/packages.config index a21c586..daf3283 100644 --- a/Xrm.Persistent.Collections.Tests/packages.config +++ b/Xrm.Persistent.Collections.Tests/packages.config @@ -3,15 +3,13 @@ - - - + diff --git a/Xrm.Persistent.Collections.nuspec b/Xrm.Persistent.Collections.nuspec index cb20cc2..5941e1b 100644 --- a/Xrm.Persistent.Collections.nuspec +++ b/Xrm.Persistent.Collections.nuspec @@ -46,7 +46,8 @@ - + + diff --git a/Xrm.Persistent.Collections/Xrm.Persistent.Collections.csproj b/Xrm.Persistent.Collections/Xrm.Persistent.Collections.csproj index 2eaa086..5451282 100644 --- a/Xrm.Persistent.Collections/Xrm.Persistent.Collections.csproj +++ b/Xrm.Persistent.Collections/Xrm.Persistent.Collections.csproj @@ -69,9 +69,6 @@ ..\packages\Microsoft.CrmSdk.CoreAssemblies.9.0.2.60\lib\net462\Microsoft.Crm.Sdk.Proxy.dll - - ..\packages\Microsoft.IdentityModel.7.0.0\lib\net35\microsoft.identitymodel.dll - ..\packages\Microsoft.CrmSdk.CoreAssemblies.9.0.2.60\lib\net462\Microsoft.Xrm.Sdk.dll diff --git a/Xrm.Persistent.Collections/packages.config b/Xrm.Persistent.Collections/packages.config index 38ca92e..2ecfdcf 100644 --- a/Xrm.Persistent.Collections/packages.config +++ b/Xrm.Persistent.Collections/packages.config @@ -2,14 +2,12 @@ - - - + From 65806c3822d99ebc60a80e82798e0d9b1ec7fdf6 Mon Sep 17 00:00:00 2001 From: Imran Akram Date: Mon, 7 Sep 2026 10:09:29 +0200 Subject: [PATCH 2/5] Release 2.2026.9.7: version bump, release notes, changelog Bump the package to 2.2026.9.7 (CalVer 2.YYYY.M.D) and rewrite the release notes and changelog to describe this as what it is: a security release, not a performance one. Version metadata had drifted - the published 2.2026.3.1 package contained an assembly stamped 2.2026.3.2. Package, AssemblyVersion and AssemblyFileVersion are now all 2.2026.9.7. Also align SQLitePCLRaw.bundle_e_sqlite3 in packages.config with the 2.1.13 floor the .nuspec now publishes, so the version built against and the version declared to consumers agree. Fix two pieces of stale package metadata: the element declared branch "main" while the repository's default branch is "master", and the copyright year was still 2025. Verified: restore, Rebuild in Release x64, 43/43 tests pass, the built assembly stamps FileVersion 2.2026.9.7, and `nuget pack` produces Xrm.Persistent.Collections.2.2026.9.7.nupkg with the expected dependency floors. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 38 +++++++++++++++++++ .../packages.config | 2 +- Xrm.Persistent.Collections.nuspec | 34 ++++++++++------- .../Properties/AssemblyInfo.cs | 4 +- Xrm.Persistent.Collections/packages.config | 2 +- 5 files changed, 63 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cc21b64..3214a27 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,43 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [2.2026.9.7] - 2026-09-07 + +### 🔒 Security Release + +Fixes a high-severity vulnerability in the bundled native SQLite binary and removes two packages that nothing depended on. No API changes, and existing database files are unaffected — this is a drop-in upgrade. + +### Security +- **CVE-2025-6965 / [GHSA-2m69-gcr7-jv3q](https://github.com/advisories/GHSA-2m69-gcr7-jv3q)** (High, CVSS 7.2) — upgraded `SQLitePCLRaw.lib.e_sqlite3` from 2.1.11 to 2.1.13. + - 2.1.11 embeds SQLite **3.49.1**. The flaw — the number of aggregate terms exceeding the available column count, leading to memory corruption — is fixed in SQLite **3.50.2**. 2.1.13 embeds SQLite **3.53.3**. + - Chosen over the 3.x line deliberately: 2.1.13 carries native assets only (no managed assembly), keeps the same `buildTransitive/net461` package layout as 2.1.11, and needs no binding-redirect changes. +- **Raised the published dependency floors so consumers actually receive the fix.** The package previously floored `SQLitePCLRaw.bundle_e_sqlite3` at 2.1.10. Because NuGet resolves lowest-applicable, downstream projects were pulling `lib.e_sqlite3` **2.1.10** — older than what this library was built against, and vulnerable. The floor is now 2.1.13, plus an explicit `SQLitePCLRaw.lib.e_sqlite3 >= 2.1.13` entry so no resolution path can select an unpatched native binary. + - **Projects referencing this package should upgrade**; bumping only the transitive package is not sufficient if they pinned the old floor. + +### Removed +- **`Microsoft.IdentityModel` 7.0.0** — Windows Identity Foundation 3.5, a `lib/net35` assembly superseded by WIF's integration into .NET 4.5. Nothing depended on it: no package in the graph declared it, no source file used it, and the built assembly carried zero references to it. `Microsoft.Xrm.Sdk` references `System.IdentityModel` — the BCL assembly, already satisfied by the framework reference — not `microsoft.identitymodel`, which is the likely origin of the confusion. +- **`SQLitePCLRaw.config.e_sqlite3` 3.0.2** — sat amid an otherwise-2.1.11 stack with nothing depending on it at that version. It shipped a second copy of `SQLitePCLRaw.batteries_v2.dll` that conflicted with the 2.1.11 copy the projects actually reference from `bundle_green`, and triggered a package-downgrade error under modern resolution. + +### Changed +- `SQLitePCLRaw.bundle_e_sqlite3` 2.1.11 → 2.1.13, so the version built against and the published floor agree. +- Package version 2.2026.3.1 → 2.2026.9.7, and assembly version 2.2026.3.2 → 2.2026.9.7. These had drifted apart: the published 2.2026.3.1 package contained an assembly stamped 2.2026.3.2. +- Copyright updated to 2019-2026. + +### Fixed +- `.nuspec` `` metadata declared branch `main`; the repository's default branch is `master`. + +### Performance +- **No measurable change is expected, and none is claimed.** The two removed packages were never loaded at runtime. The SQLite 3.49.1 → 3.53.3 jump is four minor releases of incremental query-planner work that this library's access pattern does not exercise — reads are single-row primary-key lookups against a `WITHOUT ROWID` table, and writes are already batched inside transactions. `sqlite-net-pcl` is unchanged at 1.9.172. + +### Verified +- Clean restore with both removed packages physically absent from the restore folder — NuGet never requested them. +- `Rebuild` succeeds in both Debug and Release (x64). +- 43/43 unit tests pass. +- The deployed `e_sqlite3.dll` reports SQLite 3.53.3. +- A NuGet audit across all remaining packages in both projects reports no known vulnerabilities at any severity. + +--- + ## [2.0.0] - 2025-01-XX ### 🎉 Major Release - .NET Framework 4.8 Upgrade @@ -142,6 +179,7 @@ using Xrm.Persistent.Collections; | From Version | To Version | Breaking Changes | Migration Effort | Database Compatible | |--------------|------------|------------------|------------------|---------------------| | 1.2022.10.3 | 2.2025.1.15 | Namespace only | Low (1-2 hours) | ✅ Yes | +| 2.2026.3.1 | 2.2026.9.7 | None | None (drop-in) | ✅ Yes | --- diff --git a/Xrm.Persistent.Collections.Tests/packages.config b/Xrm.Persistent.Collections.Tests/packages.config index daf3283..d7e773c 100644 --- a/Xrm.Persistent.Collections.Tests/packages.config +++ b/Xrm.Persistent.Collections.Tests/packages.config @@ -6,7 +6,7 @@ - + diff --git a/Xrm.Persistent.Collections.nuspec b/Xrm.Persistent.Collections.nuspec index 5941e1b..4cae406 100644 --- a/Xrm.Persistent.Collections.nuspec +++ b/Xrm.Persistent.Collections.nuspec @@ -2,14 +2,14 @@ Xrm.Persistent.Collections - 2.2026.3.1 + 2.2026.9.7 Xrm Persistent Collections Imran Akram Imran Akram false MIT https://github.com/imranakram/Xrm.Persistent.Collections - + icon.png README.md @@ -22,23 +22,31 @@ Persistent dictionary storage for Dynamics CRM/XRM with SQLite backend - Major upgrade to .NET Framework 4.8 with significant performance improvements (15-25%). + Security release. No API changes; existing database files are unaffected. - Version Format: CalVer (2.YYYY.M.D) - Previous version: 1.2022.10.3 + Version Format: CalVer (2.YYYY.M.D) - Previous version: 2.2026.3.1 - Breaking Changes: - - Namespace changed from Innofactor.Xrm.Persistent.Collections to Xrm.Persistent.Collections + Security: + - Fixes CVE-2025-6965 / GHSA-2m69-gcr7-jv3q (High, CVSS 7.2) by upgrading + SQLitePCLRaw.lib.e_sqlite3 from 2.1.11 to 2.1.13. The 2.1.11 package embeds + SQLite 3.49.1; the flaw is fixed in SQLite 3.50.2, and 2.1.13 embeds 3.53.3. + - Raises the SQLitePCLRaw.bundle_e_sqlite3 floor from 2.1.10 to 2.1.13 and adds an + explicit SQLitePCLRaw.lib.e_sqlite3 floor. Because NuGet resolves lowest-applicable, + consumers of the previous release were pulling an unpatched native binary. Projects + referencing this package should upgrade. - Enhancements: - - Upgraded to .NET Framework 4.8 (from 4.6.2) - - Updated SQLite to 1.9.172 (10-15% faster) - - TLS 1.2/1.3 support for Dynamics 365 Online - - Doubled test coverage (27 comprehensive tests) - - Full backward compatibility with existing database files + Removed (unused, no runtime impact): + - Microsoft.IdentityModel 7.0.0 (Windows Identity Foundation 3.5, superseded by WIF's + integration into .NET 4.5). Nothing in the graph or the source depended on it. + - SQLitePCLRaw.config.e_sqlite3 3.0.2, which shipped a conflicting second copy of + SQLitePCLRaw.batteries_v2.dll. + + Performance: + - No measurable change is expected or claimed. This release is a security fix. See CHANGELOG.md for full details. - Copyright © 2019-2025 + Copyright © 2019-2026 dynamics crm xrm dynamics365 dataverse sqlite persistence cache dictionary storage job-engine akavache diff --git a/Xrm.Persistent.Collections/Properties/AssemblyInfo.cs b/Xrm.Persistent.Collections/Properties/AssemblyInfo.cs index 15af008..33b67cf 100644 --- a/Xrm.Persistent.Collections/Properties/AssemblyInfo.cs +++ b/Xrm.Persistent.Collections/Properties/AssemblyInfo.cs @@ -33,5 +33,5 @@ // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] // Using CalVer format: MAJOR.YYYY.M.D (e.g., 2.2026.3.2S) -[assembly: AssemblyVersion("2.2026.3.2")] -[assembly: AssemblyFileVersion("2.2026.3.2")] +[assembly: AssemblyVersion("2.2026.9.7")] +[assembly: AssemblyFileVersion("2.2026.9.7")] diff --git a/Xrm.Persistent.Collections/packages.config b/Xrm.Persistent.Collections/packages.config index 2ecfdcf..a320b48 100644 --- a/Xrm.Persistent.Collections/packages.config +++ b/Xrm.Persistent.Collections/packages.config @@ -4,7 +4,7 @@ - + From af6e7a72ea97287b1d060ffa1e0eae11027f67e3 Mon Sep 17 00:00:00 2001 From: Imran Akram Date: Mon, 7 Sep 2026 10:21:35 +0200 Subject: [PATCH 3/5] Fix connection concurrency and pin AssemblyVersion to 2.0.0.0 Concurrency ----------- PersistentBlobCache holds a single SQLiteConnection, which sqlite-net does not make thread-safe, so every operation against it has to be serialized. It was not: - Read() took no lock at all while Write() held _writeSemaphore, so a read could execute against a connection a concurrent write was mutating. Both now share a single _dbSemaphore. - CreateConnection() and Write() acquired their semaphore inside the try block. If the wait threw - ObjectDisposedException after Dispose(), for example - the finally released a permit that had never been taken, corrupting the count. The acquire now happens before the try, so a post-Dispose call surfaces a clean ObjectDisposedException. - CreateConnection() assigned _db and only then set the journal mode and created the schema, so a caller reaching the non-null fast path in between could query a CacheItem table that did not exist yet. The connection is built in a local and published only once initialised. - Get(IEnumerable, string) and GetObjectsCreatedAt() built a lazy IEnumerable>, awaited it with Task.WhenAll, then re-enumerated it through .Result - creating a second set of tasks and running every chunked query against the database twice. Both are now materialized before being awaited. - GetObjectsCreatedAt() also walked its keys argument twice without materializing it, so a single-use sequence produced a partial result. The semaphore waits use ConfigureAwait(false), which answers the todo that was on the write path. It matters here: LocalDictionary blocks on these tasks with .Wait()/.Result, which would deadlock on a context. Versioning ---------- AssemblyVersion is pinned to 2.0.0.0 and no longer tracks the CalVer release. It is the identity the CLR binds against, so bumping it every release forced consumers to add or update a binding redirect just to take a patch. AssemblyFileVersion carries the release version. Anyone upgrading from 2.2026.3.1 can drop an existing redirect for this assembly or retarget it to 2.0.0.0. Tests ----- Five tests added. GetCreatedAt_Enumerates_The_Supplied_Keys_Only_Once fails against the pre-fix code, which was confirmed by reverting the production change and re-running. The other four are stress coverage for the shared-connection paths and do not deterministically reproduce a race - they guard against regressions rather than prove the fix. Verified: Rebuild in Release x64, 48/48 tests pass across six consecutive runs, the assembly stamps AssemblyVersion 2.0.0.0 and FileVersion 2.2026.9.7, and nuget pack succeeds. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 18 +- .../PersistentBlobCacheTests.cs | 158 ++++++++++++++++++ Xrm.Persistent.Collections.nuspec | 19 ++- .../Backend/PersistentBlobCache.cs | 69 +++++--- .../Properties/AssemblyInfo.cs | 10 +- 5 files changed, 250 insertions(+), 24 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3214a27..c6fa2ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,19 +24,33 @@ Fixes a high-severity vulnerability in the bundled native SQLite binary and remo ### Changed - `SQLitePCLRaw.bundle_e_sqlite3` 2.1.11 → 2.1.13, so the version built against and the published floor agree. -- Package version 2.2026.3.1 → 2.2026.9.7, and assembly version 2.2026.3.2 → 2.2026.9.7. These had drifted apart: the published 2.2026.3.1 package contained an assembly stamped 2.2026.3.2. +- Package version 2.2026.3.1 → 2.2026.9.7. +- **`AssemblyVersion` is now pinned to `2.0.0.0`** and no longer tracks the CalVer release. It is the identity the CLR binds against, so bumping it every release forced every consuming application to add or update a binding redirect just to take a patch. `AssemblyFileVersion` carries the real release version (`2.2026.9.7`). `AssemblyVersion` will change only on a breaking release. + - Previously these had also drifted: the published 2.2026.3.1 package contained an assembly stamped 2.2026.3.2. + - **Consumers upgrading from 2.2026.3.1 can remove any binding redirect** they were carrying for this assembly, or point it at `2.0.0.0`. - Copyright updated to 2019-2026. +### Fixed — Concurrency +- **Reads were completely unsynchronized.** `PersistentBlobCache.Read()` executed against the shared `SQLiteConnection` while holding no lock, and writes held a separate `_writeSemaphore`. sqlite-net's `SQLiteConnection` is not thread-safe and this class holds a single connection, so a read could run against a connection a concurrent write was mutating. Reads and writes now share one `_dbSemaphore`, so every operation against the connection is serialized. +- **A throwing `WaitAsync` could corrupt the semaphore count.** `CreateConnection()` and `Write()` both acquired their semaphore *inside* the `try`, so if the wait threw — `ObjectDisposedException` after `Dispose()`, for example — the `finally` released a permit that had never been taken. The acquire now happens before the `try`. Calls made after `Dispose()` now surface a clean `ObjectDisposedException` instead of leaving the semaphore in a corrupt state. +- **The connection was published before it was initialised.** `CreateConnection()` assigned `_db` and only then set the journal mode and created the schema. A caller reaching the non-null fast path in between could query a `CacheItem` table that did not exist yet. The connection is now built in a local and assigned only once the schema is in place. +- **`GetObjectsCreatedAt()` walked its `keys` argument twice without materializing it**, so a single-use sequence (an iterator, a `yield return` method, a LINQ chain over a stream) silently came back empty the second time and produced a partial result. Now materialized once. +- **`Get(IEnumerable, string)` and `GetObjectsCreatedAt()` issued every query twice.** Both built a lazy `IEnumerable>`, awaited it with `Task.WhenAll`, then re-enumerated it via `.Result` — which created a fresh set of tasks and re-ran every chunked query against the database. Both sequences are now materialized with `.ToArray()` before being awaited. + ### Fixed - `.nuspec` `` metadata declared branch `main`; the repository's default branch is `master`. ### Performance - **No measurable change is expected, and none is claimed.** The two removed packages were never loaded at runtime. The SQLite 3.49.1 → 3.53.3 jump is four minor releases of incremental query-planner work that this library's access pattern does not exercise — reads are single-row primary-key lookups against a `WITHOUT ROWID` table, and writes are already batched inside transactions. `sqlite-net-pcl` is unchanged at 1.9.172. +### Added +- Five tests covering the concurrency fixes: mixed concurrent reads and writes, concurrent operations against an unopened connection, concurrent `CreateConnection()` calls, bulk `Get` across multiple internal chunks, and single-use-sequence handling in `GetCreatedAt`. The last of these fails against the pre-fix code; the others are stress coverage and do not deterministically reproduce a race. + ### Verified - Clean restore with both removed packages physically absent from the restore folder — NuGet never requested them. - `Rebuild` succeeds in both Debug and Release (x64). -- 43/43 unit tests pass. +- 48/48 unit tests pass, stable across repeated runs. +- The built assembly stamps `AssemblyVersion 2.0.0.0` and `FileVersion 2.2026.9.7`. - The deployed `e_sqlite3.dll` reports SQLite 3.53.3. - A NuGet audit across all remaining packages in both projects reports no known vulnerabilities at any severity. diff --git a/Xrm.Persistent.Collections.Tests/PersistentBlobCacheTests.cs b/Xrm.Persistent.Collections.Tests/PersistentBlobCacheTests.cs index ed0fb01..eda4124 100644 --- a/Xrm.Persistent.Collections.Tests/PersistentBlobCacheTests.cs +++ b/Xrm.Persistent.Collections.Tests/PersistentBlobCacheTests.cs @@ -1,6 +1,8 @@ namespace Xrm.Persistent.Collections.Backend { using System; + using System.Collections; + using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; @@ -284,6 +286,162 @@ await cache.Insert("expired-key", Encoding.UTF8.GetBytes("expired"), Assert.DoesNotContain("expired-key", keys); } + [Fact] + public async Task Concurrent_Reads_And_Writes_Do_Not_Throw() + { + // Arrange + await cache.CreateConnection(); + const int operations = 150; + + // Act - reads and writes issued against the same connection at the same time. + // Before reads were serialized against writes, this raced on a shared + // SQLiteConnection, which sqlite-net does not guard. + var work = new List(); + for (var i = 0; i < operations; i++) + { + var key = "concurrent-" + i; + var payload = Encoding.UTF8.GetBytes("value-" + i); + + work.Add(Task.Run(() => cache.Insert(key, payload))); + work.Add(Task.Run(() => cache.GetOrDefault(key, string.Empty))); + work.Add(Task.Run(() => cache.GetAllKeys())); + } + + await Task.WhenAll(work); + + // Assert - every write landed and is readable + for (var i = 0; i < operations; i++) + { + var stored = await cache.Get("concurrent-" + i); + Assert.Equal("value-" + i, Encoding.UTF8.GetString(stored)); + } + } + + [Fact] + public async Task Concurrent_Operations_On_Unopened_Connection_Do_Not_Throw() + { + // Arrange - deliberately no CreateConnection() first, so many callers race + // to open the connection at once. + + // Act + var work = Enumerable.Range(0, 50) + .Select(i => Task.Run(async () => + { + await cache.Insert("cold-" + i, Encoding.UTF8.GetBytes("v" + i)); + return await cache.GetOrDefault("cold-" + i, string.Empty); + })) + .ToArray(); + + var results = await Task.WhenAll(work); + + // Assert + Assert.All(results, r => Assert.NotEmpty(r)); + } + + [Fact] + public async Task Get_With_Keys_Spanning_Multiple_Chunks_Returns_All_Values() + { + // Arrange - the internal chunk size is 950, so this spans three chunks + await cache.CreateConnection(); + const int count = 2000; + + var items = new Dictionary(); + for (var i = 0; i < count; i++) + { + items["bulk-" + i] = Encoding.UTF8.GetBytes("payload-" + i); + } + + await cache.Insert(items); + + // Act + var fetched = await cache.Get(items.Keys); + + // Assert + Assert.Equal(count, fetched.Count); + for (var i = 0; i < count; i++) + { + Assert.Equal("payload-" + i, Encoding.UTF8.GetString(fetched["bulk-" + i])); + } + } + + [Fact] + public async Task GetCreatedAt_Enumerates_The_Supplied_Keys_Only_Once() + { + // Arrange - GetObjectsCreatedAt walks its keys argument twice. Passing a + // sequence that refuses a second enumeration pins that behaviour down. + await cache.CreateConnection(); + + var items = new Dictionary(); + for (var i = 0; i < 20; i++) + { + items["stamped-" + i] = Encoding.UTF8.GetBytes("v" + i); + } + + await cache.Insert(items); + + // Act + var stamps = await cache.GetCreatedAt(new SingleUseSequence(items.Keys)); + + // Assert - an entry comes back for every key that was supplied + Assert.Equal(items.Count, stamps.Count); + foreach (var key in items.Keys) + { + Assert.True(stamps.ContainsKey(key)); + } + } + + [Fact] + public async Task Concurrent_CreateConnection_Publishes_A_Fully_Initialised_Connection() + { + // Arrange - many callers open the connection at once. The connection field is + // only assigned after the schema exists, so no caller can observe a connection + // whose CacheItem table has not been created yet. + var opens = Enumerable.Range(0, 64) + .Select(_ => Task.Run(() => cache.CreateConnection())) + .ToArray(); + + // Act + await Task.WhenAll(opens); + + // Assert - the schema is usable immediately afterwards + await cache.Insert("after-open", Encoding.UTF8.GetBytes("ok")); + var stored = await cache.Get("after-open"); + Assert.Equal("ok", Encoding.UTF8.GetString(stored)); + } + #endregion Public Methods + + #region Private Types + + /// + /// A sequence that throws if anything walks it more than once, so that code + /// relying on repeated enumeration of a caller-supplied + /// fails loudly instead of silently returning nothing the second time. + /// + private sealed class SingleUseSequence : IEnumerable + { + private readonly IEnumerable source; + private bool enumerated; + + public SingleUseSequence(IEnumerable source) + { + this.source = source; + } + + public IEnumerator GetEnumerator() + { + if (enumerated) + { + throw new InvalidOperationException("Sequence was enumerated more than once."); + } + + enumerated = true; + return source.GetEnumerator(); + } + + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + } + + #endregion Private Types } } \ No newline at end of file diff --git a/Xrm.Persistent.Collections.nuspec b/Xrm.Persistent.Collections.nuspec index 4cae406..396192d 100644 --- a/Xrm.Persistent.Collections.nuspec +++ b/Xrm.Persistent.Collections.nuspec @@ -41,8 +41,25 @@ - SQLitePCLRaw.config.e_sqlite3 3.0.2, which shipped a conflicting second copy of SQLitePCLRaw.batteries_v2.dll. + Concurrency fixes: + - Reads are now serialized against writes. PersistentBlobCache.Read took no lock while + sharing a single non-thread-safe SQLiteConnection with writes. + - CreateConnection and Write acquired their semaphore inside the try block, so a + throwing WaitAsync released a permit that was never taken. + - CreateConnection published the connection before creating the schema, so a caller + could query a CacheItem table that did not exist yet. + - Get(IEnumerable) and GetObjectsCreatedAt re-enumerated a lazy task sequence, issuing + every chunked query against the database twice. + - GetObjectsCreatedAt walked its keys argument twice without materializing it, so a + single-use sequence produced a partial result. + + AssemblyVersion is now pinned to 2.0.0.0 and no longer tracks the release version, so + consumers no longer need a binding redirect for each patch. AssemblyFileVersion carries + the release version. Consumers upgrading from 2.2026.3.1 can drop any existing binding + redirect for this assembly or retarget it to 2.0.0.0. + Performance: - - No measurable change is expected or claimed. This release is a security fix. + - No measurable change is expected or claimed from the dependency updates. See CHANGELOG.md for full details. diff --git a/Xrm.Persistent.Collections/Backend/PersistentBlobCache.cs b/Xrm.Persistent.Collections/Backend/PersistentBlobCache.cs index 2daad8f..999bebe 100644 --- a/Xrm.Persistent.Collections/Backend/PersistentBlobCache.cs +++ b/Xrm.Persistent.Collections/Backend/PersistentBlobCache.cs @@ -17,7 +17,14 @@ public class PersistentBlobCache : IBlobCache private readonly Type _cacheItemType; private readonly SemaphoreSlim _createSemaphore; private readonly string _databasePath; - private readonly SemaphoreSlim _writeSemaphore; + + /// + /// Serializes every operation against , reads included. + /// sqlite-net's is not thread-safe, and this class + /// holds a single connection, so concurrent use has to be serialized here. + /// + private readonly SemaphoreSlim _dbSemaphore; + private SQLiteConnection _db; #endregion Private Fields @@ -27,7 +34,7 @@ public class PersistentBlobCache : IBlobCache public PersistentBlobCache(string databasePath) { _databasePath = databasePath; - _writeSemaphore = new SemaphoreSlim(1, 1); + _dbSemaphore = new SemaphoreSlim(1, 1); _createSemaphore = new SemaphoreSlim(1, 1); _cacheItemType = typeof(CacheItem); } @@ -47,9 +54,11 @@ public async Task CreateConnection() return; } + // Acquire outside the try: if WaitAsync throws (for example ObjectDisposedException + // after Dispose), the finally must not release a permit that was never taken. + await _createSemaphore.WaitAsync().ConfigureAwait(false); try { - await _createSemaphore.WaitAsync(); if (_db != null) { return; @@ -61,9 +70,14 @@ public async Task CreateConnection() Directory.CreateDirectory(directory); } - _db = new SQLiteConnection(_databasePath); - _db.ExecuteScalar("PRAGMA journal_mode = WAL"); - EnsureSchema(); + // Build the connection in a local and publish it to _db only once the + // journal mode and schema are in place. Assigning _db first would let a + // caller that sees the field non-null through the fast path above query a + // CacheItem table that does not exist yet. + var connection = new SQLiteConnection(_databasePath); + connection.ExecuteScalar("PRAGMA journal_mode = WAL"); + EnsureSchema(connection); + _db = connection; } finally { @@ -75,7 +89,7 @@ public void Dispose() { _db?.Close(); _db?.Dispose(); - _writeSemaphore?.Dispose(); + _dbSemaphore?.Dispose(); _createSemaphore?.Dispose(); } @@ -128,7 +142,8 @@ public async Task> Get(IEnumerable keys, str Data = p.Data }); }); - }); + }) + .ToArray(); // materialize: enumerating twice would issue every query again await Task.WhenAll(tasks).ConfigureAwait(false); @@ -219,7 +234,9 @@ SELECT CreatedAt AS UtcTicks FROM CacheItem public async Task> GetObjectsCreatedAt(IEnumerable keys) { - keys = keys.Distinct(); + // Materialized because it is enumerated twice below; a one-shot sequence + // would otherwise come back empty the second time. + keys = keys.Distinct().ToArray(); var utcTicks = DateTime.UtcNow.Ticks; var typeName = typeof(T).FullName; @@ -246,7 +263,8 @@ AND Key IN ({keyParameters}) chunkKeys.CopyTo(args, 2); return o.Query(sql, args); }); - }); + }) + .ToArray(); // materialize: enumerating twice would issue every query again await Task.WhenAll(tasks).ConfigureAwait(false); var foundKeys = tasks @@ -441,7 +459,7 @@ DELETE FROM CacheItem #region Private Methods - private void EnsureSchema() + private static void EnsureSchema(SQLiteConnection connection) { var tableSQL = @" CREATE TABLE IF NOT EXISTS CacheItem @@ -454,34 +472,47 @@ CREATE TABLE IF NOT EXISTS CacheItem PRIMARY KEY (Key, Type) ) WITHOUT ROWID;"; - _db.Execute(tableSQL); + connection.Execute(tableSQL); } private async Task Read(Func readOperation) { - if (_db == null) + // Reads take the same lock as writes. The connection is shared and sqlite-net + // does not guard its own state, so an unsynchronized read can run against a + // connection that a concurrent write is mutating. + await _dbSemaphore.WaitAsync().ConfigureAwait(false); + try { - await CreateConnection(); - } + if (_db == null) + { + await CreateConnection().ConfigureAwait(false); + } - return readOperation(_db); + return readOperation(_db); + } + finally + { + _dbSemaphore.Release(); + } } private async Task Write(Action writeOperation) { + // Acquire outside the try, so a throwing WaitAsync cannot reach the finally + // and release a permit that was never taken. + await _dbSemaphore.WaitAsync().ConfigureAwait(false); try { - await _writeSemaphore.WaitAsync(); // todo: safe to configure await here? ¯\_(ツ)_/¯ if (_db == null) { - await CreateConnection(); + await CreateConnection().ConfigureAwait(false); } writeOperation(_db); } finally { - _writeSemaphore.Release(); + _dbSemaphore.Release(); } } diff --git a/Xrm.Persistent.Collections/Properties/AssemblyInfo.cs b/Xrm.Persistent.Collections/Properties/AssemblyInfo.cs index 33b67cf..a5ba5f0 100644 --- a/Xrm.Persistent.Collections/Properties/AssemblyInfo.cs +++ b/Xrm.Persistent.Collections/Properties/AssemblyInfo.cs @@ -32,6 +32,12 @@ // You can specify all the values or you can default the Build and Revision Numbers // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] -// Using CalVer format: MAJOR.YYYY.M.D (e.g., 2.2026.3.2S) -[assembly: AssemblyVersion("2.2026.9.7")] +// +// AssemblyVersion is deliberately pinned to the major version and does NOT track +// the CalVer release version. It is the identity the CLR binds against, so changing +// it on every release would force every consuming application to add or update a +// binding redirect just to pick up a patch. Bump it only on a breaking change. +// +// AssemblyFileVersion carries the real release version, in CalVer: MAJOR.YYYY.M.D. +[assembly: AssemblyVersion("2.0.0.0")] [assembly: AssemblyFileVersion("2.2026.9.7")] From 21f43e1bf77d7a363ada6c251c80a02ce9dbe1e2 Mon Sep 17 00:00:00 2001 From: Imran Akram Date: Mon, 7 Sep 2026 10:33:26 +0200 Subject: [PATCH 4/5] Rebuild the .nuspec dependency list from the real reference graph The published dependency list had drifted from what the library needs, in both directions. Removed, because nothing needs them declared directly: - Microsoft.Bcl.AsyncInterfaces, System.Buffers, System.Memory, System.Numerics.Vectors, System.Runtime.CompilerServices.Unsafe, System.Threading.Tasks.Extensions, System.ValueTuple. These were hand-maintained transitive leaves and nothing kept them in sync - four of them floored *below* the versions the library was built against (System.Memory 4.5.5 declared against 4.6.3 built). They arrive through Microsoft.CrmSdk.CoreAssemblies -> System.Text.Json at versions Microsoft ships and tests together. - SQLitePCLRaw.bundle_e_sqlite3, also dropped from both packages.config files. Nothing referenced it: no csproj HintPath or Import points into it, and the solution builds and passes its tests with the package physically absent from the restore folder. Declaring it pulled a second, conflicting copy of SQLitePCLRaw.batteries_v2.dll alongside the one bundle_green supplies - the same failure mode as the config.e_sqlite3 package removed earlier on this branch. Added, because a fresh consumer genuinely needs them: - SQLitePCLRaw.bundle_green, .core and .provider.dynamic_cdecl at 2.1.11. sqlite-net initialises its native provider through SQLitePCLRaw.batteries_v2, and sqlite-net-pcl 1.9.172 alone floors that stack at 2.1.2. These floors match what this release is built and tested against. Correcting an earlier assessment on this branch: System.Text.Json, System.Text.Encodings.Web and System.ServiceModel.Http/Primitives were never missing from a consumer's graph. Microsoft.CrmSdk.CoreAssemblies 9.0.2.60 declares them itself and floors System.Text.Json at 8.0.5, the build patched for CVE-2024-43485. Verification ------------ The eight declared dependencies resolve to a 20-package closure. The release was packed to a local feed and restored into a fresh net48 project whose only PackageReference is this library, then run. That consumer: - builds with 0 warnings and 0 errors - resolves SQLitePCLRaw.lib.e_sqlite3 2.1.13 and deploys a native e_sqlite3.dll reporting SQLite 3.53.3 - resolves System.Text.Json 8.0.5 transitively - round-trips an Entity carrying EntityReference, OptionSetValue and Money attributes through a real database file - reports no known vulnerabilities under a NuGetAudit over the whole closure at the lowest severity threshold Solution still builds and 48/48 tests pass. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 11 +++++ .../packages.config | 1 - Xrm.Persistent.Collections.nuspec | 40 ++++++++++++++----- Xrm.Persistent.Collections/packages.config | 1 - 4 files changed, 41 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c6fa2ed..f1e87fc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,17 @@ Fixes a high-severity vulnerability in the bundled native SQLite binary and remo - **Raised the published dependency floors so consumers actually receive the fix.** The package previously floored `SQLitePCLRaw.bundle_e_sqlite3` at 2.1.10. Because NuGet resolves lowest-applicable, downstream projects were pulling `lib.e_sqlite3` **2.1.10** — older than what this library was built against, and vulnerable. The floor is now 2.1.13, plus an explicit `SQLitePCLRaw.lib.e_sqlite3 >= 2.1.13` entry so no resolution path can select an unpatched native binary. - **Projects referencing this package should upgrade**; bumping only the transitive package is not sufficient if they pinned the old floor. +### Fixed — Package dependencies +The `.nuspec` dependency list had drifted from what the library actually needs, in both directions. It is now derived from the real reference graph and verified end to end. + +- **Declared floors sat below the versions the library was built against** — `System.Buffers` 4.5.1 against 4.6.1, `System.Memory` 4.5.5 against 4.6.3, `System.Numerics.Vectors` 4.5.0 against 4.6.1, `System.Runtime.CompilerServices.Unsafe` 6.0.0 against 6.1.2. These were hand-maintained transitive leaves that nothing kept in sync. They are no longer declared directly; they arrive through `Microsoft.CrmSdk.CoreAssemblies` → `System.Text.Json` at versions Microsoft ships and tests together. +- **The SQLitePCLRaw managed stack was not declared at all.** `sqlite-net` initialises its native provider through `SQLitePCLRaw.batteries_v2`, and `sqlite-net-pcl` 1.9.172 alone floors that at 2.1.2. `bundle_green`, `core` and `provider.dynamic_cdecl` are now declared at 2.1.11 — the versions this release is built and tested against. +- **`SQLitePCLRaw.bundle_e_sqlite3` is no longer declared, and has been dropped from both `packages.config` files.** Nothing referenced it: no csproj `HintPath` or `Import` pointed into it, and the solution builds and passes its tests with the package physically absent. Declaring it would have pulled a second, conflicting copy of `SQLitePCLRaw.batteries_v2.dll` alongside the one `bundle_green` supplies — the same failure mode as the `config.e_sqlite3` package removed above. + +Contrary to an earlier reading, `System.Text.Json`, `System.Text.Encodings.Web` and `System.ServiceModel.Http`/`Primitives` were never missing from a consumer's graph: `Microsoft.CrmSdk.CoreAssemblies` 9.0.2.60 declares them itself, and floors `System.Text.Json` at 8.0.5 — the build patched for CVE-2024-43485. + +The eight declared dependencies now resolve to a 20-package closure. Verified by packing the release, restoring it into a fresh net48 project whose only `PackageReference` is this library, and running a smoke test that round-trips an `Entity` with `EntityReference`, `OptionSetValue` and `Money` attributes through a real database file. The consumer builds with no warnings, resolves `SQLitePCLRaw.lib.e_sqlite3` 2.1.13, deploys a native `e_sqlite3.dll` reporting SQLite 3.53.3, and a NuGet audit over the whole closure reports no known vulnerabilities. + ### Removed - **`Microsoft.IdentityModel` 7.0.0** — Windows Identity Foundation 3.5, a `lib/net35` assembly superseded by WIF's integration into .NET 4.5. Nothing depended on it: no package in the graph declared it, no source file used it, and the built assembly carried zero references to it. `Microsoft.Xrm.Sdk` references `System.IdentityModel` — the BCL assembly, already satisfied by the framework reference — not `microsoft.identitymodel`, which is the likely origin of the confusion. - **`SQLitePCLRaw.config.e_sqlite3` 3.0.2** — sat amid an otherwise-2.1.11 stack with nothing depending on it at that version. It shipped a second copy of `SQLitePCLRaw.batteries_v2.dll` that conflicted with the 2.1.11 copy the projects actually reference from `bundle_green`, and triggered a package-downgrade error under modern resolution. diff --git a/Xrm.Persistent.Collections.Tests/packages.config b/Xrm.Persistent.Collections.Tests/packages.config index d7e773c..5fe8d26 100644 --- a/Xrm.Persistent.Collections.Tests/packages.config +++ b/Xrm.Persistent.Collections.Tests/packages.config @@ -6,7 +6,6 @@ - diff --git a/Xrm.Persistent.Collections.nuspec b/Xrm.Persistent.Collections.nuspec index 396192d..2b7176a 100644 --- a/Xrm.Persistent.Collections.nuspec +++ b/Xrm.Persistent.Collections.nuspec @@ -58,6 +58,18 @@ the release version. Consumers upgrading from 2.2026.3.1 can drop any existing binding redirect for this assembly or retarget it to 2.0.0.0. + Package dependencies: + - The dependency list is rebuilt from the real reference graph. Declared floors that + sat below the versions built against have been dropped in favour of the transitive + versions Microsoft ships together; the SQLitePCLRaw managed stack (bundle_green, + core, provider.dynamic_cdecl) is now declared, since sqlite-net initialises its + provider through batteries_v2 and sqlite-net-pcl alone floors that at 2.1.2. + - SQLitePCLRaw.bundle_e_sqlite3 is no longer declared and has been removed from the + projects. Nothing referenced it, and it supplied a second conflicting copy of + SQLitePCLRaw.batteries_v2.dll. + - Verified by restoring this package into a fresh net48 project as its only + PackageReference and round-tripping a CRM Entity through a real database file. + Performance: - No measurable change is expected or claimed from the dependency updates. @@ -67,19 +79,27 @@ dynamics crm xrm dynamics365 dataverse sqlite persistence cache dictionary storage job-engine akavache - - + - - - - - - - - + + + + + + + + + + + diff --git a/Xrm.Persistent.Collections/packages.config b/Xrm.Persistent.Collections/packages.config index a320b48..9e9e558 100644 --- a/Xrm.Persistent.Collections/packages.config +++ b/Xrm.Persistent.Collections/packages.config @@ -4,7 +4,6 @@ - From 370260a1b4fd14cb0eaba8326cace154743d8bf9 Mon Sep 17 00:00:00 2001 From: Imran Akram Date: Mon, 7 Sep 2026 11:10:38 +0200 Subject: [PATCH 5/5] Run CI on master, the default branch The workflow triggered on main, vNext and develop for pushes, and on main and vNext for pull requests. The repository's default branch is master, and none of those other branches exist here, so CI has never run: not on a push to master, and not on any pull request into it. The pull_request filter matches the target branch, so adding master there is what makes a PR into master build. This is the same main/master confusion as the .nuspec element corrected earlier on this branch. Co-Authored-By: Claude Opus 5 --- .github/workflows/ci.yml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ce17340..6ddadd1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -2,9 +2,11 @@ name: CI on: push: - branches: [ main, vNext, develop ] + branches: [ master, main, vNext, develop ] pull_request: - branches: [ main, vNext ] + # Filters on the target branch. master is the default branch, so without it + # no pull request into master was ever built. + branches: [ master, main, vNext ] jobs: build-and-test: