Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@ name: CI

on:
push:
branches: [ main, vNext, develop ]
branches: [ master, main, vNext, develop ]
pull_request:
branches: [ main, vNext ]
# Filters on the target branch. master is the default branch, so without it
# no pull request into master was ever built.
branches: [ master, main, vNext ]

jobs:
build-and-test:
Expand Down
63 changes: 63 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,68 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [2.2026.9.7] - 2026-09-07

### 🔒 Security Release

Fixes a high-severity vulnerability in the bundled native SQLite binary and removes two packages that nothing depended on. No API changes, and existing database files are unaffected — this is a drop-in upgrade.

### Security
- **CVE-2025-6965 / [GHSA-2m69-gcr7-jv3q](https://github.com/advisories/GHSA-2m69-gcr7-jv3q)** (High, CVSS 7.2) — upgraded `SQLitePCLRaw.lib.e_sqlite3` from 2.1.11 to 2.1.13.
- 2.1.11 embeds SQLite **3.49.1**. The flaw — the number of aggregate terms exceeding the available column count, leading to memory corruption — is fixed in SQLite **3.50.2**. 2.1.13 embeds SQLite **3.53.3**.
- Chosen over the 3.x line deliberately: 2.1.13 carries native assets only (no managed assembly), keeps the same `buildTransitive/net461` package layout as 2.1.11, and needs no binding-redirect changes.
- **Raised the published dependency floors so consumers actually receive the fix.** The package previously floored `SQLitePCLRaw.bundle_e_sqlite3` at 2.1.10. Because NuGet resolves lowest-applicable, downstream projects were pulling `lib.e_sqlite3` **2.1.10** — older than what this library was built against, and vulnerable. The floor is now 2.1.13, plus an explicit `SQLitePCLRaw.lib.e_sqlite3 >= 2.1.13` entry so no resolution path can select an unpatched native binary.
- **Projects referencing this package should upgrade**; bumping only the transitive package is not sufficient if they pinned the old floor.

### Fixed — Package dependencies
The `.nuspec` dependency list had drifted from what the library actually needs, in both directions. It is now derived from the real reference graph and verified end to end.

- **Declared floors sat below the versions the library was built against** — `System.Buffers` 4.5.1 against 4.6.1, `System.Memory` 4.5.5 against 4.6.3, `System.Numerics.Vectors` 4.5.0 against 4.6.1, `System.Runtime.CompilerServices.Unsafe` 6.0.0 against 6.1.2. These were hand-maintained transitive leaves that nothing kept in sync. They are no longer declared directly; they arrive through `Microsoft.CrmSdk.CoreAssemblies` → `System.Text.Json` at versions Microsoft ships and tests together.
- **The SQLitePCLRaw managed stack was not declared at all.** `sqlite-net` initialises its native provider through `SQLitePCLRaw.batteries_v2`, and `sqlite-net-pcl` 1.9.172 alone floors that at 2.1.2. `bundle_green`, `core` and `provider.dynamic_cdecl` are now declared at 2.1.11 — the versions this release is built and tested against.
- **`SQLitePCLRaw.bundle_e_sqlite3` is no longer declared, and has been dropped from both `packages.config` files.** Nothing referenced it: no csproj `HintPath` or `Import` pointed into it, and the solution builds and passes its tests with the package physically absent. Declaring it would have pulled a second, conflicting copy of `SQLitePCLRaw.batteries_v2.dll` alongside the one `bundle_green` supplies — the same failure mode as the `config.e_sqlite3` package removed above.

Contrary to an earlier reading, `System.Text.Json`, `System.Text.Encodings.Web` and `System.ServiceModel.Http`/`Primitives` were never missing from a consumer's graph: `Microsoft.CrmSdk.CoreAssemblies` 9.0.2.60 declares them itself, and floors `System.Text.Json` at 8.0.5 — the build patched for CVE-2024-43485.

The eight declared dependencies now resolve to a 20-package closure. Verified by packing the release, restoring it into a fresh net48 project whose only `PackageReference` is this library, and running a smoke test that round-trips an `Entity` with `EntityReference`, `OptionSetValue` and `Money` attributes through a real database file. The consumer builds with no warnings, resolves `SQLitePCLRaw.lib.e_sqlite3` 2.1.13, deploys a native `e_sqlite3.dll` reporting SQLite 3.53.3, and a NuGet audit over the whole closure reports no known vulnerabilities.

### Removed
- **`Microsoft.IdentityModel` 7.0.0** — Windows Identity Foundation 3.5, a `lib/net35` assembly superseded by WIF's integration into .NET 4.5. Nothing depended on it: no package in the graph declared it, no source file used it, and the built assembly carried zero references to it. `Microsoft.Xrm.Sdk` references `System.IdentityModel` — the BCL assembly, already satisfied by the framework reference — not `microsoft.identitymodel`, which is the likely origin of the confusion.
- **`SQLitePCLRaw.config.e_sqlite3` 3.0.2** — sat amid an otherwise-2.1.11 stack with nothing depending on it at that version. It shipped a second copy of `SQLitePCLRaw.batteries_v2.dll` that conflicted with the 2.1.11 copy the projects actually reference from `bundle_green`, and triggered a package-downgrade error under modern resolution.

### Changed
- `SQLitePCLRaw.bundle_e_sqlite3` 2.1.11 → 2.1.13, so the version built against and the published floor agree.
- Package version 2.2026.3.1 → 2.2026.9.7.
- **`AssemblyVersion` is now pinned to `2.0.0.0`** and no longer tracks the CalVer release. It is the identity the CLR binds against, so bumping it every release forced every consuming application to add or update a binding redirect just to take a patch. `AssemblyFileVersion` carries the real release version (`2.2026.9.7`). `AssemblyVersion` will change only on a breaking release.
- Previously these had also drifted: the published 2.2026.3.1 package contained an assembly stamped 2.2026.3.2.
- **Consumers upgrading from 2.2026.3.1 can remove any binding redirect** they were carrying for this assembly, or point it at `2.0.0.0`.
- Copyright updated to 2019-2026.

### Fixed — Concurrency
- **Reads were completely unsynchronized.** `PersistentBlobCache.Read()` executed against the shared `SQLiteConnection` while holding no lock, and writes held a separate `_writeSemaphore`. sqlite-net's `SQLiteConnection` is not thread-safe and this class holds a single connection, so a read could run against a connection a concurrent write was mutating. Reads and writes now share one `_dbSemaphore`, so every operation against the connection is serialized.
- **A throwing `WaitAsync` could corrupt the semaphore count.** `CreateConnection()` and `Write()` both acquired their semaphore *inside* the `try`, so if the wait threw — `ObjectDisposedException` after `Dispose()`, for example — the `finally` released a permit that had never been taken. The acquire now happens before the `try`. Calls made after `Dispose()` now surface a clean `ObjectDisposedException` instead of leaving the semaphore in a corrupt state.
- **The connection was published before it was initialised.** `CreateConnection()` assigned `_db` and only then set the journal mode and created the schema. A caller reaching the non-null fast path in between could query a `CacheItem` table that did not exist yet. The connection is now built in a local and assigned only once the schema is in place.
- **`GetObjectsCreatedAt<T>()` walked its `keys` argument twice without materializing it**, so a single-use sequence (an iterator, a `yield return` method, a LINQ chain over a stream) silently came back empty the second time and produced a partial result. Now materialized once.
- **`Get(IEnumerable<string>, string)` and `GetObjectsCreatedAt<T>()` issued every query twice.** Both built a lazy `IEnumerable<Task<…>>`, awaited it with `Task.WhenAll`, then re-enumerated it via `.Result` — which created a fresh set of tasks and re-ran every chunked query against the database. Both sequences are now materialized with `.ToArray()` before being awaited.

### Fixed
- `.nuspec` `<repository>` metadata declared branch `main`; the repository's default branch is `master`.

### Performance
- **No measurable change is expected, and none is claimed.** The two removed packages were never loaded at runtime. The SQLite 3.49.1 → 3.53.3 jump is four minor releases of incremental query-planner work that this library's access pattern does not exercise — reads are single-row primary-key lookups against a `WITHOUT ROWID` table, and writes are already batched inside transactions. `sqlite-net-pcl` is unchanged at 1.9.172.

### Added
- Five tests covering the concurrency fixes: mixed concurrent reads and writes, concurrent operations against an unopened connection, concurrent `CreateConnection()` calls, bulk `Get` across multiple internal chunks, and single-use-sequence handling in `GetCreatedAt`. The last of these fails against the pre-fix code; the others are stress coverage and do not deterministically reproduce a race.

### Verified
- Clean restore with both removed packages physically absent from the restore folder — NuGet never requested them.
- `Rebuild` succeeds in both Debug and Release (x64).
- 48/48 unit tests pass, stable across repeated runs.
- The built assembly stamps `AssemblyVersion 2.0.0.0` and `FileVersion 2.2026.9.7`.
- The deployed `e_sqlite3.dll` reports SQLite 3.53.3.
- A NuGet audit across all remaining packages in both projects reports no known vulnerabilities at any severity.

---

## [2.0.0] - 2025-01-XX

### 🎉 Major Release - .NET Framework 4.8 Upgrade
Expand Down Expand Up @@ -142,6 +204,7 @@ using Xrm.Persistent.Collections;
| From Version | To Version | Breaking Changes | Migration Effort | Database Compatible |
|--------------|------------|------------------|------------------|---------------------|
| 1.2022.10.3 | 2.2025.1.15 | Namespace only | Low (1-2 hours) | ✅ Yes |
| 2.2026.3.1 | 2.2026.9.7 | None | None (drop-in) | ✅ Yes |


---
Expand Down
158 changes: 158 additions & 0 deletions Xrm.Persistent.Collections.Tests/PersistentBlobCacheTests.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
namespace Xrm.Persistent.Collections.Backend
{
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
Expand Down Expand Up @@ -284,6 +286,162 @@ await cache.Insert("expired-key", Encoding.UTF8.GetBytes("expired"),
Assert.DoesNotContain("expired-key", keys);
}

[Fact]
public async Task Concurrent_Reads_And_Writes_Do_Not_Throw()
{
// Arrange
await cache.CreateConnection();
const int operations = 150;

// Act - reads and writes issued against the same connection at the same time.
// Before reads were serialized against writes, this raced on a shared
// SQLiteConnection, which sqlite-net does not guard.
var work = new List<Task>();
for (var i = 0; i < operations; i++)
{
var key = "concurrent-" + i;
var payload = Encoding.UTF8.GetBytes("value-" + i);

work.Add(Task.Run(() => cache.Insert(key, payload)));
work.Add(Task.Run(() => cache.GetOrDefault(key, string.Empty)));
work.Add(Task.Run(() => cache.GetAllKeys()));
}

await Task.WhenAll(work);

// Assert - every write landed and is readable
for (var i = 0; i < operations; i++)
{
var stored = await cache.Get("concurrent-" + i);
Assert.Equal("value-" + i, Encoding.UTF8.GetString(stored));
}
}

[Fact]
public async Task Concurrent_Operations_On_Unopened_Connection_Do_Not_Throw()
{
// Arrange - deliberately no CreateConnection() first, so many callers race
// to open the connection at once.

// Act
var work = Enumerable.Range(0, 50)
.Select(i => Task.Run(async () =>
{
await cache.Insert("cold-" + i, Encoding.UTF8.GetBytes("v" + i));
return await cache.GetOrDefault("cold-" + i, string.Empty);
}))
.ToArray();

var results = await Task.WhenAll(work);

// Assert
Assert.All(results, r => Assert.NotEmpty(r));
}

[Fact]
public async Task Get_With_Keys_Spanning_Multiple_Chunks_Returns_All_Values()
{
// Arrange - the internal chunk size is 950, so this spans three chunks
await cache.CreateConnection();
const int count = 2000;

var items = new Dictionary<string, byte[]>();
for (var i = 0; i < count; i++)
{
items["bulk-" + i] = Encoding.UTF8.GetBytes("payload-" + i);
}

await cache.Insert(items);

// Act
var fetched = await cache.Get(items.Keys);

// Assert
Assert.Equal(count, fetched.Count);
for (var i = 0; i < count; i++)
{
Assert.Equal("payload-" + i, Encoding.UTF8.GetString(fetched["bulk-" + i]));
}
}

[Fact]
public async Task GetCreatedAt_Enumerates_The_Supplied_Keys_Only_Once()
{
// Arrange - GetObjectsCreatedAt walks its keys argument twice. Passing a
// sequence that refuses a second enumeration pins that behaviour down.
await cache.CreateConnection();

var items = new Dictionary<string, byte[]>();
for (var i = 0; i < 20; i++)
{
items["stamped-" + i] = Encoding.UTF8.GetBytes("v" + i);
}

await cache.Insert(items);

// Act
var stamps = await cache.GetCreatedAt(new SingleUseSequence(items.Keys));

// Assert - an entry comes back for every key that was supplied
Assert.Equal(items.Count, stamps.Count);
foreach (var key in items.Keys)
{
Assert.True(stamps.ContainsKey(key));
}
}

[Fact]
public async Task Concurrent_CreateConnection_Publishes_A_Fully_Initialised_Connection()
{
// Arrange - many callers open the connection at once. The connection field is
// only assigned after the schema exists, so no caller can observe a connection
// whose CacheItem table has not been created yet.
var opens = Enumerable.Range(0, 64)
.Select(_ => Task.Run(() => cache.CreateConnection()))
.ToArray();

// Act
await Task.WhenAll(opens);

// Assert - the schema is usable immediately afterwards
await cache.Insert("after-open", Encoding.UTF8.GetBytes("ok"));
var stored = await cache.Get("after-open");
Assert.Equal("ok", Encoding.UTF8.GetString(stored));
}

#endregion Public Methods

#region Private Types

/// <summary>
/// A sequence that throws if anything walks it more than once, so that code
/// relying on repeated enumeration of a caller-supplied <see cref="IEnumerable{T}"/>
/// fails loudly instead of silently returning nothing the second time.
/// </summary>
private sealed class SingleUseSequence : IEnumerable<string>
{
private readonly IEnumerable<string> source;
private bool enumerated;

public SingleUseSequence(IEnumerable<string> source)
{
this.source = source;
}

public IEnumerator<string> GetEnumerator()
{
if (enumerated)
{
throw new InvalidOperationException("Sequence was enumerated more than once.");
}

enumerated = true;
return source.GetEnumerator();
}

IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}

#endregion Private Types
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,6 @@
<HintPath>..\packages\Microsoft.CrmSdk.CoreAssemblies.9.0.2.60\lib\net462\Microsoft.Crm.Sdk.Proxy.dll</HintPath>
</Reference>
<Reference Include="Microsoft.CSharp" />
<Reference Include="microsoft.identitymodel, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.IdentityModel.7.0.0\lib\net35\microsoft.identitymodel.dll</HintPath>
</Reference>
<Reference Include="Microsoft.TestPlatform.CoreUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.TestPlatform.ObjectModel.17.10.0\lib\net462\Microsoft.TestPlatform.CoreUtilities.dll</HintPath>
</Reference>
Expand Down Expand Up @@ -239,7 +236,7 @@
<_SQLitePclRawAssemblies Include="$(MSBuildProjectDirectory)\..\packages\SQLitePCLRaw.provider.e_sqlite3.2.1.11\lib\netstandard2.0\SQLitePCLRaw.provider.e_sqlite3.dll" />
<_SQLitePclRawAssemblies Include="$(MSBuildProjectDirectory)\..\packages\SQLitePCLRaw.provider.dynamic_cdecl.2.1.11\lib\netstandard2.0\SQLitePCLRaw.provider.dynamic_cdecl.dll" />
<_SQLitePclRawAssemblies Include="$(MSBuildProjectDirectory)\..\packages\SQLitePCLRaw.bundle_green.2.1.11\lib\net461\SQLitePCLRaw.batteries_v2.dll" />
<_SQLitePclRawNative Include="$(MSBuildProjectDirectory)\..\packages\SQLitePCLRaw.lib.e_sqlite3.2.1.11\runtimes\win-x64\native\e_sqlite3.dll" />
<_SQLitePclRawNative Include="$(MSBuildProjectDirectory)\..\packages\SQLitePCLRaw.lib.e_sqlite3.2.1.13\runtimes\win-x64\native\e_sqlite3.dll" />
<_AdditionalRuntimeAssemblies Include="$(MSBuildProjectDirectory)\..\packages\System.Runtime.CompilerServices.Unsafe.6.1.2\lib\net462\System.Runtime.CompilerServices.Unsafe.dll" />
</ItemGroup>
<Copy SourceFiles="@(_SQLitePclRawAssemblies)" DestinationFolder="$(OutputPath)" SkipUnchangedFiles="true" Condition="'@(_SQLitePclRawAssemblies)' != ''" />
Expand Down
5 changes: 1 addition & 4 deletions Xrm.Persistent.Collections.Tests/packages.config
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,12 @@
<package id="Microsoft.ApplicationInsights" version="2.23.0" targetFramework="net48" />
<package id="Microsoft.Bcl.AsyncInterfaces" version="8.0.0" targetFramework="net48" />
<package id="Microsoft.CrmSdk.CoreAssemblies" version="9.0.2.60" targetFramework="net48" />
<package id="Microsoft.IdentityModel" version="7.0.0" targetFramework="net48" />
<package id="Microsoft.Win32.Registry" version="5.0.0" targetFramework="net48" />
<package id="Newtonsoft.Json" version="13.0.4" targetFramework="net48" />
<package id="sqlite-net-pcl" version="1.9.172" targetFramework="net48" />
<package id="SQLitePCLRaw.bundle_e_sqlite3" version="2.1.11" targetFramework="net48" />
<package id="SQLitePCLRaw.bundle_green" version="2.1.11" targetFramework="net48" />
<package id="SQLitePCLRaw.config.e_sqlite3" version="3.0.2" targetFramework="net48" />
<package id="SQLitePCLRaw.core" version="2.1.11" targetFramework="net48" />
<package id="SQLitePCLRaw.lib.e_sqlite3" version="2.1.11" targetFramework="net48" />
<package id="SQLitePCLRaw.lib.e_sqlite3" version="2.1.13" targetFramework="net48" />
<package id="SQLitePCLRaw.provider.dynamic_cdecl" version="2.1.11" targetFramework="net48" />
<package id="SQLitePCLRaw.provider.e_sqlite3" version="2.1.11" targetFramework="net48" />
<package id="System.Buffers" version="4.6.1" targetFramework="net48" />
Expand Down
Loading
Loading