Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,25 @@ Sample results (net10.0, BenchmarkDotNet, Windows 11):

Async overhead is negligible (~1–3% time, 0–5% allocations).

## Streaming mapped rows and server diagnostics

`NzCommand.ExecuteRowsAsync` maps each row while the reader is active, so callers can process large result sets without materializing them first:

```csharp
await using var command = connection.CreateCommand("SELECT ID, NAME FROM MY_TABLE");
await foreach (var item in command.ExecuteRowsAsync(
reader => new Item(reader.GetInt64(0), reader.GetString(1)), cancellationToken))
{
await ProcessAsync(item, cancellationToken);
}
```

The mapper is synchronous and the returned `IAsyncEnumerable<T>` streams rows; cancellation is passed to database reads, and disposing the enumeration closes the reader. `command.Notices` contains the notices from that command's most recent execution. The existing connection-wide `NoticeReceived` event remains available.

For server failures, `NetezzaException.Message` contains the primary backend message. `SqlState`, `Severity`, `Detail`, and `Hint` expose structured fields when supplied. `RawResponse` keeps the complete decoded backend payload, and `Diagnostics` provides all fields by protocol code, including fields unknown to this driver.

The runnable [examples project](src/examples/JustyBase.NetezzaDriver.Examples) includes mapped streaming in its async example and server diagnostic fields in its error-handling example.

## Testing

```bash
Expand Down
64 changes: 64 additions & 0 deletions src/JustyBase.NetezzaDriver.Benchmarks/FieldAccessBench.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
using BenchmarkDotNet.Attributes;

namespace JustyBase.NetezzaDriver.Benchmarks;

/// <summary>
/// Measures field-access choices while streaming the same deterministic rows
/// from a live Netezza table. The returned checksum guards against accidentally
/// benchmarking a path that does not consume the values.
/// </summary>
[MemoryDiagnoser]
public class FieldAccessBench
{
private const string Query = "SELECT ROWID::BIGINT AS ID64 FROM JUST_DATA..FACTPRODUCTINVENTORY ORDER BY ROWID LIMIT 100000";
private NzConnection _connection = null!;

[GlobalSetup]
public void Setup()
{
_connection = new NzConnection(Config.UserName, Config.Password, Config.Host, Config.DbName, Config.Port);
_connection.Open();
}

[GlobalCleanup]
public void Cleanup() => _connection.Dispose();

[Benchmark(Baseline = true)]
public long GetValueAndCast()
{
using var command = _connection.CreateCommand(Query);
using var reader = command.ExecuteReader();
long checksum = 0;
while (reader.Read())
{
checksum = unchecked(checksum + (long)reader.GetValue(0));
}
return checksum;
}

[Benchmark]
public long GenericGetFieldValue()
{
using var command = _connection.CreateCommand(Query);
using var reader = command.ExecuteReader();
long checksum = 0;
while (reader.Read())
{
checksum = unchecked(checksum + reader.GetFieldValue<long>(0));
}
return checksum;
}

[Benchmark]
public long TypedGetInt64()
{
using var command = _connection.CreateCommand(Query);
using var reader = command.ExecuteReader();
long checksum = 0;
while (reader.Read())
{
checksum = unchecked(checksum + reader.GetInt64(0));
}
return checksum;
}
}
30 changes: 29 additions & 1 deletion src/JustyBase.NetezzaDriver.Benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,32 @@
| Method | Mean | Error | StdDev | Allocated |
|---------------------------------- |--------:|---------:|---------:|----------:|
| ExternalUnloadAndLoadNz | 1.690 s | 0.0165 s | 0.0138 s | 28.6 KB |
| ExternalUnloadAndLoadOriginalOdbc | 1.683 s | 0.0058 s | 0.0054 s | 7.9 KB |
| ExternalUnloadAndLoadOriginalOdbc | 1.683 s | 0.0058 s | 0.0054 s | 7.9 KB |

### Generic field access experiment

`FieldAccessBench` reads the same 100,000 `BIGINT` values from
`JUST_DATA..FACTPRODUCTINVENTORY` and checks a checksum. The following
before/after runs evaluated a direct `GetFieldValue<long>` fast path; the
candidate was removed because the improvement was not consistent across JIT
and NativeAOT and allocation savings did not meet the 10% acceptance threshold.

| Runtime | Version | Method | Mean | Allocated |
|---------|---------|--------|-----:|----------:|
| .NET 10 JIT, before | existing cast | 474.5 ms | 60.28 MB |
| .NET 10 JIT, before | `GetFieldValue<long>` | 507.6 ms | 60.28 MB |
| .NET 10 JIT, candidate | `GetFieldValue<long>` | 490.8 ms | 57.99 MB |
| .NET 10 NativeAOT, before | existing cast | 328.5 ms | 60.28 MB |
| .NET 10 NativeAOT, before | `GetFieldValue<long>` | 347.5 ms | 60.28 MB |
| .NET 10 NativeAOT, candidate | `GetFieldValue<long>` | 358.7 ms | 58.00 MB |

Each run used three measured iterations and one warmup. The candidate saved
about 24 bytes per row (3.8% of total measured allocations), while its generic
access time improved 3.3% in JIT and regressed 3.2% in NativeAOT. The timing
spread and opposite direction do not establish a throughput gain. The benchmark
can be rerun with:

```bash
dotnet run -c Release --project src/JustyBase.NetezzaDriver.Benchmarks --framework net10.0 -- \
--filter '*FieldAccessBench*' --runtimes net10.0 --iterationCount 3 --warmupCount 1
```
44 changes: 44 additions & 0 deletions src/JustyBase.NetezzaDriver.Tests/AsyncTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,50 @@ public async Task ExecuteReaderAsync_ShouldReadLargeResultSet()
Assert.True(rows >= 1000);
}

[Fact]
public async Task ExecuteRowsAsync_ShouldMapRowsIncrementallyAndCloseReaderOnEarlyExit()
{
var ct = TestContext.Current.CancellationToken;
await using NzConnection connection = new NzConnection(Config.UserName, Config.Password, Config.Host, Config.DbName, Config.Port);
await connection.OpenAsync(ct);
await using var command = connection.CreateCommand("SELECT 1 AS VALUE UNION ALL SELECT 2 UNION ALL SELECT 3");

List<int> values = [];
await foreach (int value in command.ExecuteRowsAsync(reader => reader.GetInt32(0), ct))
{
values.Add(value);
}

Assert.Equal([1, 2, 3], values);

await foreach (int _ in command.ExecuteRowsAsync(reader => reader.GetInt32(0), ct))
{
break;
}

await using var followUp = connection.CreateCommand("SELECT 42 AS VALUE");
Assert.Equal(42, await followUp.ExecuteScalarAsync(ct));
}

[Fact]
public async Task ExecuteRowsAsync_ShouldDisposeReaderWhenMapperThrows()
{
var ct = TestContext.Current.CancellationToken;
await using NzConnection connection = new NzConnection(Config.UserName, Config.Password, Config.Host, Config.DbName, Config.Port);
await connection.OpenAsync(ct);
await using var command = connection.CreateCommand("SELECT 1 AS VALUE UNION ALL SELECT 2");

await Assert.ThrowsAsync<InvalidOperationException>(async () =>
{
await foreach (int _ in command.ExecuteRowsAsync<int>(_ => throw new InvalidOperationException("mapping failed"), ct))
{
}
});

await using var followUp = connection.CreateCommand("SELECT 42 AS VALUE");
Assert.Equal(42, await followUp.ExecuteScalarAsync(ct));
}

[Fact]
public async Task Reader_GetBytesAndGetChars_ShouldWorkForTextValues()
{
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
using System.Text;

namespace JustyBase.NetezzaDriver.Tests;

[Trait("Category", "Unit")]
public sealed class BackendDiagnosticResponseParserTests
{
[Fact]
public void Parse_ReadsStructuredErrorAndPreservesUnknownDiagnosticFields()
{
const string payload = "SERROR\0VERROR\0C42601\0Msyntax error\0Dunexpected token\0Hcheck the query\0Zserver extension\0\0";
var response = BackendDiagnosticResponseParser.Parse(Encoding.UTF8.GetBytes(payload));

Assert.Equal("syntax error", response.Message);
Assert.Equal("ERROR", response.Severity);
Assert.Equal("42601", response.SqlState);
Assert.Equal("unexpected token", response.Detail);
Assert.Equal("check the query", response.Hint);
Assert.Equal("server extension", response.Diagnostics['Z']);
Assert.Equal(payload, response.RawResponse);

var exception = new NetezzaException(response);
Assert.Equal("syntax error", exception.Message);
Assert.Equal("42601", exception.SqlState);
Assert.Equal(payload, exception.RawResponse);
}

[Fact]
public void Parse_PrefersNonLocalizedSeverity()
{
var response = BackendDiagnosticResponseParser.Parse(
Encoding.UTF8.GetBytes("Slocalized\0Vnonlocalized\0Mmessage\0\0"));

Assert.Equal("nonlocalized", response.Severity);
}

[Theory]
[InlineData("permission denied\0", "permission denied")]
[InlineData("\0\0", "Netezza backend returned an empty error response")]
public void Parse_UsesPlainTextFallback(string payload, string expectedMessage)
{
var response = BackendDiagnosticResponseParser.Parse(Encoding.UTF8.GetBytes(payload));

Assert.Equal(expectedMessage, response.Message);
Assert.Equal(payload, response.RawResponse);
Assert.Empty(response.Diagnostics);
Assert.Null(response.SqlState);
}
}
Loading
Loading