Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/data-annotations/continuous-aggregates.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,9 @@ Structured aggregates (those configured through attributes rather than a raw vie

- Duplicate output column names are rejected with an `InvalidOperationException`. The check compares the bucket column, all `[GroupByColumn]` columns, and every `[Aggregate]` alias after resolving them to database column names. A source column that collides with the bucket column name — previously surfacing only at `migrate` time — is now caught at model build.
- A property designated by a property-level `[TimeBucket]` that does not exist on the entity raises an `InvalidOperationException` (this cannot occur through attributes alone, but the same check guards Fluent-configured models).
- An aggregate with no time-bucket designation (property-level `[TimeBucket]`) and no property mapping to the default bucket column `time_bucket` emits a warning through the configured EF logger. The view still exposes a `time_bucket` column, but it cannot be queried through the entity; previously this surfaced only at query time as an opaque Postgres "column does not exist" error. Remedy by designating a property or mapping one to `time_bucket`. This is a warning rather than an exception because deliberately not exposing the bucket is legal.

> :warning: **Note:** Entities scaffolded with a raw view definition are exempt from both checks, because the structured projection fields are unused on that path.
> :warning: **Note:** Entities scaffolded with a raw view definition are exempt from all checks, because the structured projection fields are unused on that path.

## Configuration Options

Expand Down
3 changes: 2 additions & 1 deletion docs/fluent-api/continuous-aggregates.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,8 +203,9 @@ Structured aggregates (those configured through the builders rather than a raw v

- Duplicate output column names are rejected with an `InvalidOperationException`. The check compares the bucket column, all GROUP BY columns, and every aggregate alias after resolving them to database column names. A source column that collides with the bucket column name is caught at model build.
- A property designated via `.WithTimeBucketProperty(...)` that does not exist on the entity raises an `InvalidOperationException`.
- An aggregate with no time-bucket designation (`.WithTimeBucketProperty(...)`) and no property mapping to the default bucket column `time_bucket` emits a warning through the configured EF logger. The view still exposes a `time_bucket` column, but it cannot be queried through the entity; previously this surfaced only at query time as an opaque Postgres "column does not exist" error. Remedy by designating a property or mapping one to `time_bucket`. This is a warning rather than an exception because deliberately not exposing the bucket is legal.

> :warning: **Note:** Entities scaffolded with a raw view definition are exempt from both checks, because the structured projection fields are unused on that path.
> :warning: **Note:** Entities scaffolded with a raw view definition are exempt from all checks, because the structured projection fields are unused on that path.

## Hierarchical Continuous Aggregates

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ public void Configure(EntityTypeBuilder<HourlyStationAggregate> builder)
propertyExpression: source => source.RecordedAt,
timeBucketGroupBy: true)

.WithTimeBucketProperty(x => x.Bucket)
.AddAggregateFunction(
agg => agg.AvgLatitude,
source => source.Location.Coordinates.Latitude,
Expand Down
5 changes: 5 additions & 0 deletions samples/Eftdb.Samples.Shared/Models/HourlySensorAggregate.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@ namespace CmdScale.EntityFrameworkCore.TimescaleDB.Samples.Shared.Models
{
public class HourlySensorAggregate
{
/// <summary>
/// Start of the hour-wide bucket this row summarizes.
/// </summary>
public DateTime TimeBucket { get; set; }

public double AvgPrimaryValue { get; set; }
public double MinPrimaryValue { get; set; }
public double MaxPrimaryValue { get; set; }
Expand Down
5 changes: 5 additions & 0 deletions samples/Eftdb.Samples.Shared/Models/TradeAggregate.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
{
public class TradeAggregate
{
/// <summary>
/// Start of the hour-wide bucket this row summarizes.
/// </summary>
public DateTime TimeBucket { get; set; }

public decimal AveragePrice { get; set; }
public decimal MaxPrice { get; set; }
public decimal MinPrice { get; set; }
Expand Down
5 changes: 5 additions & 0 deletions samples/Eftdb.Samples.Shared/Models/WeatherAggregate.cs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,11 @@ namespace CmdScale.EntityFrameworkCore.TimescaleDB.Samples.Shared.Models
[CompressionPolicy(After = "90 days", ScheduleInterval = "1 day")]
public class WeatherAggregate
{
/// <summary>
/// Start of the day-wide bucket this row summarizes.
/// </summary>
public DateTime TimeBucket { get; set; }

// Avg aggregate function
[Aggregate(EAggregateFunction.Avg, nameof(WeatherData.Temperature))]
public double AverageTemperature { get; set; }
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Scaffolding;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Generators;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.ContinuousAggregate;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.CompressionPolicy;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Scaffolding;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Generators;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.ContinuousAggregate;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.ContinuousAggregatePolicy;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Scaffolding;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Generators;
using CmdScale.EntityFrameworkCore.TimescaleDB.Abstractions;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.Hypertable;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Scaffolding;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Generators;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.Hypertable;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.ReorderPolicy;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Scaffolding;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Generators;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.Hypertable;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.ContinuousAggregate;
Expand Down
1 change: 0 additions & 1 deletion src/Eftdb.Design/Generators/TimescaleCSharpHelper.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Generators;
using Microsoft.EntityFrameworkCore.Design.Internal;
using Microsoft.EntityFrameworkCore.Storage;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.ContinuousAggregatePolicy;
using CmdScale.EntityFrameworkCore.TimescaleDB.Abstractions;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.ContinuousAggregate;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Generators;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Scaffolding;
using CmdScale.EntityFrameworkCore.TimescaleDB.Internals;
using Microsoft.EntityFrameworkCore;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.CompressionPolicy;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.ContinuousAggregate;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.ContinuousAggregatePolicy;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Generators;
using CmdScale.EntityFrameworkCore.TimescaleDB.Operations;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Migrations.Design;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ private static void ValidateViewColumns(IModel model, IEntityType entityType)
return;
}

StoreObjectIdentifier? aggregateStoreIdentifier = GetStoreObjectIdentifier(entityType);
StoreObjectIdentifier? aggregateStoreIdentifier = EntityStoreObjectResolver.GetStoreObjectIdentifier(entityType);
if (aggregateStoreIdentifier == null)
{
return;
Expand All @@ -53,7 +53,7 @@ private static void ValidateViewColumns(IModel model, IEntityType entityType)
string bucketColumnName = ResolveBucketColumnName(entityType, aggregateStoreIdentifier.Value, materializedViewName);

IEntityType? parentEntityType = ResolveParent(model, entityType);
StoreObjectIdentifier? parentStoreIdentifier = parentEntityType == null ? null : GetStoreObjectIdentifier(parentEntityType);
StoreObjectIdentifier? parentStoreIdentifier = parentEntityType == null ? null : EntityStoreObjectResolver.GetStoreObjectIdentifier(parentEntityType);

List<string> outputColumns = [bucketColumnName];
outputColumns.AddRange(ResolveGroupByColumns(entityType, parentEntityType, parentStoreIdentifier));
Expand All @@ -65,7 +65,7 @@ private static void ValidateViewColumns(IModel model, IEntityType entityType)
if (!seen.Add(column))
{
throw new InvalidOperationException(
$"The continuous aggregate '{DisplayName(entityType)}' (materialized view '{materializedViewName}') " +
$"The continuous aggregate '{EntityStoreObjectResolver.DisplayName(entityType)}' (materialized view '{materializedViewName}') " +
$"produces the output column '{column}' more than once. Rename the conflicting property or use " +
$"WithTimeBucketProperty to map the bucket column to a distinct property.");
}
Expand All @@ -88,7 +88,7 @@ private static string ResolveBucketColumnName(IEntityType entityType, StoreObjec
if (property == null)
{
throw new InvalidOperationException(
$"The continuous aggregate '{DisplayName(entityType)}' (materialized view '{materializedViewName}') " +
$"The continuous aggregate '{EntityStoreObjectResolver.DisplayName(entityType)}' (materialized view '{materializedViewName}') " +
$"designates '{targetPropertyName}' as its time-bucket property, but no such property exists on the entity.");
}

Expand Down Expand Up @@ -154,24 +154,5 @@ private static IEnumerable<string> ResolveAggregateAliasColumns(IEntityType enti
string? parentName = entityType.FindAnnotation(ContinuousAggregateAnnotations.ParentName)?.Value as string;
return string.IsNullOrWhiteSpace(parentName) ? null : ParentEntityTypeResolver.Resolve(model, parentName);
}

private static StoreObjectIdentifier? GetStoreObjectIdentifier(IEntityType entityType)
{
string? tableName = entityType.GetTableName();
if (!string.IsNullOrWhiteSpace(tableName))
{
return StoreObjectIdentifier.Table(tableName, entityType.GetSchema());
}

string? viewName = entityType.GetViewName();
if (!string.IsNullOrWhiteSpace(viewName))
{
return StoreObjectIdentifier.View(viewName, entityType.GetViewSchema() ?? entityType.GetSchema());
}

return null;
}

private static string DisplayName(IEntityType entityType) => entityType.ClrType?.Name ?? entityType.Name;
}
}
39 changes: 39 additions & 0 deletions src/Eftdb/Internals/EntityStoreObjectResolver.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata;

namespace CmdScale.EntityFrameworkCore.TimescaleDB.Internals
{
/// <summary>
/// Resolves the store object (table or view) an entity maps to, and a human-readable display
/// name for diagnostics.
/// </summary>
internal static class EntityStoreObjectResolver
{
/// <summary>
/// Returns the table store object when the entity maps to a table, otherwise the view store
/// object, or <c>null</c> when the entity maps to neither.
/// </summary>
public static StoreObjectIdentifier? GetStoreObjectIdentifier(IEntityType entityType)
{
string? tableName = entityType.GetTableName();
if (!string.IsNullOrWhiteSpace(tableName))
{
return StoreObjectIdentifier.Table(tableName, entityType.GetSchema());
}

string? viewName = entityType.GetViewName();
if (!string.IsNullOrWhiteSpace(viewName))
{
return StoreObjectIdentifier.View(viewName, entityType.GetViewSchema() ?? entityType.GetSchema());
}

return null;
}

/// <summary>
/// Returns the CLR type name for diagnostics, falling back to the EF entity-type name for
/// shared-type or keyless entities without a distinct CLR type.
/// </summary>
public static string DisplayName(IEntityType entityType) => entityType.ClrType?.Name ?? entityType.Name;
}
}
87 changes: 87 additions & 0 deletions src/Eftdb/Internals/TimescaleModelValidator.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.ContinuousAggregate;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Diagnostics;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.Extensions.Logging;
using Npgsql.EntityFrameworkCore.PostgreSQL.Infrastructure.Internal;

namespace CmdScale.EntityFrameworkCore.TimescaleDB.Internals
{
#pragma warning disable EF1001 // Npgsql internal validator/options are the intended base to preserve Npgsql validations.
/// <summary>
/// Extends Npgsql's model validator with TimescaleDB-specific model warnings. Derives from
/// <see cref="NpgsqlModelValidator"/> all Npgsql validations still run via <see cref="Validate"/>'s <c>base</c> call.
/// </summary>
internal class TimescaleModelValidator(
ModelValidatorDependencies dependencies,
RelationalModelValidatorDependencies relationalDependencies,
INpgsqlSingletonOptions npgsqlSingletonOptions)
: NpgsqlModelValidator(dependencies, relationalDependencies, npgsqlSingletonOptions)
{
public override void Validate(IModel model, IDiagnosticsLogger<DbLoggerCategory.Model.Validation> logger)
{
base.Validate(model, logger);

foreach (IEntityType entityType in model.GetEntityTypes())
{
WarnWhenBucketColumnUnmapped(entityType, logger);
}
}

/// <summary>
/// Warns when a structured continuous aggregate leaves its bucket column unmapped: no
/// <see cref="ContinuousAggregateAnnotations.TimeBucketTargetProperty"/> designation and no
/// property resolving to the fallback bucket column. Such a model is legal — the entity simply
/// cannot query the bucket — but any LINQ touching the (absent) bucket property fails at runtime
/// with an opaque "column does not exist" error, so the warning points at the fix. Entities
/// carrying a raw <see cref="ContinuousAggregateAnnotations.ViewDefinition"/> are exempt, matching
/// the scaffolded raw-definition exemption in the view-column validation convention.
/// </summary>
private static void WarnWhenBucketColumnUnmapped(IEntityType entityType, IDiagnosticsLogger<DbLoggerCategory.Model.Validation> logger)
{
string? materializedViewName = entityType.FindAnnotation(ContinuousAggregateAnnotations.MaterializedViewName)?.Value as string;
if (string.IsNullOrWhiteSpace(materializedViewName))
{
return;
}

string? viewDefinition = entityType.FindAnnotation(ContinuousAggregateAnnotations.ViewDefinition)?.Value as string;
if (!string.IsNullOrWhiteSpace(viewDefinition))
{
return;
}

string? targetPropertyName = entityType.FindAnnotation(ContinuousAggregateAnnotations.TimeBucketTargetProperty)?.Value as string;
if (!string.IsNullOrWhiteSpace(targetPropertyName))
{
return;
}

StoreObjectIdentifier? aggregateStoreIdentifier = EntityStoreObjectResolver.GetStoreObjectIdentifier(entityType);
if (aggregateStoreIdentifier == null)
{
return;
}

// Without a designation the bucket column is always the fallback; warn only when nothing maps to it.
string bucketColumn = DefaultValues.ContinuousAggregateTimeBucketColumnName;
string? mappedColumn = ColumnNameResolver.Resolve(entityType, bucketColumn, aggregateStoreIdentifier.Value);
if (!string.IsNullOrWhiteSpace(mappedColumn))
{
return;
}

logger.Logger.LogWarning(
"The continuous aggregate '{Aggregate}' (materialized view '{MaterializedView}') exposes its bucket column as " +
"'{BucketColumn}', but no property maps to that column, so the bucket cannot be queried through the entity. " +
"Designate the bucket property with WithTimeBucketProperty(...), annotate a property with [TimeBucket], or map a " +
"property to '{BucketColumnFix}' with HasColumnName.",
EntityStoreObjectResolver.DisplayName(entityType),
materializedViewName,
bucketColumn,
bucketColumn);
}
}
}
#pragma warning restore EF1001
1 change: 1 addition & 0 deletions src/Eftdb/TimescaleDbContextOptionsBuilderExtensions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ public void ApplyServices(IServiceCollection services)
services.AddSingleton<IConventionSetPlugin, TimescaleDbConventionSetPlugin>();
services.AddScoped<IMigrationsModelDiffer, TimescaleMigrationsModelDiffer>();
services.Replace(ServiceDescriptor.Scoped<IMigrationsSqlGenerator, TimescaleDbMigrationsSqlGenerator>());
services.Replace(ServiceDescriptor.Singleton<IModelValidator, TimescaleModelValidator>());
services.TryAddEnumerable(
ServiceDescriptor.Scoped<IMethodCallTranslatorPlugin, TimescaleDbMethodCallTranslatorPlugin>());
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.CompressionPolicy;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.CompressionPolicy;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.Hypertable;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Scaffolding;
using Microsoft.EntityFrameworkCore.Scaffolding.Metadata;
using static CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.CompressionPolicy.CompressionPolicyScaffoldingExtractor;

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.CompressionPolicy;
#pragma warning disable EF1001 // IOperationReporter and AnnotationCodeGeneratorDependencies are design-time internals.
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.CompressionPolicy;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.Hypertable;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.CompressionPolicy;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Generators;
using CmdScale.EntityFrameworkCore.TimescaleDB.Operations;
using CmdScale.EntityFrameworkCore.TimescaleDB.Tests.Utils;
using Microsoft.EntityFrameworkCore.Design;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.ContinuousAggregate;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.ContinuousAggregate;
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Scaffolding;
using Microsoft.EntityFrameworkCore.Scaffolding.Metadata;
using static CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.ContinuousAggregate.ContinuousAggregateScaffoldingExtractor;

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
using CmdScale.EntityFrameworkCore.TimescaleDB.Design.Features.ContinuousAggregate;
#pragma warning disable EF1001 // IOperationReporter and AnnotationCodeGeneratorDependencies are design-time internals.
using CmdScale.EntityFrameworkCore.TimescaleDB.Abstractions;
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.ContinuousAggregate;
Expand Down
Loading
Loading