feat: support stream buffer policies

[ngjaying]

Summary

• add BUFFER_FULL_POLICY="block|dropOldest" to stream definitions
• resolve one buffer policy for the entire rule and reject conflicting policies across multiple streams
• require an explicit stream policy to enable shared-stream backpressure
• make blocked shared-stream delivery cancellable so stopping a subscriber cannot leave the shared topology stuck
• retain the deprecated disableBufferFullDiscard rule option only as a compatibility fallback

Deprecation

The rule option disableBufferFullDiscard is deprecated. It remains supported so existing rules keep their previous behavior, but it is not recommended for new or updated rules. Configure BUFFER_FULL_POLICY on stream definitions instead. The stream-level option is the source of truth and takes precedence over the deprecated rule option.

Compatibility

Existing rules keep their previous buffer behavior. When BUFFER_FULL_POLICY is omitted, non-shared streams fall back to the deprecated disableBufferFullDiscard rule option, whose default remains false. QoS 1/2 does not implicitly change the buffer policy.

A shared stream without BUFFER_FULL_POLICY keeps the legacy dropOldest behavior. As before, disableBufferFullDiscard=true is invalid for such a shared stream. Configure BUFFER_FULL_POLICY="block" on the shared stream to opt into lossless in-memory channel delivery and backpressure.

For rules with multiple streams, every stream must resolve to the same policy. A conflict is reported during planning with the stream names and resolved policies.

This prevents channel-full data loss but does not add checkpoint or source-offset recovery to the shared source component.

Tests

• TZ=UTC go test ./internal/xsql
• go test ./internal/processor
• go test ./internal/topo/planner ./internal/topo ./internal/topo/node
• go test -race ./internal/topo/node ./internal/topo ./internal/topo/planner
• repeated planner policy tests with -count=20

Supersedes #4024.

[codecov]

Codecov Report

❌ Patch coverage is 94.07895% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.10%. Comparing base (dc7d58b) to head (496cd04).
⚠️ Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/topo/node/node.go	82.14%	5 Missing ⚠️
internal/topo/planner/planner_graph.go	92.86%	2 Missing and 1 partial ⚠️
internal/topo/planner/planner.go	97.62%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4133      +/-   ##
==========================================
+ Coverage   71.03%   71.10%   +0.07%     
==========================================
  Files         473      473              
  Lines       55882    56006     +124     
==========================================
+ Hits        39693    39819     +126     
+ Misses      13127    13125       -2     
  Partials     3062     3062              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ngjaying]

Simplified the backpressure implementation in c3abb6f:

• removed per-output buffer policies and AddOutputWithPolicy
• each node now uses its single resolved rule/stream policy
• a shared or connection subtopology fixes its policy at creation and rejects reuse with a conflicting policy
• retained only per-output cancellation state so removing a blocked subscriber still releases the sender

This matches the invariant that one rule has one buffer-full policy and avoids pretending mixed policies can be isolated on a sequential broadcast path.

[ngjaying]

Updated in cfaaa3d to make the new stream policy fully opt-in and backward compatible:

• QoS 1/2 no longer changes the buffer policy implicitly
• DisableBufferFullDiscard is restored from *bool to bool
• conf.go, its added tests, and rule deserialization changes are restored to master
• streams without BUFFER_FULL_POLICY use the legacy rule option/default
• a shared stream without an explicit policy still rejects rule-level disableBufferFullDiscard=true, matching the previous API

The PR description and documentation now reflect these semantics.

[ngjaying]

Added coverage in e10de5a for the new buffer-policy paths:

• resolveBufferFullPolicy: 100%
• graphStreamInfos: 100%, including missing nodes, tables, unsupported types, invalid metadata, and invalid inline options
• defaultNode.AddOutput / RemoveOutput: 100%
• covered replacement cancellation and blocked unbuffered-output removal
• covered both block and case-insensitive dropOldest parser values
• removed the obsolete SrcSubTopo.AddOperator error-return path left from the earlier per-output policy design

Targeted package tests and race repetitions pass locally. Remote CI/Codecov is now recalculating the patch.

[ngjaying]

Clarification: the rule-level disableBufferFullDiscard option is deprecated by this design. It is retained only for backward compatibility when a non-shared stream has no explicit policy. New and updated rules should configure BUFFER_FULL_POLICY="block|dropOldest" on the stream; the stream option takes precedence.