feat(tonic-xds): add gRFC A42 ring-hash picker + member tracking#2695
Conversation
Implements the ring-hash LB picker on the loadbalance/ stack, with ring
construction and the hash-position walk mirroring grpc-go's ringhash balancer:
- RingHashPicker: builds an A42-conformant ring (uniform per-member weighting) —
size = smallest multiple of N >= min_ring_size, clamped to max_ring_size;
entries keyed xxh64("{addr}_{i}", 0); ring held lock-free behind ArcSwap.
pick() reads RouteDecision.request_hash (per-request random fallback), finds
the ring position closest to that hash and walks clockwise to the first ready
host (None/Unavailable if no ring host is ready).
- ChannelPicker::on_members_changed hook (default no-op; P2C inherits it),
delegating to RingHashPicker::rebuild.
- LoadBalancer tracks `members` (full healthy-EDS set, independent of
connection/ejection state) and rebuilds the picker's ring once per discovery
drain. Outlier detection composes for free: ejected hosts stay in the ring
but are not picked (not in `ready`).
Currently it supports uniform weighting and an eager-connect pick that selects
the first ready host. The remaining A42 connection semantics — IDLE-start with
connect-on-pick, queuing while CONNECTING, the TRANSIENT_FAILURE-aware walk,
weight-proportional rings, and aggregated-connectivity-state rules — are gated
on the load balancer's connection model and deferred. The picker is not yet
selected by lb_policy (default-wired in a later change).
Tests: 16 picker unit tests + 1 LoadBalancer member-tracking integration test.
madhurishgupta
force-pushed
the
madhurishgupta/a42-pr2-ring-hash-picker
branch
from
f9b2dcc to
575ebed
Compare
|
|
||
| /// Ring-hash LB configuration (gRFC A42 `ring_hash_lb_config`). | ||
| #[derive(Debug, Clone, Copy)] | ||
| pub(crate) struct RingHashConfig { |
There was a problem hiding this comment.
Maybe add a validation to RingHashConfig? This can prevent massive vector allocation for ring object when the config value is invalid.
There was a problem hiding this comment.
This validation is part of third PR in which CDS wiring will be implemented.
|
Is the plan to only support uniform weighting after the 4 PRs? A42 requires EDS and locality weighting. It's ok to defer it after this PR but to declare proper A42 support adding weight support is needed. |
| for (k, addr) in members.iter().enumerate() { | ||
| let target = (ring_size as u128 * (k as u128 + 1)).div_ceil(n as u128) as u64; | ||
| for i in 0..(target - emitted) { | ||
| let key = format!("{addr}_{i}"); |
There was a problem hiding this comment.
can reuse the same String buffer in each iteration instead of allocating every time
There was a problem hiding this comment.
Makes sense, made the change
Build the per-entry ring key into a single reused String (clear + write!) instead of allocating a fresh String each iteration, turning O(ring_size) allocations per rebuild into O(1). The key contents are unchanged, so the ring is identical (pinned-digest tests still pass).
| /// entries. Each entry is keyed | ||
| /// `xxh64("{addr}_{i}", 0)`, `i` being the member's previous appearance | ||
| /// count, and entries are then sorted by hash. | ||
| fn build_ring(config: &RingHashConfig, members: &IndexSet<EndpointAddress>) -> Vec<RingEntry> { |
There was a problem hiding this comment.
one Rust idiomatic style suggestion: use a new type pattern for Ring(Vec<RingEntry>) so that only valid rings can be represented. Can then move most of the build and pick logic into method on the Ring type, and the picker does minimal delegation between config and the ring.
There was a problem hiding this comment.
Thanks for mentioning this, it makes sense.
Create a new struct Ring with new and pick functions.
Introduce `struct Ring(Vec<RingEntry>)` whose only constructor sorts the entries, so a Ring is always sorted by hash and Ring::pick's binary search is sound by construction rather than by an implicit convention. Move ring building and the hash-position walk onto Ring; RingHashPicker now just extracts the request hash and delegates to the loaded ring.
Correct. Uniform weighting is an intentional initial scope for this PR. |
Add a TODO marking that full A42 sizes the ring by each endpoint's EDS and locality weight; the picker currently uses uniform weights.
3 participants
Summary
Adds the gRFC A42 ring-hash load-balancing picker to the loadbalance stack, giving consistent-hash request affinity: requests carrying the same hash key are routed to the same backend.
What's included
Behavior notes
Testing
Added UTs.
cargo fmt, clippy, and cargo test -p tonic-xds all clean.
##nPlan (A42 series)