Skip to content

REFACTOR_PROGRESS.md

Latest commit

 

History

History
223 lines (178 loc) · 7.58 KB

REFACTOR_PROGRESS.md

File metadata and controls

223 lines (178 loc) · 7.58 KB
Raw

Tocin Monoio Refactor Progress

Completed Tasks ✅

1. Updated Decoder Trait with GAT (Generic Associated Types)

  • File: tocin/src/codec/mod.rs
  • Changes:
    • Changed Decoder::Item to Decoder::Item<'a> (GAT)
    • Added arena: &'a defiant::Arena parameter to decode() method
    • Updated Codec trait to use type Decode<'a> instead of type Decode
    • Updated trait bounds to use for<'a> Decoder<Item<'a> = ...>

2. Updated tocin-defiant Codec Implementation

  • File: tocin-defiant/src/codec.rs
  • Changes:
    • Implemented GAT in ProstDecoder
    • Updated decode() to accept arena parameter
    • Changed from creating temporary arena to using passed arena
    • Note: Currently creates temp arena in decode_chunk (line 381 of decode.rs) - needs future refactor

3. Removed tokio-stream Dependencies

  • Files:
    • tocin/src/codec/encode.rs
    • tocin/src/client/grpc.rs
    • tocin/src/server/grpc.rs
    • tocin/src/request.rs
    • tocin/src/codec/decode.rs
  • Changes:
    • Replaced use tokio_stream::Stream with use futures_core::Stream
    • Replaced use tokio_stream::StreamExt with use futures_util::StreamExt
    • Replaced tokio_stream::adapters::Fuse with futures_util::stream::Fuse

4. Removed SyncWrapper

  • File: tocin/src/codec/decode.rs
  • Changes:
    • Removed SyncWrapper from Streaming<T> struct
    • Changed decoder: SyncWrapper<Box<...>> to decoder: Box<...>
    • Changed body: SyncWrapper<Body> to body: Body
    • Updated all SyncWrapper::new() calls
    • Rationale: Monoio is thread-per-core, so !Send types are acceptable

5. Disabled Tower Dependencies

  • Files:
    • tocin/src/service/interceptor.rs
    • tocin/src/service/layered.rs
    • tocin/src/service/recover_error.rs
    • tocin/src/service/mod.rs
  • Changes:
    • Commented out all tower-dependent code with /* ... */
    • Kept Interceptor trait (core functionality)
    • Added TODO comments for re-implementation
    • Disabled exports in service/mod.rs

Current Status 🚧

tocin Package Status

  • ✅ Compiles with warnings only (no errors)
  • ⚠️ Has unused import warnings
  • ⚠️ Has feature flag warnings (expected - features removed from Cargo.toml)

Remaining Compilation Errors (Workspace Level)

The main blocking issues are:

  1. GAT Lifetime Parameters Missing - error[E0107]

    • Streaming<T::Decode> needs to be Streaming<...> with proper lifetime handling
    • Codec<Decode = M2> trait bounds need updating to Decode<'a> = M2
    • Locations:
      • tocin/src/server/grpc.rs:401
      • tocin/src/client/grpc.rs:218, 238, 274
    • Root Cause: Streaming<T> type assumes T is a concrete type, but with GATs, decoded items have lifetime parameters

  2. Streaming Type Refactor Needed

    • Current: Streaming<T> where T: 'static
    • Target: Need to handle Streaming<T> where T has lifetime from arena
    • This is a fundamental design change mentioned in INTEGRATION_PLAN.md

  3. Missing tokio-stream in Other Packages

    • Several other workspace packages still use tokio_stream
    • These will need similar refactoring

Next Steps 📋

Immediate (Blocking Compilation)

  1. Refactor Streaming Type

    • Option A: Make Streaming<T> work with GAT types
    • Option B: Use callback pattern instead of Stream trait (per INTEGRATION_PLAN.md)
    • Option C: Temporary: Use owned types for now, optimize with arena later

  2. Fix GAT Trait Bounds

    • Update all Codec<Decode = M> to handle Decode<'a>
    • May need higher-ranked trait bounds for<'a> Codec<Decode<'a> = M>

Medium Term

  1. Remove Tokio-Based Transport Layer

    • Remove tocin/src/transport/ hyper/tokio code
    • Files to update:
      • tocin/src/transport/server/mod.rs
      • tocin/src/transport/server/conn.rs
      • tocin/src/transport/server/incoming.rs

  2. Create Monoio-Based Transport Layer

    • Implement using monoio-http
    • Add arena support at connection level
    • Per-connection arena ownership (not pooled)

  3. Update Service Traits with Arena Lifetimes

    • Add arena: &'a Arena to service method signatures
    • Change from #[async_trait] to #[async_trait(?Send)]
    • Implement callback pattern for streaming

  4. Update tocin-build Codegen

    • Generate traits with arena parameters
    • Generate #[async_trait(?Send)]
    • Split client streaming into on_message + finish
    • Server streaming uses callback instead of Stream

Key Design Decisions

Arena Ownership Strategy

  • Per-connection arena (not pooled, not thread-local)
  • Connection handler owns arena for its lifetime
  • Reset between requests for bounded memory
  • Memory profile: num_connections × peak_message_size

Runtime

  • Monoio (thread-per-core) instead of Tokio (work-stealing)
  • Tasks never migrate threads (!Send is acceptable)
  • Rationale: Arena references are !Send, require thread pinning

Stream Handling

  • Callback pattern instead of Stream trait
  • Framework manages stream loop
  • User implements per-message handlers
  • Solves lifetime issues with arena-borrowed data

Files Modified

Core Codec Layer

  • ✅ tocin/src/codec/mod.rs - Decoder trait with GAT
  • ✅ tocin/src/codec/decode.rs - Updated Streaming (partial)
  • ✅ tocin/src/codec/encode.rs - Removed tokio-stream
  • ✅ tocin-defiant/src/codec.rs - Arena-aware decoder

Service Layer

  • ✅ tocin/src/service/interceptor.rs - Tower code disabled
  • ✅ tocin/src/service/layered.rs - Tower code disabled
  • ✅ tocin/src/service/recover_error.rs - Tower code disabled
  • ✅ tocin/src/service/mod.rs - Updated exports

gRPC Handlers

  • ✅ tocin/src/server/grpc.rs - Removed tokio-stream
  • ✅ tocin/src/client/grpc.rs - Removed tokio-stream
  • ⚠️ Both need GAT updates

Other

  • ✅ tocin/src/request.rs - Removed tokio-stream

Dependencies Status

Removed (from tocin/Cargo.toml)

  • ✅ tokio (except features still used by other deps)
  • ✅ hyper
  • ✅ hyper-util
  • ✅ tower
  • ✅ tower-service
  • ✅ tower-layer
  • ✅ h2
  • ✅ axum
  • ✅ tokio-stream
  • ✅ sync_wrapper

Added (to workspace Cargo.toml)

  • ✅ monoio = { version = "0.2", features = ["sync"] }
  • ✅ monoio-http = "0.3"
  • ✅ defiant = { path = "../defiant/defiant" }
  • ✅ futures-core = "0.3"
  • ✅ futures-util = "0.3"

Technical Debt / TODOs

  1. Arena in decode_chunk (tocin/src/codec/decode.rs:381)

    • Currently creates temp arena per decode
    • Need to thread arena from connection handler

  2. Tower Integration

    • All tower-dependent middleware is disabled
    • Need to decide: re-implement without tower, or re-add tower conditionally

  3. Feature Flags

    • Removed server, channel, router features from Cargo.toml
    • Getting warnings about these missing features
    • Need to restore or update feature flag usage

  4. Streaming Refactor

    • Major: Replace Stream trait with callback pattern
    • See INTEGRATION_PLAN.md sections on "Streaming Challenge"

Performance Targets

From INTEGRATION_PLAN.md:

Current Tonic

  • Small messages (12 bytes): 237-253ns
  • Medium messages (1KB): ~400ns
  • Large messages (84KB): 105µs
  • Many allocations: 100+ per decode

Target with Defiant + Arena

  • Goal: <150ns for 10KB messages (~50% improvement)
  • 1-2 allocations per message (vs 100+)
  • Single-copy decode
  • Zero allocations after arena warmup

References

  • INTEGRATION_PLAN.md - Complete integration strategy
  • /home/dan/Development/en/memory/conversation_including_api.txt - Design discussions
  • Defiant: github.com/dwerner/defiant
  • Monoio: github.com/bytedance/monoio
  • Monoio-http: github.com/monoio-rs/monoio-http