Skip to content

Strategy modes: fast / reliable / reliable-fast #4

Description

@BleedingDev

Goal

Offer clear service modes that let customers choose the trade‑off between speed, reliability, and price.

Who Uses This

  • Customers who value faster UX (fast mode).
  • Customers who value higher quality/coverage (reliable mode).
  • Premium customers who want both speed and reliability (reliable-fast).

What “Done” Means (Business)

  • The API supports three modes: fast, reliable, reliable-fast.
  • Each mode has clear, documented semantics (what customers can expect).
  • Premium mode can be gated by entitlements (integration point for billing), without breaking existing users.
  • Behavior remains deterministic and predictable across modes.

Constraints

  • Autocomplete must remain responsive (low latency expectations).
  • Backward compatibility for existing integrations.

Out of Scope

  • Implementing billing itself.
  • Provider-specific commercial agreements or provider onboarding.

Technical notes: #4 (comment)

Activity

  1. coderabbitai commented on Dec 28, 2025

    @coderabbitai

    📝 CodeRabbit Plan Mode

    Generate an implementation plan and prompts that you can use with your favorite coding agent.

    • Create Plan
    Examples

    🔗 Similar Issues

    Related Issues

    👤 Suggested Assignees

    🧪 Issue enrichment is currently in open beta.

    You can configure auto-planning by selecting labels in the issue_enrichment configuration.

    To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

    issue_enrichment:
      auto_enrich:
        enabled: false

    💬 Have feedback or questions? Drop into our discord!

  2. BleedingDev commented on Dec 28, 2025

    @BleedingDev
    ContributorAuthor

    Technical Notes

    API & Schema

    • Extend packages/rpc/src/suggest.ts to include reliable-fast.
    • apps/service-bun/src/request.ts continues to default to reliable.

    Execution Semantics

    • fast: single provider, strict timeout, fail-fast.
    • reliable: multi-stage serial fallbacks.
    • reliable-fast: parallel providers, merge + dedupe + return within a strict overall budget.

    Cache Key Decision

    Current cache key includes the chosen strategy. Proposed: make cache shared by query only so different strategies benefit from the same cached result.

    • Change apps/service-bun/src/cache.ts key generation accordingly.
    • Optional migration strategy: read both the new key and legacy key for a transition period.

    Entitlements Hook (integration point)

    • Gate reliable-fast in hosted mode via the payments/entitlements layer (implemented in the payments issue).

    Tests

    • Schema acceptance for 3 modes.
    • Plan selection behavior (serial vs parallel).
    • Cache key behavior and hit consistency across strategies.
  3. BleedingDev commented on Dec 30, 2025

    @BleedingDev
    ContributorAuthor

    Status update: fast + reliable strategies are implemented in the service (apps/service-bun/src/service.ts) and documented; reliable-fast is still marked as planned in docs (apps/docs/content/en/explanation/strategies.md), so this remains open for that mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions