feat: AI Crate Digger — conversational music recommendations via graph-informed dialogue

[SimplicityGuy]

Overview

A conversational recommendation engine that combines the knowledge graph with LLM dialogue to help users discover music through natural conversation. Unlike static recommendation lists, the AI Crate Digger understands context, responds to feedback, and explains its reasoning through graph relationships.

"I'm into dark ambient but I want something with more rhythm" → "Based on your interest in dark ambient, let me trace a path through the graph: dark ambient → industrial ambient → rhythmic noise → power electronics. Here are artists along that path: Lustmord → Coil → SPK → Throbbing Gristle. But if you want to stay closer to ambient with just a pulse underneath, check out Biosphere's Substrata or Gas's Pop — they bridge ambient and minimal techno."

The key differentiator from generic AI recommendations: every suggestion is backed by graph relationships the user can explore, and the conversation maintains state across turns.

Conversation Modes

1. Open Discovery

• "I want to find something new"
• Uses taste fingerprint + graph exploration to suggest unexplored areas
• "Based on your collection, you've never explored krautrock despite loving electronic music. Krautrock is the missing link — here's why..."

2. Directed Search

• "Something like Boards of Canada but more acoustic"
• Starts from a known entity, traverses graph relationships with LLM-guided filtering
• Explains the connection path between the starting point and each suggestion

3. Mood/Vibe Matching

• "Something for a rainy Sunday morning"
• Maps vibe descriptions to genre/style combinations using the genre tree
• "Rainy Sunday morning sounds like: ambient, chamber music, post-classical, or slowcore. Based on your taste fingerprint, I'd lean toward post-classical..."

4. Gap-Informed Recommendations

• "What should I buy next?"
• Combines gap analysis, taste fingerprint, and graph proximity
• "You're 2 albums away from completing the Aphex Twin discography, and you're missing a key early release..."

5. Challenge Mode

• "Surprise me with something outside my comfort zone"
• Identifies blindspots from taste fingerprint
• Traces a path from familiar territory to the unfamiliar suggestion
• "You've never touched reggae, but here's a path from your dub techno collection: Basic Channel → Rhythm & Sound → Wackies → King Tubby"

Architecture

Conversation Engine (api/crate_digger/engine.py)

@dataclass
class ConversationState:
    session_id: str
    user_id: str | None
    messages: list[Message]
    context: DiggerContext

@dataclass
class DiggerContext:
    taste_fingerprint: dict | None  # loaded on first turn if authenticated
    discussed_entities: list[int]   # node IDs mentioned in conversation
    rejected_suggestions: list[int] # user said "no" to these
    preferences: dict               # extracted from conversation ("more rhythm", "less vocals")
    current_focus: str | None       # genre/style/artist being explored

Tool-Use Pattern

The LLM operates in a tool-use loop, similar to the MCP server but optimized for recommendation dialogue:

Tool	Purpose
search_graph	Find entities matching a description
get_similar_artists	Find similar artists to a reference
find_path	Trace connections between entities
get_taste_fingerprint	Load user's taste profile
get_blindspots	Find genres/styles the user hasn't explored
get_label_dna	Understand a label's musical identity
check_collection	Check if user owns a release
get_gap_analysis	Find missing releases for an artist/label

These tools call existing API endpoints internally — no new data access patterns needed.

Recommendation Ranking

For each suggestion, compute a composite score:

• Graph proximity: How close is this to things the user already likes?
• Novelty: How different is this from the user's existing collection?
• Path explainability: Can we trace a clear, interesting path from known → suggested?
• Conversation relevance: Does this match what the user asked for in this conversation?

Proposed Endpoints

API Endpoints (api/routers/crate_digger.py)

Endpoint	Description
POST /api/crate-digger/chat	Send a message in the crate digger conversation
GET /api/crate-digger/sessions	List user's recent sessions (authenticated)
GET /api/crate-digger/sessions/{id}	Retrieve a past conversation
DELETE /api/crate-digger/sessions/{id}	Delete a conversation
POST /api/crate-digger/sessions/{id}/feedback	Rate a session (helpful/not helpful)

Request Shape

{
  "session_id": "abc123",
  "message": "I'm into dark ambient but I want something with more rhythm"
}

Response Shape

{
  "session_id": "abc123",
  "response": {
    "text": "Great starting point! Let me trace some paths through the graph from dark ambient toward more rhythmic territory...",
    "suggestions": [
      {
        "entity_id": 789,
        "entity_type": "artist",
        "name": "Biosphere",
        "why": "Bridges ambient and minimal techno — Substrata has that dark ambient feel but Patashnik adds rhythm",
        "path_from_input": ["dark ambient", "ambient techno", "minimal techno"],
        "in_collection": false,
        "top_release": {"id": 456, "title": "Substrata", "year": 1997}
      }
    ],
    "follow_up_prompts": [
      "Tell me more about Biosphere",
      "Something darker than that",
      "What about something with actual beats?"
    ]
  },
  "context": {
    "current_focus": "rhythmic ambient",
    "discussed_entities": [789, 790, 791]
  }
}

Explore UI — "Dig" Pane

Add a Dig pane to the Explore sidebar — a chat interface for music discovery.

Layout

1. Chat Window — scrollable message thread with user messages and AI responses
2. Suggestion Cards — inline cards for recommended artists/releases, each showing:
   • Artist/release name and year
   • "Why this?" expandable section showing graph path
   • "Explore" button to open in graph view
   • "In Collection" / "Add to Wantlist" badge (if authenticated)
   • Thumbs up/down feedback buttons
3. Quick Prompts — suggested conversation starters based on user's collection:
   • "What should I listen to next?"
   • "Surprise me with something new"
   • "Deep dive into [top genre]"
4. Session Sidebar — collapsible list of past conversations

UI Details

• Chat bubbles styled distinctly for user vs AI
• Suggestion cards are interactive — hovering highlights the entity in the graph if visible
• Graph path visualization: clicking "Why this?" shows a mini path diagram
• Streaming response display (SSE) for real-time feel
• Conversation persists across page refreshes within session

Integration Points

• MCP Server: Share tool definitions and graph access patterns. MCP = external AI access, Crate Digger = internal UI access to similar capabilities
• Natural Language Queries (feat: Natural Language Graph Queries — conversational access to the knowledge graph #203): NLQ handles factual queries; Crate Digger handles recommendations. Route based on intent classification: factual → NLQ, recommendation → Crate Digger
• Collection Storyteller (feat: Collection Storyteller — AI-generated narratives from collection patterns #202): Crate Digger can reference collection narratives for context
• Taste Fingerprint: Primary signal for personalization
• Sample Chain (feat: Sample Chain Tracker — sampling relationships and lineage in the knowledge graph #201): "Who sampled this artist?" as a discovery vector
• Notifier Service (feat: Notifier Service — watchlists and alerts for new music data #86): "Your Crate Digger found a new release from an artist you discussed"

Implementation Notes

• LLM provider abstraction shared with Storyteller (feat: Collection Storyteller — AI-generated narratives from collection patterns #202) and NLQ (feat: Natural Language Graph Queries — conversational access to the knowledge graph #203)
• Conversation state stored in Redis (TTL: 7 days) with PostgreSQL backup for saved sessions
• Tool-use loop uses existing API endpoints — no direct DB access from the conversation engine
• Streaming responses via SSE for real-time chat feel
• Cost control: cap conversation length (20 turns), token budget per turn, rate limit (5 conversations/hour)
• Feedback data (thumbs up/down) stored for recommendation quality improvement
• Session feedback used to tune system prompts over time
• Prompt strategy: system prompt includes tool schemas + recommendation philosophy; user context injected per turn
• Suggestion deduplication: track discussed entities across conversation, don't repeat

Acceptance Criteria

• Conversational interface supports multi-turn dialogue with state
• All 5 conversation modes produce relevant recommendations
• Each suggestion includes graph path explanation
• Tool-use loop calls existing API endpoints correctly
• Taste fingerprint and collection data inform personalization
• Thumbs up/down feedback captured per suggestion
• Streaming responses via SSE
• Session persistence and history
• Cost controls: turn limit, token budget, rate limiting
• Graceful degradation when LLM unavailable
• Integration with NLQ (feat: Natural Language Graph Queries — conversational access to the knowledge graph #203) for intent routing
• ≥80% test coverage

[SimplicityGuy]

Status Update — 2026-03-25

Not started. No implementation or design work has been merged to main for this feature.