Feature/followup latency#96
Merged
Merged
Conversation
P1(AI): followup RAG에서 실패·저속한 LLM 리랭커를 기본 비활성화(followup_rerank_enabled=false)하고 candidate 확장 없이 top_k 직접 검색. _build_rag_context 전체에 타임아웃(followup_rag_timeout_sec=1.5s) 폴백 추가 → 첫 토큰까지 ~3s에서 ~1.5s 목표. P2(FE): SESSION_MESSAGE_DELTA가 짧은 질문에서 0.3s에 몰려 와도 보이도록 useTypewriter로 누적 텍스트를 점진 표시(StageQuestion/QuestionBubble). 긴 스트림은 남은 양 비례 step으로 따라잡아 지연 없음.
리랭커가 프로덕션에서 항상 실패→검색순서 폴백 중이라 가치 0에 ~1-2s만 소모. 모든 RAG 경로(질문풀·꼬리질문·피드백)가 이미 검색순서를 쓰고 있어 제거는 동작 무변경이며 지연·비용만 감소. - rag/reranker.py, chain/prompts/rerank.py, test_reranker.py 삭제 - 3개 consumer: candidate_k 후보 fetch + rerank_hits 제거 → rag_top_k 직접 검색 - settings: rerank_enabled/rerank_candidate_k/followup_rerank_enabled 제거 (followup_rag_timeout_sec 타임아웃 안전망은 유지)
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
변경 사항