Discussions

• 1 You must be logged in to vote

  🙏

  how to pass “provider=CUDAExecutionProvider” to onnx engine？

  hpx502766238 asked Aug 23, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  📣

  About Concurrency and Stability

  gdw439 announced Jun 18, 2025 in Announcements

  

  

  1
• 1 You must be logged in to vote

  💬

  How often are docker images rebuild to resolve vulnerability fixes?

  molntamas started Apr 30, 2025 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  Is there support for faster serialization format than json for embedding

  honeyspoon asked Aug 12, 2025 in Q&A · Closed · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  💬

  Can we run infinity with gguf quantized models ?

  Greatz08 started Jul 1, 2025 in General

  

  

  2
• 1 You must be logged in to vote

  💡

  Update torch and cuda

  Wolframko started May 19, 2025 in Ideas

  

  

  1
• 1 You must be logged in to vote

  🙏

  padding='max_length'

  Hummer12007 asked May 12, 2025 in Q&A · Unanswered

  

  

  4
• 2 You must be logged in to vote

  💡

  Add support for newer models, ie google/siglip2

  JosefAschauer started May 8, 2025 in Ideas

  

  

  0
• 1 You must be logged in to vote

  🙏

  How do I select lora adapters in embedding models

  zacksiri asked Apr 22, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  mxbai-rerank-large-v2

  qdrddr asked Mar 14, 2025 in Q&A · Unanswered

  

  

  

  

  7
• 1 You must be logged in to vote

  📣

  Error when using /rerank endpoint with mixedbread-ai/mxbai-rerank-large-v2

  jaehyeongAN announced Mar 20, 2025 in Announcements · Closed

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  Does the infinity have support for an on-demand model loading like ollama, specifically 'keep_alive'?

  ouz-m asked Mar 5, 2025 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  📣

  Additional option for alternative model Paths to load different model formats

  John42506176Linux announced Aug 13, 2024 in Announcements

  

  

  

  

  5
• 1 You must be logged in to vote

  💬

  How to add a progress bar during embedding

  rangehow started Feb 19, 2025 in General

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  Integration with facebook/nllb-200-distilled-600M translation model

  viper-108 asked Feb 17, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Loading model NOT from Huggingface?

  molntamas asked Feb 10, 2025 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙌

  Analysis: CLI startup time

  wirthual started Jan 10, 2025 in Show and tell

  

  

  2
• 2 You must be logged in to vote

  💡

  Support for Colpali

  amit-jain started Sep 29, 2024 in Ideas

  

  

  

  

  8
• 1 You must be logged in to vote

  📣

  Separating infinity server capability from reranking capability

  noamgershi announced Dec 31, 2024 in Announcements

  

  

  3
• 1 You must be logged in to vote

  🙏

  Running models on seperate devices

  Sens612 asked Dec 2, 2024 in Q&A · Answered

  

  

  4
• 2 You must be logged in to vote

  🙏

  Code 502 when using OpenAI client

  liux2 asked Nov 22, 2024 in Q&A · Closed · Answered

  

  

  5
• 1 You must be logged in to vote

  📣

  Use different revision for more one model.

  rohilrg-diabolocom announced Nov 22, 2024 in Announcements

  

  

  

  

  6
• 1 You must be logged in to vote

  🙏

  Custom fine tuned embedding models are supported

  kjoth asked Feb 27, 2024 in Q&A · Answered

  

  

  

  

  5
• 1 You must be logged in to vote

  🙏

  Can't embed more than 2048 items

  ahmedkooli asked Oct 30, 2024 in Q&A · Closed · Answered

  

  

  1
• 2 You must be logged in to vote

  📣

  [INFO] How to scale ?

  tsensei announced Oct 21, 2024 in Announcements

  

  

  

  

  8