how much does better models help writing the final output?

[sfarrell5123]

I'd like pluggable LLM use

I have it running on my rtx 3090, I guess all the VRAM use is in embedding ? it seems to not use much GPU compute. I put ollama on my 4060ti - so the 3090 was dedicated to NaneSage

I am keen to use openroute, and gemini flash 2 - it's nearly free and very good at writing documents.

I am keen on getting it to even more deep dive. more levels of research / summarizing etc. But I quickly run out of VRAM. any tips ?

[masterFoad]

I am working on adding a generic llm interface soon : openai api support etc

Is that good?

Also will improve the report output

Working on many things, hopefully new updates soon

More deep dive is better, i want to add more subqueries to the subqueries also

[sfarrell5123]

openai api would be great, just leave the base URL open to change, that way I can use openrouter (which allows every LLM on the planet - using the openai interface - so you dont have to recode if someone say wants to use sonnet).

This is an awesome project, I had already drafted up ideas of writing my own, but this seems better than what I might do.

I love your focus on research + embedding, and local GPU usage. very smart.

[sfarrell5123]

I tried to fork and add changes, but the changes ended in your code.
I don't care if you keep them or remove them.