diff --git a/CHANGELOG.md b/CHANGELOG.md index 3df9f2e702ba..2b87129da4bd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,253 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.11.4] - 2026-09-21 + +### Added + +- 📉 **Far smaller slim image.** A slim build now comes down at around 175 MB, near enough 89% smaller than the last release, the local models, the packages around them and the tools that installed them all gone from it; what that changes about the way an instance behaves is set out under Changed below and in the documentation. [Commit](https://github.com/open-webui/open-webui/commit/cb942bb94c8dc7941336088fb3392e2398ff56c1), [Commit](https://github.com/open-webui/open-webui/commit/d27aa72ab4a7b5632b4ad49e8467081ad3d7ebb4) +- 📦 **Smaller standard image.** The image no longer carries a second copy of Python, two sets of fonts nothing ever loaded, packages nothing imports, or the tool that installed them, taking about 170 MB off a standard build. [#29731](https://github.com/open-webui/open-webui/pull/29731), [#29723](https://github.com/open-webui/open-webui/pull/29723), [#29725](https://github.com/open-webui/open-webui/pull/29725), [#29726](https://github.com/open-webui/open-webui/pull/29726), [#29728](https://github.com/open-webui/open-webui/pull/29728), [Commit](https://github.com/open-webui/open-webui/commit/91f8775b28b52c9ae7f2ab990cf2490bda8055d6), [Commit](https://github.com/open-webui/open-webui/commit/98fcb844e1b19f7dd6289af26273cdec5447dc52), [Commit](https://github.com/open-webui/open-webui/commit/508de20779168003e538bb936b49a31d4a8fb8bb), [Commit](https://github.com/open-webui/open-webui/commit/a1c02098aa2687c72482a59117efe643b785df51) +- 🧑‍💻 **Skills from a terminal.** Skills a connected terminal server offers now sit beside workspace skills everywhere skills are picked — the "$" and "/" menus, the integrations menu and the skills panel, each marked Terminal — and are used the same way: picking one puts its instructions, its folder and the files it ships with in front of the model, and a model that was only told a skill exists can open it itself. They follow whichever terminal is selected and clear when that changes. [Commit](https://github.com/open-webui/open-webui/commit/e69236bccb1e12d14045b09b76ebac0490014851) +- 📔 **Terminal instructions file.** A model working with a terminal is now handed the AGENTS.md sitting in that terminal's home directory, read afresh at the start of every turn, so the instructions you keep beside your work reach the model without being pasted in. [Commit](https://github.com/open-webui/open-webui/commit/946be432375057dc18dbcc7c3513feb322a83a4b), [Commit](https://github.com/open-webui/open-webui/commit/f6922a4c4293449805938c80fbe54174994a1fc6) +- 📇 **Automatic skill discovery.** Every skill you can reach is now listed to the model by name and description, and the full text of one is loaded only when it decides to use it; before, a skill you had not selected in the message box was invisible to it, and this applies to models with built-in tools on. [Commit](https://github.com/open-webui/open-webui/commit/e69236bccb1e12d14045b09b76ebac0490014851) +- 🌄 **Model background images.** A workspace model can now carry a background image, uploaded in its editor and drawn behind the chat whenever that model is selected, sitting below a folder's own background and above your personal one, and it travels with the model through export and import. [Commit](https://github.com/open-webui/open-webui/commit/66addbd6b47bfb25cf9aa37ec70d24db5b28b6b6) +- 📓 **Skill creation from a chat.** Typing "/skills:create" in a chat that already has content, with a terminal selected, turns the workflow you just went through into a reusable skill written to ".agents/skills" under that terminal's root working directory rather than whatever folder the shell happens to sit in, and authored to Open WebUI's skill standards. [Commit](https://github.com/open-webui/open-webui/commit/113c56fc8c1986359556107a20e206159ca32f59), [Commit](https://github.com/open-webui/open-webui/commit/924a4a10fbd0be508a69faf66bf08ac9761bc24d), [Commit](https://github.com/open-webui/open-webui/commit/58078ab3045cc7aee409d9215f68adf0e02c9165), [Commit](https://github.com/open-webui/open-webui/commit/d25f6c7135e0aee93dc2c840ce320d97617a6e47), [Commit](https://github.com/open-webui/open-webui/commit/a096961a31499be23890b98a040ecf2917405568) +- 🖥️ **Terminal tabs per command.** The terminal pane now carries a tab for each command a model is running alongside your own shell, so you can watch them as they go, move between them and take the shell yourself, each tab opening with the command that produced it. Opening the pane puts you in a tab, starting your shell where nothing else is running, and closing the last tab folds the pane away again. [Commit](https://github.com/open-webui/open-webui/commit/54a7a7a7ce22725074c29c7e827446f5dce1421c), [Commit](https://github.com/open-webui/open-webui/commit/f3eade42aead0a3b96ada3a300d6c294c89f74a8), [Commit](https://github.com/open-webui/open-webui/commit/de1203f9b5d3d8b2a84bf5c89d1c61542469367a), [Commit](https://github.com/open-webui/open-webui/commit/675b9839f17df94f866c871cbdb9881323a18a8a) +- 📂 **Folder uploads to terminals.** The file browser beside a terminal takes a folder now, dropped onto it or picked from the Upload Folder entry in its menus, and rebuilds what is inside it as it goes, subfolders and all, where before a drop uploaded only the files sitting loose at the top. [Commit](https://github.com/open-webui/open-webui/commit/f80ef8bd001d7ef650fb278d6fbd05bea4afad0f), [Commit](https://github.com/open-webui/open-webui/commit/4cc0d48b4d83503199bcc5f2322881722971a499), [Commit](https://github.com/open-webui/open-webui/commit/674760bfc1122e0a19fe299e05a86d1cbb528e6f) +- ⚖️ **Side-by-side file comparison.** Picking two files in the file browser lights up a Compare button that lays them side by side or one above the other, numbering the lines, marking what was added and removed down to the part of the line that changed, counting both, and letting you swap which is which or leave whitespace out of it, a choice it remembers; a terminal too old to offer the comparison says so rather than failing quietly. [Commit](https://github.com/open-webui/open-webui/commit/8556033c6b64fa53e44152ebab1f889a578529af), [Commit](https://github.com/open-webui/open-webui/commit/3808eace6c2beb1904f0f04fdd443ec544f94acb) +- 🌐 **Suggested prompts in your language.** A fresh install now offers its suggested starter prompts in the language the interface is set to, rather than the same four in English for everyone, and the model defaults panel carries a way back to those defaults once its own suggestions have been edited. [Commit](https://github.com/open-webui/open-webui/commit/30881dbcc966206cc15fb0b2276b41d88f7c0f09), [Commit](https://github.com/open-webui/open-webui/commit/9fba2843b16a111b035949f5865e29ec6cf5ac9c) +- 📏 **Exa result length cap.** Web search through Exa can be held to a number of characters per result, set beside its key in the admin web search settings or as "EXA_MAX_CONTENT_LENGTH". [Commit](https://github.com/open-webui/open-webui/commit/12b14124b9376eccf2ba7cf3dba92bc145493703) +- 🇪🇺 **European web search option.** Staan can now be picked as the web search provider, giving deployments that need search inside the EU an option they do not have to host themselves, configured from the admin web search settings or through "STAAN_API_KEY", "STAAN_MARKET" and "STAAN_MAX_SNIPPETS". [#30138](https://github.com/open-webui/open-webui/pull/30138), [#26006](https://github.com/open-webui/open-webui/discussions/26006), [#30303](https://github.com/open-webui/open-webui/pull/30303), [#30301](https://github.com/open-webui/open-webui/issues/30301) +- 🗣️ **Per-language names and descriptions.** A model, tool, skill, function, banner or arena entry can now hold its name, description, starter prompts and valve labels once per language, written in a searchable table in its own editor or brought in as a JSON file, which is refused where a translation drops one of the placeholders the original fills in. The interface shows the wording for the language it is set to, and falls back to the plain text where that language has none. [Commit](https://github.com/open-webui/open-webui/commit/7b6562e3358956ec71eed05352538a646d047734), [Commit](https://github.com/open-webui/open-webui/commit/858ab727d2d80bb3c9ef40f01a32de7a12ca75c9), [Commit](https://github.com/open-webui/open-webui/commit/3facfa61d413945d7144d26b827fcce1f3aef7c5), [Commit](https://github.com/open-webui/open-webui/commit/85b11a4f3504434bd9cd0748629b8a84817fb4f2) +- ✏️ **Interface text you can reword.** An administrator can now replace the interface's own wording language by language from a panel in the admin settings, with a replacement refused where it drops one of the placeholders the original fills in. [Commit](https://github.com/open-webui/open-webui/commit/67ac1a4e937271a02bfb02a330aa99dc8192cbf4) +- 🔓 **Turning off the sign-in form.** The box asking for an email address and a password can now be taken off the sign-in page from the authentication settings, where until now it could only be set before the server started, leaving single sign-on or a directory to sign people in. [Commit](https://github.com/open-webui/open-webui/commit/c4a349651e34bf5d0920bb5e26707ae9961ac39f) +- 🏷️ **Custom file metadata.** Metadata attached to an uploaded file now travels with the pieces that file is split into and arrives with the retrieved sources, the oversized and internal fields left out, and an operator can name in "RAG_SOURCE_METADATA_KEYS" which of those fields the model itself gets to see alongside the text. [#29499](https://github.com/open-webui/open-webui/pull/29499), [#29486](https://github.com/open-webui/open-webui/issues/29486), [Commit](https://github.com/open-webui/open-webui/commit/894655f66b9563890e63c76090ddecc90a311eb2), [#29502](https://github.com/open-webui/open-webui/pull/29502), [#29696](https://github.com/open-webui/open-webui/pull/29696) +- 🗑️ **Quick delete shortcut.** Holding Shift over a note in the list or grid, or over a row on the automations page, turns its trailing controls into a delete button, removing the entry in one click rather than the three the menu and its confirmation ask for. [#29635](https://github.com/open-webui/open-webui/pull/29635), [#29633](https://github.com/open-webui/open-webui/issues/29633), [#29640](https://github.com/open-webui/open-webui/pull/29640), [#29637](https://github.com/open-webui/open-webui/issues/29637) +- 🔀 **Diffs are drawn as diffs.** A diff or patch block in a reply is now laid out as one, with the file and hunk headings, the old and new line numbers, and the added and removed lines picked out down to the part of the line that changed, and a button to switch to the plain text and back. [Commit](https://github.com/open-webui/open-webui/commit/254e29b9af634145dde0450451a36f0dfbd8f421) +- ✒️ **A formatting switch for notes.** A note you can edit carries a Formatting switch in its menu: turned off, Markdown you type or paste stays as the characters you wrote, a paste keeps its plain text, and a web address is left as text, while formatting already in the note is untouched. [Commit](https://github.com/open-webui/open-webui/commit/d8f27e745bd31c89efa25cbe0f41bb0f70576fc5) +- 🎛️ **Admin model list filters.** The models page in the admin settings can now be narrowed to the base models a connection offers or to the ones built in the workspace, alongside the filters for enabled, disabled, visible and hidden. [Commit](https://github.com/open-webui/open-webui/commit/9e634c0c56e0a060717f1e24a11b082daf664036) +- 🔍 **Settings search by name.** The search box in settings now matches each setting's own name and description rather than a keyword list kept per page, in whichever language you are using and regardless of accents, it leaves out anything your permissions do not let you change, and opening a result no longer clears what you typed. [Commit](https://github.com/open-webui/open-webui/commit/7cbaabe02fc3c7c040008e82cf85f590bd0c346a), [Commit](https://github.com/open-webui/open-webui/commit/9d98ffcddf792be8d567ea39370d39fcac649acd), [Commit](https://github.com/open-webui/open-webui/commit/c82634b9d01adadbf9780ff84a0a8168fdc4fdea) +- 🆘 **A model for every chat.** Opening a conversation whose model has since been retired no longer leaves it stranded with nothing selected; it falls back to your default model, then the configured default, then the first one available, and a chat that still has a live model keeps it. [#29757](https://github.com/open-webui/open-webui/pull/29757) +- ⚡ **Non-blocking search.** Searching the text of chats and knowledge, and the grep a model runs over a knowledge base, now run beside the rest of the server rather than in front of it, so a long search no longer keeps other requests waiting. [#29621](https://github.com/open-webui/open-webui/pull/29621), [Commit](https://github.com/open-webui/open-webui/commit/d9c8de9c39fca7c4756e0f9bd599b09ebe435208) +- ✴️ **Lighter shared note editing.** Notes written by several people at once no longer echo every keystroke back to the server once per watcher; the traffic between editors falls by half with two of them and keeps falling as more join, so shared notes stay smooth as the room grows. [#28185](https://github.com/open-webui/open-webui/pull/28185) +- 🔢 **Sorted knowledge listings.** The "ls", "tree" and "find" commands a model runs over a knowledge base now sort by name and take "-t", "-S" and "-r" for newest first, largest first and reversed. [#29840](https://github.com/open-webui/open-webui/pull/29840) +- 🪪 **Authentication type header.** A request to an OpenAI or Ollama connection now carries "X-OpenWebUI-Auth-Type", saying whether the person behind it signed in through the browser or called with an API key; its name is set with "FORWARD_USER_INFO_HEADER_AUTH_TYPE" and "{{AUTH_TYPE}}" works in custom headers. [Commit](https://github.com/open-webui/open-webui/commit/ee46e2664ab23bacc536fed21c06ab19067d695b) +- 💡 **Follow-up ghost text.** Once a reply has finished, the first of the follow-up questions it suggests now sits greyed inside the empty message box as well as under the reply: Tab writes it out, and typing anything of your own clears it away. [Commit](https://github.com/open-webui/open-webui/commit/7aaa4a692e949724913834efc47c57572042703b) +- 🪵 **Model refusal logging.** A request turned away with "Model not found" now writes a warning naming the account, the model and why it was refused, while the message the caller sees stays as vague as before. [Commit](https://github.com/open-webui/open-webui/commit/f5fcf4c89fb27ed39825875d573446fa84aa2a5b) +- 🤲 **Cheaper chat requests behind Redis.** Where several servers share their websocket traffic through Redis, a chat request no longer pulls the whole shared model list from Redis to ask about a couple of models; each worker keeps its own copy and refetches only when the list actually changed, so the cost stops growing with the number of models configured. [#28176](https://github.com/open-webui/open-webui/pull/28176), [#28167](https://github.com/open-webui/open-webui/issues/28167) +- 💽 **Long conversations stream cheaper.** Writing or reading a single message no longer loads, checks and rewrites the entire conversation to change a few hundred bytes, so streaming into a long chat stops getting more expensive as it grows, and the whole round trip a message event costs falls to a fraction on conversations of thousands of messages. [#28184](https://github.com/open-webui/open-webui/pull/28184), [#28169](https://github.com/open-webui/open-webui/issues/28169) +- 🧱 **In-place streaming appends.** Streamed replies can be assembled with CPython's in-place string append, turned on with "ENABLE_CHAT_RESPONSE_STREAM_INPLACE_APPEND" and left off by default while it is rolled out gradually. [Commit](https://github.com/open-webui/open-webui/commit/113c56fc8c1986359556107a20e206159ca32f59), [Commit](https://github.com/open-webui/open-webui/commit/924a4a10fbd0be508a69faf66bf08ac9761bc24d), [#30066](https://github.com/open-webui/open-webui/pull/30066) +- 📘 **Direct connection guidance.** The connections and integrations pages in the personal settings now say plainly that a direct connection leans on your browser session to keep requests running, which suits testing and temporary use rather than everyday work. [Commit](https://github.com/open-webui/open-webui/commit/5d66dd6ad291db17894953714c01fd4303093e7e), [Commit](https://github.com/open-webui/open-webui/commit/15e2259a2ae1b7dec1f93a52078304dd3a79a14c) +- 🔄 **General improvements.** Various improvements were implemented across the application to enhance performance, stability, and security. +- 🌐 **Translation updates.** Translations for Japanese, Traditional Chinese, Korean, Finnish, Russian, Ukrainian, German, Spanish, Portuguese (Brazil), Arabic, Arabic (Bahrain), Azerbaijani, Bulgarian, Bengali, Tibetan, Bosnian, Catalan, Cebuano, Czech, Danish, Basque, Croatian, Dutch, Italian, Turkish, Simplified Chinese, Bosnian, Catalan, Danish, Estonian, Finnish, Galician, Croatian, Kabyle, Norwegian Bokmål, Portuguese (Portugal), Romanian and Turkmen were enhanced and expanded. + +### Fixed + +- 🛡️ **Security Advisory**: This release includes security and access-control fixes. We recommend updating production deployments at your earliest convenience. Not all security fixes in this version may be enumerated in the fixed section. Some may be withheld for a short time to give administrators time to upgrade. [Advisories](https://github.com/open-webui/open-webui/security) +- 🔑 **Tokens stay out of logs.** A failure part way through signing in with an identity provider no longer writes the credentials it was handed into the application log, recording the provider and the error it reported instead. [#29709](https://github.com/open-webui/open-webui/pull/29709) +- 🔐 **Sign-in role mapping.** Roles sent by an identity provider were in some setups not applied, leaving an account at the default role, and a sign-in whose roles cannot be read is now refused rather than let through. [Commit](https://github.com/open-webui/open-webui/commit/10d1cfe6375f207acaa531e857edb575ded2cfc3) +- 🚫 **Blocked sign-in groups saved.** Comma-separated group names typed into the blocked groups field now take effect once saved, where the save stored the text as written and the sign-in check then read it as nothing; a group name carrying a comma of its own survives a save too. [Commit](https://github.com/open-webui/open-webui/commit/3fc1146c13d7b4b7a67c7fd058014436f80e0dfd) +- 🚪 **Signing out ends the session.** Signing out or having every token revoked left any live connection that account already had in place, and those are now cut at the same moment; a token already revoked can no longer be replayed to cut someone else's newer session. [Commit](https://github.com/open-webui/open-webui/commit/3a6d0fd203050401481bdb8621b827089d393817) +- 🛡 **Diagrams and SVG stay on this origin.** Mermaid diagrams and SVG previews, attached or uploaded, no longer follow references pointing at another origin, a picture, style or configuration line among them, and a diagram that reaches outside is refused with an error; a file whose drawing leans on an external sprite now shows that part blank. [#30271](https://github.com/open-webui/open-webui/pull/30271) +- 📲 **Terminal sessions follow access changes.** A terminal connection now re-checks your access to it every ten seconds for as long as it stays open, so removing someone's permission or deactivating their account ends their live terminal on the next check, on every worker and not only the one they signed in through. [Commit](https://github.com/open-webui/open-webui/commit/a1189a2d757407530a0c1c85a41b46cd6a4f7da5), [Commit](https://github.com/open-webui/open-webui/commit/e8bd0661d3774248c318286849041b94bb861909) +- 🔒 **Connection model listing access.** The endpoints that list the models on a single Ollama or OpenAI connection could be reached by a signed-in account of any role, and now check the caller's role. [#29619](https://github.com/open-webui/open-webui/pull/29619) +- 🔏 **Knowledge file access.** A file's access through a knowledge base now follows its actual attachment alone, rather than also a collection name left behind on the file record. [#29937](https://github.com/open-webui/open-webui/pull/29937) +- 🌳 **Knowledge directories stay inside their base.** A directory id supplied to the knowledge file routes is now refused where it belongs to a different knowledge base, where a caller-supplied id could reach into a directory tree outside the one being addressed. [Commit](https://github.com/open-webui/open-webui/commit/a9541c18ca2a056f4ccfe56b9c733cba74b86b49), [#29887](https://github.com/open-webui/open-webui/pull/29887) +- 🍪 **Cookie forwarding scope.** A connection authenticating as the signed-in person is handed this instance's browser cookies only where its new Forward cookies switch is on, which an instance relying on it has to turn on after upgrading. [Commit](https://github.com/open-webui/open-webui/commit/b71744b17823f7a3a9a64cb363e79a7164ed5b23), [Commit](https://github.com/open-webui/open-webui/commit/3cda47cdb449aac970426f0929c9e5f8bb8bd807), [Commit](https://github.com/open-webui/open-webui/commit/f9f815c86220f809fad5ee1300c44f55dea74c79) +- 🗝️ **Revocation list fallback.** Where the revocation list is kept in Redis and Redis cannot be reached, a token is now accepted rather than the request failing, so signing out may not take effect until Redis is back. [Commit](https://github.com/open-webui/open-webui/commit/c1615bec2f5f143084b5fcc04f42ebfa0dade9df), [Commit](https://github.com/open-webui/open-webui/commit/6a85abb3f5e002f3069007213e16e4ef60776890) +- 🗒 **Home page chat drafts.** A message typed into a chat started from the home page now comes back after a reload, text, uploads and all, where the draft was stored under a key the page never read again. [#29762](https://github.com/open-webui/open-webui/pull/29762), [#29760](https://github.com/open-webui/open-webui/issues/29760) +- 🏞️ **S3-hosted chat images.** A chat image whose host echoes a Content-Encoding nothing asked for, an S3 or MinIO object uploaded with that metadata among them, no longer breaks the reply it sits in; it is sent as its link the way an unreachable image already was. [#29623](https://github.com/open-webui/open-webui/pull/29623) +- 🪆 **Branch descent guard.** Stepping between reply branches walked a chat's children without tracking where it had already been, so a history that pointed back at itself spun forever and locked the tab, and a child id with no message behind it threw; every one of the eleven places that walk now shares a single guarded helper. [#30070](https://github.com/open-webui/open-webui/pull/30070) +- 🗒️ **Note save on exit.** A note's title and its attachments save a moment after they change, and leaving the note inside that moment dropped the save, so the notes list kept showing the old title until the page was reloaded; the pending save now finishes before the next page opens. [#29745](https://github.com/open-webui/open-webui/pull/29745), [#29744](https://github.com/open-webui/open-webui/issues/29744) +- 📷 **Attachments sent without text.** Sending an image or a file with nothing typed to a model that has skills attached replaced the empty message with the list of skill names, so the model answered with its own catalogue instead of looking at what you sent. [#30045](https://github.com/open-webui/open-webui/pull/30045), [#30040](https://github.com/open-webui/open-webui/issues/30040) +- 🖍️ **SVG attachments read as text.** An SVG attached to a chat went up as a picture and came back rejected by every model that tried to decode it, and is now read as its source text instead, which needs a model that accepts file uploads. [#30102](https://github.com/open-webui/open-webui/pull/30102), [#30100](https://github.com/open-webui/open-webui/issues/30100) +- 🩹 **Notes survive a small edit.** Asked to add a section or change a few lines, a model would send the whole note back and lose the rest of it with nothing to undo, because the editing tool never said when to edit a range instead; it now spells out the range rules the handler already enforced. [#30048](https://github.com/open-webui/open-webui/pull/30048) +- 📋 **Note paste placement.** A plain paste replaced more of the note than was selected, and a paste into a code block broke out of the block instead of going inside it; both now land exactly where they were dropped. [Commit](https://github.com/open-webui/open-webui/commit/d8f27e745bd31c89efa25cbe0f41bb0f70576fc5) +- 🧿 **Insert into note on read-only notes.** The Insert into note action no longer appears on a note you may only read, which offered it where it could not land. [#30223](https://github.com/open-webui/open-webui/pull/30223), [#30174](https://github.com/open-webui/open-webui/issues/30174) +- 🤝 **Note sharing survives a save.** Saving a note you had been given access to took that sharing away, so the note went private and everyone else lost it. [#30175](https://github.com/open-webui/open-webui/pull/30175) +- 🌍 **Collaborative link handling.** In a note two people are writing at once, a plain web address arriving from the other editor was turned into a link on your screen but not on theirs; it is now left as written. [Commit](https://github.com/open-webui/open-webui/commit/d8f27e745bd31c89efa25cbe0f41bb0f70576fc5) +- 📑 **Tika 4 document formatting.** Where document extraction runs against a Tika server on version 4, the text now comes back as Markdown rather than a flat run of characters, so headings, lists and tables survive into what the model reads. [Commit](https://github.com/open-webui/open-webui/commit/ba34bee2d17e1c00238c85a8aab1a11a60db164e) +- ⏳ **Session expiry accuracy.** A session with an identity provider now expires with the token it actually calls with, rather than whichever of its two tokens ran out first, which renewed it early and dropped it whenever that failed. [Commit](https://github.com/open-webui/open-webui/commit/aaaf26fb8ede28854dd660b11b85ba6bafe64007) +- 🧩 **Tool steps in finished replies.** A reply that called tools showed each step as it ran, then lost them the moment the reply completed, because the provider's closing message replaced everything on screen rather than joining it; the closing message is now merged into what is already there, and a tool call is no longer mistaken for its own result. [Commit](https://github.com/open-webui/open-webui/commit/e1bfefdf9f8f2012cecfc7d81079bd6fff147932), [Commit](https://github.com/open-webui/open-webui/commit/31b272d3c93b87636c920f2aa68d6caaa07ae79a) +- 🧭 **Streamed replies follow along.** A reply arriving over the response events now keeps the view at the bottom as it is written, as replies on the older path already did, unless you have scrolled up yourself. [Commit](https://github.com/open-webui/open-webui/commit/dfde08aa7391d924359b27f5768411d7a533a6ec), [Commit](https://github.com/open-webui/open-webui/commit/88e78b7819b28ffe91f3dff24d8c5d992004197e) +- 🚿 **Tool results arrive when the tools are done.** The results of a model's tool calls now reach the reply as soon as the round that produced them finishes, instead of being held back with the rest of the streaming until the throttle let them through, and a channel reply or a continuation no longer carries the picture data a tool handed the model. [Commit](https://github.com/open-webui/open-webui/commit/478d1785fd27f400c614a5700f5b38ec9de412f9) +- 🧗 **Tagged blocks while streaming.** A reply using reasoning, solution or code interpreter tags briefly showed the raw tag text in the message as it arrived, because the chunk carrying it reached the browser before the cleaned output did; the cleaned output is now sent the moment a tag is taken out, and the code an interpreter block writes fills that block rather than the message body. [Commit](https://github.com/open-webui/open-webui/commit/58b36765a7c20f5943a3180bd289de48876d0878) +- 📗 **Excel in the code interpreter.** Reading or writing a spreadsheet in code the browser runs failed outright because the library that handles them never reached the browser, and it is now shipped alongside the rest. [#30140](https://github.com/open-webui/open-webui/pull/30140), [#30130](https://github.com/open-webui/open-webui/issues/30130) +- 📀 **Bundled charts and formatting.** Drawing a chart with seaborn in code the browser runs fetched the library over the internet at that moment rather than taking it from what ships, and the code editor's Format button failed outright for anyone who is not an administrator; both now work from what comes with the application, offline included. [#30148](https://github.com/open-webui/open-webui/pull/30148), [#30145](https://github.com/open-webui/open-webui/issues/30145) +- 🙋 **Typed answer submission.** In the card a model puts up to ask you a question, choosing one of its options on the last question sends your answers straight away, but typing your own into the Other box left Submit answers greyed out with no way to send it, and it now turns on as soon as that box has text. [#29494](https://github.com/open-webui/open-webui/pull/29494), [#29311](https://github.com/open-webui/open-webui/issues/29311) +- 🥇 **A truthful Recommended badge.** The card a model puts up to ask you a question marks its first option Recommended, but nothing told the model that, so the badge fell on whichever option happened to be listed first; models are now asked to put the option they recommend there. [#30196](https://github.com/open-webui/open-webui/pull/30196), [#30195](https://github.com/open-webui/open-webui/issues/30195) +- 🎚️ **Partial settings permissions.** An account barred from changing the interface settings can now save its system prompt, notifications, audio, keyboard shortcuts and pinned models, which were refused along with them. [Commit](https://github.com/open-webui/open-webui/commit/98a920168e2eea435ac15e1ad3d679946631e41d) +- 🎧 **Speech file types kept.** Saving the audio settings emptied the list of file types accepted for speech recognition, so transcription then turned away the recordings it had taken before. [#30208](https://github.com/open-webui/open-webui/pull/30208) +- 🚻 **Group picker without permission.** The picker for sharing a chat, a note or a knowledge base with a group was offered to accounts not allowed to share with groups, and is now hidden from them. [#30189](https://github.com/open-webui/open-webui/pull/30189) +- 🗜️ **Per-field settings saves.** Only the settings actually changed are now stored, rather than the whole object, so another tab's older copy no longer overwrites them and a default an administrator changes still reaches everyone, and pinning a model, reordering the list or picking a default from outside the settings window now saves the same way. [Commit](https://github.com/open-webui/open-webui/commit/98a920168e2eea435ac15e1ad3d679946631e41d), [#30183](https://github.com/open-webui/open-webui/pull/30183) +- ⚠️ **Failed settings feedback.** Settings that could not be saved were shown as saved anyway until the page was reloaded, because the interface stored them locally without waiting on the server; a failure now raises an error and leaves the panel as it was. [Commit](https://github.com/open-webui/open-webui/commit/98a920168e2eea435ac15e1ad3d679946631e41d) +- 🔠 **Menu text scaling.** The entries in the menus that drop down across the interface now scale with the rest of it, rather than staying at a fixed size while the menu around them grew. [#29493](https://github.com/open-webui/open-webui/pull/29493), [#29488](https://github.com/open-webui/open-webui/issues/29488) +- 🪞 **Account menu highlighting.** An account menu entry carrying a pin button beside it now lights up across the whole row in the shared colour, and a long label no longer pushes the pin out of the menu. [Commit](https://github.com/open-webui/open-webui/commit/e723dcda58f638a5da2398743d22d3e5854042bf) +- 🔲 **Shift-click file selection.** Shift-clicking a file in the terminal's file browser now adds that range to what is already selected, and clicking through a selected file removes its range. [Commit](https://github.com/open-webui/open-webui/commit/f80ef8bd001d7ef650fb278d6fbd05bea4afad0f), [Commit](https://github.com/open-webui/open-webui/commit/4cc0d48b4d83503199bcc5f2322881722971a499), [Commit](https://github.com/open-webui/open-webui/commit/674760bfc1122e0a19fe299e05a86d1cbb528e6f) +- 🛟 **Attachment name collisions.** Attaching a file to a message on an instance that puts attachments in a terminal's working directory replaced whatever file of that name was sitting there; the upload now takes the next free name, "report (1).pdf" beside "report.pdf", and the attachment shows the name it was saved under, though two uploads arriving at the same moment from different browsers can still land on the same name. [Commit](https://github.com/open-webui/open-webui/commit/d70053e44993f271d534fb87d2b40724b028fca2) +- 📛 **Failed upload feedback.** A file that could not be written to the terminal now says so, rather than passing in silence while the browser refreshed as though it had arrived. [Commit](https://github.com/open-webui/open-webui/commit/f80ef8bd001d7ef650fb278d6fbd05bea4afad0f), [Commit](https://github.com/open-webui/open-webui/commit/4cc0d48b4d83503199bcc5f2322881722971a499), [Commit](https://github.com/open-webui/open-webui/commit/674760bfc1122e0a19fe299e05a86d1cbb528e6f) +- ☑️ **Unchecked checkbox defaults.** A prompt variable written as a checkbox with a default of false opened the form already ticked, as did False, "false", 0 and "0", because any non-empty default counted as ticked; it is now ticked only where the value really is true. [#30037](https://github.com/open-webui/open-webui/pull/30037), [#30036](https://github.com/open-webui/open-webui/issues/30036) +- 🎯 **Prefilled question timing.** Opening a chat from a link holding a question sent it before the box had it, so a question naming a variable went off with the variable unfilled; the send now waits for the text to be in place and filled in. [Commit](https://github.com/open-webui/open-webui/commit/3808eace6c2beb1904f0f04fdd443ec544f94acb), [Commit](https://github.com/open-webui/open-webui/commit/a23b579233276e40159eb615917b9aeb7d5ed5c9) +- 📜 **Task list height.** The list of steps a model works through ran as long as it needed and pushed the rest of the reply down the page; it now stops at a quarter of the window's height and scrolls within itself. [Commit](https://github.com/open-webui/open-webui/commit/307b9b9133f0c7ad899f4b76226059da6f3177eb) +- ⎋ **Escape key targeting.** The shortcut for closing a dialog always shut the settings window, whichever dialog was actually in front, so a dialog opened from within settings took both away at once; each dialog now answers the shortcut for itself, as it already did for the escape key. [#29830](https://github.com/open-webui/open-webui/pull/29830), [#29817](https://github.com/open-webui/open-webui/issues/29817) +- 🎹 **Message pair shortcut.** The shortcut that adds an empty question and answer to a chat also sent whatever was typed in the message box, so the pair arrived alongside a message you had not meant to send yet. [#30167](https://github.com/open-webui/open-webui/pull/30167) +- 🥁 **Collapsing the task list.** Folding away the list of tasks under a reply also sent whatever you had typed in the message box. [#30202](https://github.com/open-webui/open-webui/pull/30202) +- 🕰️ **Temporary chat links.** A link carrying the temporary chat marker opened an ordinary chat, because the marker was written into the address but never read back when the page loaded, and such a link now opens the temporary chat it promises. [Commit](https://github.com/open-webui/open-webui/commit/67adde31936d1d2e656140f9edd898b1fbb18a1c) +- ⌨️ **Recording a new shortcut.** Pressing a key combination to record it as a shortcut also ran whatever that combination was already bound to, so setting one up did the thing you were trying to rebind. [#30160](https://github.com/open-webui/open-webui/pull/30160) +- 🔃 **Stale tab reload.** A tab still running the previous build met an error page after the server was updated instead of loading the new one, because every image built from Docker carried the same version stamp; the stamp now follows the build, and a stale tab reloads as it was meant to. [#29832](https://github.com/open-webui/open-webui/pull/29832), [#29831](https://github.com/open-webui/open-webui/issues/29831) +- 📌 **Channel code headers.** The bar naming a piece of code in a channel thread or its pinned messages now sits flush at the top of the panel, clipped to the block, rather than floating over the code as it scrolls. [#29836](https://github.com/open-webui/open-webui/pull/29836), [#29835](https://github.com/open-webui/open-webui/issues/29835) +- 📨 **Duplicated proxy headers.** A reply proxied from a terminal server or an OpenAI or Ollama connection no longer carries that server's own "Server" and "Date" headers, which had a reverse proxy in front logging a duplicate line for every one. [#29841](https://github.com/open-webui/open-webui/pull/29841), [#29824](https://github.com/open-webui/open-webui/issues/29824), [#29843](https://github.com/open-webui/open-webui/pull/29843) +- 🎣 **Testing an image connection.** The Verify button beside an image generation connection saved the whole image configuration first and then tested whichever engine was active rather than the connection beside it, and it now tests exactly that connection and changes nothing. [Commit](https://github.com/open-webui/open-webui/commit/64bbdf7a73724986fac8bcf6e880fe32ef9ac495) +- 🪝 **Responses API tool strictness.** A workspace tool, MCP server or OpenAPI server reaching a model on the Responses API was turned into a strict schema where it had never asked to be, so the model filled every optional field with empty strings, zeros and empty arrays, and search and filter tools were handed values where leaving them out was meant. [#30046](https://github.com/open-webui/open-webui/pull/30046), [#27750](https://github.com/open-webui/open-webui/issues/27750) +- 🪃 **Responses API tool calling.** A forced tool choice sent to a connection on the Responses API went out in the wrong shape and was refused by providers that check it, and a tool call in a reply that was not streamed came back as empty text, so nothing reading the API ever saw it. [#30095](https://github.com/open-webui/open-webui/pull/30095), [#30085](https://github.com/open-webui/open-webui/issues/30085) +- ⛓️ **Missing tools in links.** Opening a chat from a link whose "tools" or "tool-ids" parameter names a tool that no longer exists, or that the account cannot see, kept that id in the selection and sent it with the message; ids matching no tool the account has are now dropped and the rest of the link works as before. [#29803](https://github.com/open-webui/open-webui/pull/29803) +- 🚫 **Model editor error messages.** A workspace model that cannot be loaded for editing now says so, instead of sending you back with "You do not have permission to edit this model" whatever the real reason. [#29694](https://github.com/open-webui/open-webui/pull/29694), [#29629](https://github.com/open-webui/open-webui/issues/29629) +- 👥 **Directory updates that were dropped.** A change your identity provider sent as an add or a remove, or without naming the attribute it was changing, was accepted and then quietly discarded, so a rename or a deactivation never reached the account; those now take effect. [Commit](https://github.com/open-webui/open-webui/commit/ad9da981680c0fd9151c803a366a01545ea907c1) +- 🗃️ **Directory request validation.** A provisioning request carrying the wrong kind of value, or an attribute Open WebUI does not support, now comes back as an error instead of passing in silence, and a sync that changes nothing no longer marks the account as touched. [Commit](https://github.com/open-webui/open-webui/commit/ad9da981680c0fd9151c803a366a01545ea907c1) +- 🔘 **Model editor save button.** Saving a workspace model whose model list could not be refreshed afterwards now reports the error and frees the Save button, rather than leaving it disabled and spinning though the model had been saved. [Commit](https://github.com/open-webui/open-webui/commit/dc98e3023fc6e0113dbad76545cdb15e014db6ad), [Commit](https://github.com/open-webui/open-webui/commit/cc5479d16d5caf28ec19a16ffa1ee4f3dbc0f63f) +- 🫱 **Rating a reply again.** Switching a reply's rating from one thumb to the other kept the score and the reason given the first time, and rating a reply whose feedback had since been deleted failed outright; both now record the rating you just gave. [#30075](https://github.com/open-webui/open-webui/pull/30075), [#30077](https://github.com/open-webui/open-webui/pull/30077) +- 📶 **Sorting feedback by user.** The User column in the admin feedback history did nothing when clicked, and now sorts by who left the feedback. [#30191](https://github.com/open-webui/open-webui/pull/30191) +- 🪂 **Closing the Edit User dialog.** Changes typed into the admin Edit User dialog and then abandoned showed on the row in the user list until the page was reloaded, and closing the dialog now drops them. [#30193](https://github.com/open-webui/open-webui/pull/30193) +- 🪙 **Model defaults save.** Saving the model defaults in the admin settings put the selected, pinned and ordered models back as they stood when the page was opened, undoing anything changed in between. [#30206](https://github.com/open-webui/open-webui/pull/30206) +- 👤 **A custom gender shown back.** An account whose gender is a wording of its own came back to an empty dropdown in the account form, and the form now shows Custom with that wording beside it. [#30215](https://github.com/open-webui/open-webui/pull/30215) +- 🗓️ **Clearing a calendar event.** Emptying the repeat, the description or the location of a calendar event did not take and the old wording came back, and those fields can be cleared again. [#30204](https://github.com/open-webui/open-webui/pull/30204) +- 📕 **Required prompt dropdowns.** A prompt's form could be sent with a required dropdown left unchosen, and now asks you to pick something first. [#30078](https://github.com/open-webui/open-webui/pull/30078) +- 🎫 **Account form required fields.** The account form now refuses to save with a required field left empty, the way the admin user dialog does, and a date of birth is no longer demanded of accounts that never set one. [#30296](https://github.com/open-webui/open-webui/pull/30296), [#30295](https://github.com/open-webui/open-webui/issues/30295) +- 🐑 **Model clones keep their base.** Cloning a model from the admin Models settings now carries the model it was built on, where the clone came out detached from it, and an arena model no longer offers Clone at all. [#30080](https://github.com/open-webui/open-webui/pull/30080), [#30079](https://github.com/open-webui/open-webui/issues/30079) +- 🧤 **Model sharing survives a save.** Saving a model you cannot fully share no longer strips the access entries you cannot re-create, where an editor resending every stored entry had each one re-checked against what its author may grant, and a save from someone with narrow rights quietly took the model private for everyone else. [Commit](https://github.com/open-webui/open-webui/commit/754c4b5762ed0d79954ff28c4e06631004dc31b3), [#30093](https://github.com/open-webui/open-webui/pull/30093), [#30087](https://github.com/open-webui/open-webui/issues/30087) +- 📠 **Prompt version saves.** Saving a new version of a prompt without Set as Production leaves the live prompt exactly as it was, where the draft quietly took its place, and the editor now shows the production text the moment a version is set. [#30231](https://github.com/open-webui/open-webui/pull/30231), [#30230](https://github.com/open-webui/open-webui/issues/30230), [#30233](https://github.com/open-webui/open-webui/pull/30233), [#30232](https://github.com/open-webui/open-webui/issues/30232) +- 💬 **Model description round trips.** A workspace model whose description is switched from the default back to custom saves again, where the switch read the field as empty and the description was dropped on save. [#30249](https://github.com/open-webui/open-webui/pull/30249), [#30247](https://github.com/open-webui/open-webui/issues/30247) +- 🎚 **Compaction threshold saves.** The context compaction threshold set in general settings now reaches the model parameters, where the value never left the page. [#30270](https://github.com/open-webui/open-webui/pull/30270), [#30269](https://github.com/open-webui/open-webui/issues/30269) +- 📢 **Speech engine defaults.** Switching the text to speech engine now applies that engine's own default voice and model, where the change kept the previous engine's settings in place. [#30289](https://github.com/open-webui/open-webui/pull/30289), [#30288](https://github.com/open-webui/open-webui/issues/30288) +- 🗞 **MinerU key in local mode.** The document settings save again in local mode with the MinerU key left empty, where the form demanded a key it did not need. [#30299](https://github.com/open-webui/open-webui/pull/30299), [#30298](https://github.com/open-webui/open-webui/issues/30298) +- 🫂 **Group dialog reset.** The new-group dialog opens empty after a group is created, where the next one came up holding the group just made. [#30280](https://github.com/open-webui/open-webui/pull/30280), [#30279](https://github.com/open-webui/open-webui/issues/30279) +- 🧵 **Thread reply notifications.** Clicking the notification for a reply written inside a thread dropped you at the bottom of the channel with the thread still shut and the reply nowhere in sight; it now opens the thread the reply belongs to. [#29856](https://github.com/open-webui/open-webui/pull/29856), [#29855](https://github.com/open-webui/open-webui/issues/29855) +- 🔽 **Dropdown arrow spacing.** The arrow in the dropdowns drawn no wider than their contents, among them the provider on a new connection, no longer overlaps the last characters of the longest choice. [#29866](https://github.com/open-webui/open-webui/pull/29866), [#29865](https://github.com/open-webui/open-webui/issues/29865) +- 🧊 **Lowercase header handling.** Headers from an upstream that writes them in lower case, as anything served by uvicorn does, are now matched without regard to case, so "Content-Encoding" is stripped and clients stop failing to decompress a body the server had already decoded. [#29843](https://github.com/open-webui/open-webui/pull/29843) +- 🛑 **Complete chat stop.** Where a chat had more than one task in flight, stopping it, deleting it, or closing a note being worked on could stop the first and leave the rest running to the end, both because a task that had already finished ended the round early and because the list being worked through was rewritten underneath it as each one was cleared away; every task is now stopped in turn, so a reply that was calling tools stops calling them rather than running on to its own limit, though instances sharing their state through Redis were not affected. [Commit](https://github.com/open-webui/open-webui/commit/e35b907f737e625b900b3a03d61890f67ab0c4b0), [#29844](https://github.com/open-webui/open-webui/pull/29844), [#29816](https://github.com/open-webui/open-webui/issues/29816) +- 🖊️ **Channel code blocks.** Where a model answers in a channel with structured output, the code inside it was drawn as plain highlighted text rather than in the editor every other message uses, so it could not be edited in place and a diff in it could not be opened for editing; it now renders the same way as everywhere else. [#29861](https://github.com/open-webui/open-webui/pull/29861) +- 📐 **Code block edits on collapse.** An unsaved edit inside a code block stays on screen when the block is folded away and opened again, where collapsing it showed the saved text though the edit was still pending. [#30284](https://github.com/open-webui/open-webui/pull/30284), [#30283](https://github.com/open-webui/open-webui/issues/30283) +- 🚨 **Code run errors with output.** A code run that ends with an error now shows the error alongside what it printed, where a run that printed anything at all showed its error nowhere. [#30286](https://github.com/open-webui/open-webui/pull/30286), [#30285](https://github.com/open-webui/open-webui/issues/30285) +- 🔕 **Reactions on read-only channels.** The reaction picker no longer appears on a channel open to you as a viewer alone, matching the reply and menu options already hidden there. [#30241](https://github.com/open-webui/open-webui/pull/30241), [#30240](https://github.com/open-webui/open-webui/issues/30240) +- 🎯 **Knowledge search accuracy on PostgreSQL.** A fresh install using PostgreSQL built its search index before a single piece of text existed to organise it around, so the index was never fit for the content that arrived afterwards and every search quietly returned the wrong passages; the index now waits until there is enough text to build on, and until then searches read everything exactly. An install already carrying such an index can restore it by rebuilding that one index. [#30143](https://github.com/open-webui/open-webui/pull/30143), [#30134](https://github.com/open-webui/open-webui/issues/30134) +- 🗂 **Knowledge file filter on first click.** The File content filter in a knowledge base now narrows the listing the first time it is clicked, where the first click only armed the checkbox and everything stayed listed until it was clicked again. [#30211](https://github.com/open-webui/open-webui/pull/30211), [#30210](https://github.com/open-webui/open-webui/issues/30210) +- 🔭 **Knowledge base search recall.** Where many knowledge bases are stored together, a search of one holding a small share of what is stored found only a small share of its matches, and the shortfall grew as the store did; the search now keeps looking until it has enough from the knowledge base you asked for, and "PGVECTOR_ITERATIVE_SCAN" switches that off or makes it strict. [#30142](https://github.com/open-webui/open-webui/pull/30142), [#30135](https://github.com/open-webui/open-webui/issues/30135) +- 🪣 **Searches that found nothing.** A knowledge search that came back empty never handed its database connection back, so enough of them left knowledge search failing outright until the server was restarted; connections are returned now on PostgreSQL and on openGauss alike. [#30142](https://github.com/open-webui/open-webui/pull/30142), [#30133](https://github.com/open-webui/open-webui/issues/30133), [#30144](https://github.com/open-webui/open-webui/pull/30144) +- 🧽 **Knowledge folder deletion.** Deleting a folder without moving what was in it up a level dropped the files from the listing but left their text in the search index and the files themselves in storage, so a model went on retrieving and citing pages from a folder that was no longer there; the text is now removed with the folder, and a file no other knowledge base holds is deleted with it unless file retention is switched on. [Commit](https://github.com/open-webui/open-webui/commit/17dbc6f001aeea25ae1df528cb79bb272eca4a77) +- 🦀 **Regex searches over chat and knowledge files.** A pattern search now runs in time proportional to the text whatever the pattern, where a nasty expression could stall the search for good, and the patterns it accepts follow RE2's rules, with no lookarounds or backreferences and character classes matching ASCII only. [Commit](https://github.com/open-webui/open-webui/commit/97e013a66169c4fcd7f26ab9e41a4ff260ffd4da) +- ⏰ **Automation schedule parsing.** An automation whose rule puts the time in "DTSTART" is now read at that hour, rather than listed at midnight and opened at nine, which moved when it ran as soon as it was saved again. [Commit](https://github.com/open-webui/open-webui/commit/540467b90a430e47e536c20710878b342de659fb) +- ⏲️ **Recurrence counts and start dates.** A rule repeating a set number of times, ten or a hundred, is read as the limited repeat it is, where any count beginning with a one was treated as one-shot, and a rule carrying its start date on the same line as its repeat text now follows the start you picked, on schedules and calendar events alike. [#29262](https://github.com/open-webui/open-webui/pull/29262) +- ⌛ **Stalled schedules and stuck tasks.** A schedule whose rule takes too long to work out no longer holds up the round that evaluates it and is skipped with a warning, and background tasks shared through Redis now expire once their worker falls silent, after "REDIS_TASK_TTL" seconds. [Commit](https://github.com/open-webui/open-webui/commit/5fb869db221d9599a576e8e2eea9c3314947d25e) +- 📣 **Model mention chips.** A mention whose ID carried anything beyond letters, digits and a little punctuation, such as the brackets in some workspace model IDs, stayed on screen as the raw "<@…>" text both in the box you type in and in the message once sent; any ID without a space in it is now drawn as a chip. [#29864](https://github.com/open-webui/open-webui/pull/29864) +- 😀 **Multi-codepoint emoji.** An emoji whose shortcode is several codepoints, the flags among them, is inserted whole, where only its first part reached the message. [#30213](https://github.com/open-webui/open-webui/pull/30213), [#30212](https://github.com/open-webui/open-webui/issues/30212) +- 🔔 **Custom webhook names.** A webhook target named with a space or a slash is now tidied the way an automatic name always was, so it can still be edited, deleted, made the default or tested afterwards. [#29947](https://github.com/open-webui/open-webui/pull/29947) +- 🪛 **Paginated MCP tool lists.** A server that hands its tools back a page at a time had only the first page read, so the rest were never offered to a model; the whole list is now collected before the tools are built. [Commit](https://github.com/open-webui/open-webui/commit/ffae4116a8d58a820f1770c41c69dafe4d51c881) +- 💭 **Anthropic thinking blocks.** Open WebUI's own thinking blocks are no longer forwarded to an OpenAI-compatible backend, which since 0.11.0 made a strict server such as NVIDIA Dynamo refuse an Anthropic client's second turn; signed blocks still pass through. [#29849](https://github.com/open-webui/open-webui/pull/29849), [#29799](https://github.com/open-webui/open-webui/issues/29799) +- 🗨️ **Channel model terminals.** A model with a terminal chosen in the workspace had that choice honoured in a chat but dropped where it answered in a channel or ran as a channel automation, so it worked without one; it now carries the same terminal everywhere, alongside the tools, filters and features it already carried. [Commit](https://github.com/open-webui/open-webui/commit/c78ad89934095c4e42e3f059d400a24fe5681de2) +- 🧺 **Deleted channel messages leave no quotes.** Deleting a channel message now clears the quote of it sitting on every reply and drops it from the reply box, where a reply kept showing the deleted message and could still be sent addressed to it. [#30314](https://github.com/open-webui/open-webui/pull/30314), [#30313](https://github.com/open-webui/open-webui/issues/30313) +- 🔌 **Terminal tools need a terminal.** A model was offered the tools that read your terminal and type into it whether or not the chat had a terminal switched on and connected in your browser, so it could reach for one that was not there; those tools are now handed over only for the terminal the chat has open. [Commit](https://github.com/open-webui/open-webui/commit/ca1eefe2937081b010ffef3985997d17d3332fa3), [Commit](https://github.com/open-webui/open-webui/commit/1ddba7e2c6f625fbc2c131eb24d3b9775ad898ad) +- 🔤 **Custom header encoding.** The custom headers a connection sends are encoded once the values are filled in, so a person's name or group carrying anything beyond plain ASCII, a line break included, no longer breaks the request or reaches the other end as something else. [Commit](https://github.com/open-webui/open-webui/commit/7a4a4b93dca34f8ce0c481b180d3eea23797e984) +- 🆎 **Chromium spellcheck corrections.** Picking a suggestion from the browser's own spelling menu in the message box did nothing, or put the misspelling straight back, because a highlight meant for the notes editor was being drawn over the selection and rebuilding the text underneath it, taking the browser's spelling marks with it; that highlight is now kept out of the message box and only drawn where the editor is not in use. [#29952](https://github.com/open-webui/open-webui/pull/29952), [#29944](https://github.com/open-webui/open-webui/issues/29944) +- 🈁 **IME composition while renaming.** Confirming a chat rename with Enter or Escape part way through typing with an input method editor now finishes the composition without saving or cancelling the rename, where the key press acted at once. [Commit](https://github.com/open-webui/open-webui/commit/85146206f60a22385ed27dae30d4f00e7e2675eb) +- 🐳 **Dotless host addresses.** With local web fetching turned on, an address pointing at a container name on the same network, "http://apprise:8000" and the like, is now accepted rather than refused as invalid. [#29945](https://github.com/open-webui/open-webui/pull/29945), [#28161](https://github.com/open-webui/open-webui/issues/28161) +- 🪧 **False skill mentions.** Something written as "<$fh>" in a message, as Perl and other languages do, was taken for a mention of a skill and quietly removed before the model saw it, and drawn on screen as a chip; only mentions naming a skill that exists and is turned on are treated as mentions now. [Commit](https://github.com/open-webui/open-webui/commit/0edd731c7422870aa109fd0b758c6d68c06655da) +- 🎒 **Skill settings survive saving.** Saving a skill from its editor cleared the tags and translations it carried and switched it back on where it had been disabled, and all of that now survives the save. [#30185](https://github.com/open-webui/open-webui/pull/30185) +- 🫥 **Webhook avatar forwarding.** Turning "ENABLE_PROFILE_IMAGE_URL_FORWARDING" off stops browsers being sent on to outside picture addresses, and a channel webhook's picture was sent on regardless; it now serves the built-in picture like the rest, in the webhooks dialog as well as the message list, where the dialog had been sending every viewer's browser straight to the outside address. [#29889](https://github.com/open-webui/open-webui/pull/29889), [#29892](https://github.com/open-webui/open-webui/pull/29892) +- 🪟 **Statistics window origin.** The window that shares chat statistics with the community accepted requests from any page that opened it and answered to anywhere; it now reads and replies only where the community site is at the other end. [#29918](https://github.com/open-webui/open-webui/pull/29918) +- 🖼️ **Tool image rendering.** Where a tool answered with an image tucked inside an object or a list rather than on its own, the image was written into the conversation as its raw text, a single screenshot costing hundreds of thousands of tokens and crowding out everything else; such an image is now taken out wherever it sits and attached to the reply, so the model is handed the picture and you see it. An older fault that let every second image through untouched goes with it, and in a saved chat such an image is now kept as a file and referred to rather than written into the conversation itself, so the chat stays small. [#29665](https://github.com/open-webui/open-webui/pull/29665), [#29208](https://github.com/open-webui/open-webui/issues/29208), [Commit](https://github.com/open-webui/open-webui/commit/d372bec70427fe2d568e052ce5e1529e2ad41da9) +- ⏱️ **Stopped reply state.** Pressing stop saved the reply as finished while the parts inside it were still marked as running, so a block went on reading "Thinking..." or "Executing..." and came back that way after every reload; those parts are now closed off as the reply is stopped, and an open tab settles at once rather than only after a reload, while a tool call still waiting for your approval keeps its prompt. [#29495](https://github.com/open-webui/open-webui/pull/29495), [#29281](https://github.com/open-webui/open-webui/issues/29281) +- 🩺 **Knowledge sync errors.** A knowledge base sync that fails now names the file it happened on and what the browser said, rather than reporting nothing beyond "Error accessing directory". [#29507](https://github.com/open-webui/open-webui/pull/29507) +- 🚧 **Terminal proxy restrictions.** Requests passed through to a terminal server are now refused where they aim at that server's administrative endpoints, are not followed on to somewhere else, and are turned away where the path carries characters a parser would rewrite. [Commit](https://github.com/open-webui/open-webui/commit/51bb8cb142f72503e861eeee25ae4dc73c26c36b) +- ⛔ **Malformed tool calls.** Where a model asked for a tool with arguments that were not an object at all, a bare list or string, the reply stopped there; the model is now told what was wrong with the call and can try again. [Commit](https://github.com/open-webui/open-webui/commit/fed94c9f5af8a59660425d52df09e15fbedb25bc) +- 📡 **Broken stream reporting.** Where something failed part way through streaming an answer out of "/api/chat/completions", the stream simply stopped, leaving a client waiting on an answer that would never finish; it now closes with an error and a proper end of stream. [Commit](https://github.com/open-webui/open-webui/commit/c0fb36c9b833a85a3a7364e195cf54f3d6c7a787) +- 🧷 **Chat unblocked after an error.** A reply that failed, on a content filter or an exhausted quota, left the chat turning away everything you typed after it and stopped the message queue. Only the failed reply now ends, so the chat carries on and the other replies in a multi-model answer keep writing. [Commit](https://github.com/open-webui/open-webui/commit/dbb17a5725f9d7f844a6eee63ffca0bd077c7d94) +- 🫙 **Empty failed replies in history.** An assistant turn that ended in an error with nothing written is no longer handed back to the model as part of the conversation when you send your next message. [Commit](https://github.com/open-webui/open-webui/commit/dbb17a5725f9d7f844a6eee63ffca0bd077c7d94) +- 📭 **Empty page uploads.** Adding a web address to a knowledge base that came back without any text failed with a bare "Error uploading file" and, where the upload itself was refused, left the row sitting in the list; the reason now reaches you as it was given, and the row is taken away. [Commit](https://github.com/open-webui/open-webui/commit/6786ae1797eadaad7464a147213790e2d272822b) +- 🎞️ **Tool embed scope.** The frames a tool call can ask to have shown, which run scripts of their own, were drawn wherever a message was rendered, a channel among them; they are now drawn only in the replies of the chat you are in, and never in a channel. [#29985](https://github.com/open-webui/open-webui/pull/29985) +- 🚰 **Rejected picture addresses.** A model entry carrying a picture address the server refuses no longer leaves that address in memory, where anyone signed in could pile them up. [#29971](https://github.com/open-webui/open-webui/pull/29971) +- 🖌 **Editing a stored image.** Asking a model to edit an image this instance already holds now works whatever host its address names, where a container name, a default port or a self-composed host made the edit fail with a generic loading error. [#29691](https://github.com/open-webui/open-webui/pull/29691), [#29220](https://github.com/open-webui/open-webui/issues/29220) +- 🧪 **Memory replies carry less.** The memory tool no longer hands the model each memory's stored metadata, and a memory now records the model's id rather than the whole model entry. [Commit](https://github.com/open-webui/open-webui/commit/e9a0164690a8b1e190bdc8f4613e9d918b26d327) +- 🔇 **Memory fully off.** With memory switched off, stored memories are no longer folded into a reply's context and the memory tools are no longer offered to the model, where both went on reaching it behind the switch. [#30228](https://github.com/open-webui/open-webui/pull/30228), [#30227](https://github.com/open-webui/open-webui/issues/30227) +- 🧠 **Memory review behind the switch.** The background review that drafts new memories from a conversation no longer runs when memories are switched off or the account is barred from them, where it went on spending a task-model call every interval turn and failing at the write. [#30309](https://github.com/open-webui/open-webui/pull/30309) +- 🏗️ **Terminal server save button.** Saving a terminal server now waits for the save to finish before the dialog closes and cannot be set off twice by a second click. [Commit](https://github.com/open-webui/open-webui/commit/1cdd7aa459d6e96905324b452600ff56369d8a4e) +- 🗺️ **Terminal file panel paths.** The file panel beside a terminal now opens the file a model just wrote even when it is named with a relative path, where the panel could not match the name, jumped to the root and dragged the session's working directory with it. [#30282](https://github.com/open-webui/open-webui/pull/30282), [#30051](https://github.com/open-webui/open-webui/issues/30051) +- 🍴 **Forked chat folder.** Forking a chat put the copy in the original's folder even where you cannot write to that folder; it is now created outside any folder unless you can. [#30069](https://github.com/open-webui/open-webui/pull/30069) +- 🪢 **Dropping a folder in place.** Dragging a folder onto the folder it already sits in failed with "Folder already exists", and is now taken for the no-op it is. [#30169](https://github.com/open-webui/open-webui/pull/30169) +- 🗝️ **Read-only folders read-only everywhere.** A folder shared with you as a viewer no longer shows its edit controls on the empty-chat page, where they appeared and a save went through or failed depending on rights the page never checked. [Commit](https://github.com/open-webui/open-webui/commit/ee3ece1e2b8c9a38c94faf354ca020d66aba801d) +- 🚿 **Deleting a folder, keeping chats.** Removing a folder while keeping the chats inside it was refused for an account not allowed to delete chats, even though nothing was being deleted, and it now goes through. [#30163](https://github.com/open-webui/open-webui/pull/30163) +- 🏷️ **Tag cleanup after deletion.** An administrator deleting someone else's chat tidied unused tags out of their own account rather than the owner's, leaving the owner with tags nothing points at. [#30171](https://github.com/open-webui/open-webui/pull/30171) +- 🌱 **Forking past an unfinished reply.** A chat that held an interrupted reply anywhere in it refused every fork from then on and never recovered; forking now waits only on a reply actually being generated, and a turn paused for tool approval still forks with its prompt showing. [#30131](https://github.com/open-webui/open-webui/pull/30131), [#30128](https://github.com/open-webui/open-webui/issues/30128) +- 💾 **Deleted tool memory.** Deleting a tool or a function left the whole of its code in memory for as long as the server ran; it is now let go of along with the rest. [#29983](https://github.com/open-webui/open-webui/pull/29983) +- 🐌 **Sign-in rate limiting.** Counting sign-in attempts through Redis no longer stops the whole worker until Redis answers, so a slow Redis stops freezing every other request with it. [#29977](https://github.com/open-webui/open-webui/pull/29977) +- ♻️ **Session pool cleanup.** The task that clears out abandoned websocket sessions now carries on through an error from Redis instead of ending for good, and stops properly at shutdown. [#29976](https://github.com/open-webui/open-webui/pull/29976), [#29979](https://github.com/open-webui/open-webui/pull/29979) +- 🛰️ **Direct connections across workers.** A reply streamed over a direct connection no longer goes quiet part way through where several servers share their websocket traffic through Redis; the events it lives on now travel between workers the way the rest already did. [Commit](https://github.com/open-webui/open-webui/commit/0180efecf362d487e0c30f040f5948c325fbe337) +- 🫧 **Empty document ids.** A save arriving for a document with no id at all was filed against that empty id and never cleared, so anyone signed in could pile them up; nothing is filed for it now. [#29980](https://github.com/open-webui/open-webui/pull/29980) +- 🔬 **Page fetch CPU spin.** Fetching a page through the browser-driven loader never returned where that page opened a WebSocket, holding a worker thread at full CPU for the life of the process and costing another core on every further fetch, which left the whole instance slow. [#30050](https://github.com/open-webui/open-webui/pull/30050), [#30024](https://github.com/open-webui/open-webui/issues/30024) +- 🔁 **Duplicate search tracebacks.** A vector database outage wrote a full traceback twice for every collection and query pair, turning one outage into hundreds of identical stack traces per message on every replica; a single record now names every collection that failed. [#29981](https://github.com/open-webui/open-webui/pull/29981) +- 🧯 **Page fetch logging.** Fetching a page through the browser-driven loader filled the log with tracebacks where the page closed while it was still pulling pieces of itself, as sites behind Cloudflare and similar do; those requests are now let go of before the page closes. [#29325](https://github.com/open-webui/open-webui/pull/29325), [#28869](https://github.com/open-webui/open-webui/issues/28869) +- 📤 **Tool export scope.** Exporting all tools at once returned every tool the account could see, the source of a tool shared for reading included; it now returns only the tools it may edit, matching the single-tool export and the way models already export. [#29310](https://github.com/open-webui/open-webui/pull/29310) +- 🗳️ **Partial workspace exports.** Exporting the prompts or the models from the workspace wrote out only the page you happened to be looking at, so most of them were quietly left out of the file; both now export everything you are allowed to. [#30187](https://github.com/open-webui/open-webui/pull/30187) +- 🧳 **Imported chats keep more.** A chat brought back from an export arrived unpinned and unarchived and without the variables it was saved with, and all three now survive the round trip. [#30155](https://github.com/open-webui/open-webui/pull/30155), [#30151](https://github.com/open-webui/open-webui/pull/30151) +- 🗂️ **Unarchiving from search.** The menu on a search result offered to archive a chat that was already archived and said it had been archived when it had been brought back, and it now names and reports whichever of the two it did. [#30177](https://github.com/open-webui/open-webui/pull/30177) +- 🙈 **Folder filters with no match.** A chat search narrowed by a folder name that matches no folder now finds nothing, where the folder filter was quietly dropped and every chat came back. [#30273](https://github.com/open-webui/open-webui/pull/30273), [#29959](https://github.com/open-webui/open-webui/discussions/29959) +- 🧹 **Sidebar after bulk actions.** Archiving, deleting or unarchiving every chat at once, or importing a batch of them, left the folders and the pinned chats in the sidebar showing what was no longer there until the page was reloaded, and a bulk action that failed no longer reports success. [Commit](https://github.com/open-webui/open-webui/commit/8b3ee2827241ccc952a3073b2a6bbfad5df01827) +- 🔐 **Model pictures follow model access.** The picture belonging to a model is now shown only to people who can see that model, where anyone signed in could fetch it and tell an existing model from an unknown one by which picture came back. [#29700](https://github.com/open-webui/open-webui/pull/29700) +- 🚪 **Webhook pictures follow channel access.** The picture belonging to a channel webhook is now shown only to people with access to that channel, where anyone signed in could fetch it or be sent on to wherever it pointed, and it is refused outright where channels are turned off. [#29703](https://github.com/open-webui/open-webui/pull/29703) +- 📎 **Safer Word document previews.** Previewing a Word document no longer renders an HTML sub-document embedded inside it, and a link in one opens only where it points at a web address, a mail address or a telephone number. [#29699](https://github.com/open-webui/open-webui/pull/29699) +- 🚦 **Citation link schemes.** A source attached to a reply now opens only where it points at a web address, falling back to the panel that shows the source rather than following anything else. [#29701](https://github.com/open-webui/open-webui/pull/29701) +- 🈚 **Citation chips inside formatted text.** A citation inside bold, italic or linked text now renders its chip, where the formatting took it and the citation vanished from the sentence. [#30278](https://github.com/open-webui/open-webui/pull/30278), [#30277](https://github.com/open-webui/open-webui/issues/30277) +- 🐍 **Saving a tool or function.** Saving a tool or function in the admin pages no longer fails with a missing module error from the built-in code formatter, which was not installing everything it needed. [#29503](https://github.com/open-webui/open-webui/pull/29503) +- 🛠️ **Tool request duplication.** A tool that writes through an address carrying part of its input no longer has that part repeated in the body of the request as well, which servers checking their input strictly turned away, so those calls now go through. [#29717](https://github.com/open-webui/open-webui/pull/29717), [#29716](https://github.com/open-webui/open-webui/issues/29716) +- 🍎 **Answers survive on Apple Silicon.** Asking a question that searches a knowledge base with a locally run reranking model no longer takes the whole server down on a Mac, losing the answer and the connection with it. [#29735](https://github.com/open-webui/open-webui/pull/29735), [#29722](https://github.com/open-webui/open-webui/issues/29722) +- 📚 **Web results stop being cited.** Pages a web search only listed are no longer offered to the model as things to cite, which had it attaching a result id to text from a different result and the citations panel resolving that to a title that looked right. [#29631](https://github.com/open-webui/open-webui/pull/29631), [#29627](https://github.com/open-webui/open-webui/issues/29627) +- 🔎 **SearchApi errors, news and links.** Web search through searchapi.io now reports a bad key instead of coming back empty, reads the news results it returns alongside its ordinary ones, and hands the web loader the resolved destination link, so citations stop pointing at a redirect page. [#30308](https://github.com/open-webui/open-webui/pull/30308), [#30305](https://github.com/open-webui/open-webui/issues/30305) +- 🔼 **Honest version checks.** An instance that cannot reach the release listing now says the check failed, instead of reporting whatever it is running as the newest version and recording nothing about it. [#29626](https://github.com/open-webui/open-webui/pull/29626), [#29580](https://github.com/open-webui/open-webui/issues/29580) +- 🏟️ **Arena models report their errors.** A message to an arena model whose provider answers with an error now shows that error in the chat, where it used to fail on something unrelated and leave the real reason unsaid, and titles and tags no longer break the same way. [#29662](https://github.com/open-webui/open-webui/pull/29662), [#29658](https://github.com/open-webui/open-webui/issues/29658) +- 🏳️ **Nameless tool calls fail once.** A model endpoint that sends a tool call with no name at all now has that call fail on the spot, rather than the missing name being kept, stored with the message and sent back on the next turn for the endpoint to reject. [#29690](https://github.com/open-webui/open-webui/pull/29690), [#29686](https://github.com/open-webui/open-webui/issues/29686) +- 🧹 **Direct connections stop leaking listeners.** A server talking to a direct connection no longer leaves a listener behind for every request that ends any way but a clean finish, which grew without limit while a connection kept failing. [#29509](https://github.com/open-webui/open-webui/pull/29509) +- 🚀 **Cheaper model refreshes.** The model registry shared through Redis is now written only when the models themselves change, rather than on every refresh because of the countdown Ollama attaches to a model it holds in memory. [Commit](https://github.com/open-webui/open-webui/commit/649c012ecf308a994ea180127f7f8f94d0aec311) +- ✍️ **Continued reply text.** Asking for the rest of a cut-off reply in a temporary chat replaced what was on screen with only the new text, because the message being continued was read back from the saved chat it did not have. It is now taken from the request before the model is called, the continuation joins the same message instead of arriving as a second one, on a connection whose provider is set to llama.cpp the model is told to carry on from the text it is handed rather than repeat it back, opening the result in the message editor no longer shows a line break where the two halves meet, and, where haptic feedback is switched on, a continuation buzzes as it streams like any other reply. [Commit](https://github.com/open-webui/open-webui/commit/77d2000eb79e1cb6ae2004d40e8cca8c9e754cd0), [Commit](https://github.com/open-webui/open-webui/commit/6c7aa3543d21442241f6c53add5ff623ec816c44), [Commit](https://github.com/open-webui/open-webui/commit/57fc344873edc0db9e9f7ff3e9fb167cd80e3ef2), [Commit](https://github.com/open-webui/open-webui/commit/7eefeef4f17118f81c87acb464ad562803a6f26c), [Commit](https://github.com/open-webui/open-webui/commit/d418840aa9c4b77f613308cdaa4f2c6a062a8715), [Commit](https://github.com/open-webui/open-webui/commit/3795d5b29253d4b8d7a0adbab457c7317c40f3c6), [Commit](https://github.com/open-webui/open-webui/commit/57acc2b68f2f9e40b53aa7e609fdd52a4b0d15c4) +- 🔗 **Cancelled edits keep attachments.** Cancelling the edit of a message no longer strips the files attached to it, where dropping the edit took the attachments down with it. [#30281](https://github.com/open-webui/open-webui/pull/30281), [#30192](https://github.com/open-webui/open-webui/issues/30192) +- 🏎️ **Faster media page reads.** The browser-driven loader pulled every image, video and font a page referenced down through the server before any text was extracted, so a page carrying a few dozen audio players took ten seconds or timed out. Those requests are now dropped before they are made, and the same page comes back in under three seconds, having pulled 4 MB where it used to pull 55. [#29742](https://github.com/open-webui/open-webui/pull/29742), [#29741](https://github.com/open-webui/open-webui/issues/29741) +- 📝 **Starting a note from search.** Starting a note from the search box now works when you are already on the notes page, keeps the whole of what you typed including characters such as ampersands and hashes, and no longer makes a further note each time the browser back button is pressed. [#29645](https://github.com/open-webui/open-webui/pull/29645), [#29642](https://github.com/open-webui/open-webui/issues/29642) +- 📱 **Apple device replies.** An assistant reply no longer comes up blank in a home screen app, an in-app browser or a desktop-class window on Apple devices, where the check that avoided the drawing fault only recognised Safari itself. [#29734](https://github.com/open-webui/open-webui/pull/29734), [#29688](https://github.com/open-webui/open-webui/issues/29688), [#26712](https://github.com/open-webui/open-webui/issues/26712) +- 📊 **Single source relevance.** A reply drawing on a single source now shows how relevant that source is, where the figure appeared only once a second source joined it and so looked as though it came and went. [#29647](https://github.com/open-webui/open-webui/pull/29647), [#29646](https://github.com/open-webui/open-webui/issues/29646) +- 🔧 **Arduino sketches upload to knowledge.** A sketch file now reaches the plain text reader like the C++ and header files beside it, rather than being handed to a document extraction server that could make nothing of it and failing the upload. [#29673](https://github.com/open-webui/open-webui/pull/29673), [#29670](https://github.com/open-webui/open-webui/issues/29670) +- 📰 **Docling conversion failures.** A file that Docling refuses or fails to convert now fails the upload with the reason Docling gave, rather than breaking with a raw error or quietly filing a placeholder that was then indexed and handed to the model in place of the file. [#30107](https://github.com/open-webui/open-webui/pull/30107), [#29808](https://github.com/open-webui/open-webui/issues/29808) +- 📄 **Uploaded text kept as written.** A file whose text contains escape sequences such as the one standing for a non-breaking space is now stored and read by the model exactly as it was written, rather than having some of them rewritten depending on where in the file they sat. [#29736](https://github.com/open-webui/open-webui/pull/29736), [#29732](https://github.com/open-webui/open-webui/issues/29732) +- 🔦 **Readable slash command labels.** The entries in the slash command menu no longer show as white text on a white background in the light theme. [#29512](https://github.com/open-webui/open-webui/pull/29512), [#29510](https://github.com/open-webui/open-webui/issues/29510) +- ⌨️ **Literal arrow sequences.** A sequence such as three hyphens after a less-than sign is now shown as the characters it is made of rather than drawn as an arrow, which had text look changed when it never was. [#29595](https://github.com/open-webui/open-webui/pull/29595), [#29594](https://github.com/open-webui/open-webui/issues/29594) +- 🖌️ **Editing an image you uploaded.** An image already held by Open WebUI can now be used with image editing, where fetching its own link back over the network could fail on a private network or without a sign-in. [Commit](https://github.com/open-webui/open-webui/commit/50413f34824ea49d5b94d3a97f3fe4bb2e881e38) +- 🖼 **Playground image edits.** Editing an image in the Images playground now works, where every attempt came back rejected since the request carried its fields under a heading the endpoint never read. [Commit](https://github.com/open-webui/open-webui/commit/c07fa08b995e8d1a1fc2d94a88d8cea691bdc5ee) +- 🎨 **A tidier attach webpage dialog.** The row holding the Add button no longer carries a grey band of its own between the address box and the button, matching every other dialog. [#29664](https://github.com/open-webui/open-webui/pull/29664), [#29663](https://github.com/open-webui/open-webui/issues/29663) +- 🖱️ **A plain note date.** The date under a note title no longer shows a pointing hand or announces itself as something to press, having never done anything when clicked. [#29708](https://github.com/open-webui/open-webui/pull/29708) +- 🌇 **Folder backgrounds on creation.** The background image picked in the Create Folder dialog now arrives on the folder when it is created from the sidebar, where the image was discarded unless the dialog was opened from an existing folder. [#30218](https://github.com/open-webui/open-webui/pull/30218), [#30217](https://github.com/open-webui/open-webui/issues/30217) +- 🖇 **Delete Chat shortcut everywhere.** The keyboard shortcut that deletes the open chat now works wherever the chat was opened from, where it did nothing unless the chat's row happened to be on screen in the sidebar at that moment. [#30165](https://github.com/open-webui/open-webui/pull/30165), [#30164](https://github.com/open-webui/open-webui/issues/30164) +- 🚮 **Retired chat variables.** A model whose system prompt no longer declares a variable a previous prompt did stops asking for it, where every new chat kept opening the dialog and refusing to send until a value was entered. [#30173](https://github.com/open-webui/open-webui/pull/30173), [#30172](https://github.com/open-webui/open-webui/issues/30172) +- 👁️ **Compact hover previews in Safari.** Holding over a chat in the sidebar shows the small preview every other browser shows, rather than one laid out at the width and spacing of a full conversation. [#29734](https://github.com/open-webui/open-webui/pull/29734) +- 🔖 **Visible title generation faults.** When a new chat's automatic title cannot be generated, the reason now reaches the log at the default level, rather than only under debug logging where a broken feature looked the same as a switched-off one. [#30106](https://github.com/open-webui/open-webui/pull/30106), [#29533](https://github.com/open-webui/open-webui/issues/29533) + +### Changed + +- 🪶 **Slim starts with nothing configured.** The slim image leaves out the local models and the libraries around them, and still starts and holds a conversation on its defaults; the features that leaned on those models each need an external service of their own. [Commit](https://github.com/open-webui/open-webui/commit/cb942bb94c8dc7941336088fb3392e2398ff56c1) +- 🗄️ **Slim database support.** The slim image runs on SQLite, its default, or on PostgreSQL; pointed at MySQL, MariaDB or another engine, or started with AWS RDS IAM logins switched on, it stops with an error instead of starting, and those deployments need the standard image. [Commit](https://github.com/open-webui/open-webui/commit/d27aa72ab4a7b5632b4ad49e8467081ad3d7ebb4) +- 📁 **Slim file storage.** The slim image keeps files on local storage, its default; configured for an S3, Google Cloud or Azure bucket, it stops with an error instead of starting, and those deployments need the standard image. [Commit](https://github.com/open-webui/open-webui/commit/d27aa72ab4a7b5632b4ad49e8467081ad3d7ebb4) +- 🧮 **Slim searches only through pgvector.** Knowledge search on the slim image needs PostgreSQL with pgvector, and configured for another vector store the instance still starts, and the failure arrives the first time something is searched rather than at startup. [Commit](https://github.com/open-webui/open-webui/commit/cb942bb94c8dc7941336088fb3392e2398ff56c1), [Commit](https://github.com/open-webui/open-webui/commit/d27aa72ab4a7b5632b4ad49e8467081ad3d7ebb4) +- 🧠 **Slim embedding requirements.** The slim image carries no embedding or reranking model, so knowledge needs OpenAI, Ollama or Azure OpenAI embeddings and an external reranker, falling back to plain cosine scoring where none is set. [Commit](https://github.com/open-webui/open-webui/commit/cb942bb94c8dc7941336088fb3392e2398ff56c1) +- ✂️ **Slim document splitting.** Splitting a document along a downloaded tokenizer is unavailable on the slim image, which leaves splitting by character or by token count. [Commit](https://github.com/open-webui/open-webui/commit/cb942bb94c8dc7941336088fb3392e2398ff56c1) +- 📃 **Slim document readers.** The slim image reads text, Markdown, CSV, HTML and XML files as they are; uploading a PDF, a Word file or a presentation fails unless one of the external document extractors is configured. [Commit](https://github.com/open-webui/open-webui/commit/cb942bb94c8dc7941336088fb3392e2398ff56c1) +- 🎙️ **Slim speech requirements.** The slim image carries neither local Whisper nor local voices, so speech to text and text to speech need an external engine before they will work. [Commit](https://github.com/open-webui/open-webui/commit/cb942bb94c8dc7941336088fb3392e2398ff56c1) +- 🕸️ **Slim web page fetching.** The slim image carries no headless browser, so a web page is fetched over plain HTTP or through an external loader, and a page that draws itself with JavaScript comes back with less of its content than on the standard image. [Commit](https://github.com/open-webui/open-webui/commit/cb942bb94c8dc7941336088fb3392e2398ff56c1), [Commit](https://github.com/open-webui/open-webui/commit/d27aa72ab4a7b5632b4ad49e8467081ad3d7ebb4) +- 🔎 **Slim leaves out DDGS.** DDGS, the metasearch provider that needs no key of its own, is not carried in the slim image, so web search there needs a provider with a key. [Commit](https://github.com/open-webui/open-webui/commit/0fa4dea5ff64ea663f162d07c9a1175c39e4aad0) +- 📥 **Slim code interpreter packages.** The slim image leaves out the code interpreter's packages, so the browser fetches numpy, pandas, matplotlib, scikit-learn and the rest from "cdn.jsdelivr.net" and the interpreter stops working where that is blocked. [Commit](https://github.com/open-webui/open-webui/commit/98fcb844e1b19f7dd6289af26273cdec5447dc52) +- 🧰 **Slim git requirements.** The slim image no longer carries git, so a tool or function whose requirements point at a "git+https://" address fails to install and needs the standard image or a published package. [Commit](https://github.com/open-webui/open-webui/commit/30eed1251301f74e0dfeaad09c41e81320f790fc) +- 🧺 **LangChain community removal.** The readers for text, HTML, Word, CSV, PDF and Azure Document Intelligence are now written here rather than taken from "langchain-community", which is no longer installed; a tool or function importing it has to name it in its own requirements from now on. [Commit](https://github.com/open-webui/open-webui/commit/05484aa055a868a49842e1ddff169c19c98b755b) +- 🧾 **Undeclared package imports.** Packages that sat in the image only by accident, among them nltk, pymongo, the Google Drive client and the Gemini SDK, are no longer installed, so a tool or function importing one must name it in its own requirements. [#29725](https://github.com/open-webui/open-webui/pull/29725), [#29726](https://github.com/open-webui/open-webui/pull/29726), [Commit](https://github.com/open-webui/open-webui/commit/a1c02098aa2687c72482a59117efe643b785df51) +- 🆔 **Model ID whitespace.** A workspace model whose ID contains a space or a tab is now refused in the editor, through the API and on import; one already stored goes on answering but cannot be saved again until it is recreated. [Commit](https://github.com/open-webui/open-webui/commit/8a19e2f867063256bb2836649e2fe80af41ef748) +- 🖇️ **Link scheme rendering.** A link in a reply, a citation or a web search result is now rendered only where it points at a web address, a mail address, a telephone number or somewhere inside this instance; anything else, an "ftp://" address or an application link such as "obsidian://" or "vscode://" among them, is shown as the text it is. Two old oddities go with it: a source written as "HTTP://" now becomes a link, and a filename merely containing the letters http no longer becomes one that leads nowhere. [#29890](https://github.com/open-webui/open-webui/pull/29890) +- 🧲 **Integrations tab is opt-in.** The Integrations tab in personal settings, where tool and terminal connections of your own are managed, is now hidden until an administrator turns on Direct Integrations under Integrations or sets "ENABLE_DIRECT_INTEGRATIONS", and hiding it leaves existing connections working. [Commit](https://github.com/open-webui/open-webui/commit/6d8e63e3666e1b1aa5540ffda0816be5f40c5271), [Commit](https://github.com/open-webui/open-webui/commit/c82634b9d01adadbf9780ff84a0a8168fdc4fdea) +- 📡 **Image connection check endpoint.** The endpoint that checks an image generation connection has moved and now takes the connection to test in the request itself, so anything calling the old address needs updating. [Commit](https://github.com/open-webui/open-webui/commit/64bbdf7a73724986fac8bcf6e880fe32ef9ac495) + ## [0.11.3] - 2026-08-31 ### Added @@ -287,6 +534,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - 🛎️ **Losing all your settings.** Your interface settings are no longer wiped by a session that failed to load them, which could happen with no action on your part and cleared everything from your theme to your model parameters; saving now changes only the settings you actually changed, and a session that cannot load them tells you instead of carrying on as though you had none. [#27766](https://github.com/open-webui/open-webui/issues/27766), [Commit](https://github.com/open-webui/open-webui/commit/ad8c79f68657bd3bcf5db6be650e498bb904b36b) - 🖲️ **Losing the collapsed sidebar.** With the sidebar collapsed, opening a chat no longer pushes the narrow sidebar strip off the edge of the screen, which left no way to reopen the sidebar short of shrinking the window to phone size. [#28501](https://github.com/open-webui/open-webui/pull/28501), [#28500](https://github.com/open-webui/open-webui/issues/28500) - 🧯 **Timers that fail without saying so.** A timer whose reply cannot be generated, such as one set against a model that has since been removed, is now recorded as failed with the reason, instead of being marked as completed while the reply never arrives. [#27785](https://github.com/open-webui/open-webui/pull/27785), [#27783](https://github.com/open-webui/open-webui/issues/27783) +- 🎲 **Timers stop for retired owners.** A timer whose owner has been deleted or demoted to pending is recorded as an error instead of running, so a retired account no longer answers through a timer it set while active. [#30220](https://github.com/open-webui/open-webui/pull/30220) - 🖊️ **Message buttons in channels.** The buttons that appear when you hover a channel message now sit above the message rather than over its content, so they can be clicked on a message that starts with a code block or a table, and so the code and table controls stay clickable too. [#27737](https://github.com/open-webui/open-webui/pull/27737), [#27736](https://github.com/open-webui/open-webui/issues/27736) - 🔡 **Searching for non-English tags and text.** Searching workspace models by tag, or prompts and automations by their contents, now finds entries containing characters outside the English alphabet, where roughly half were missed depending on which settings were in force when each one was saved. [#28399](https://github.com/open-webui/open-webui/pull/28399) - 🔭 **Searching the calendar without an end date.** Asking a model to search your calendar without naming an end date now works on PostgreSQL, where the open-ended range was too large for the database to accept and the search failed outright. [Commit](https://github.com/open-webui/open-webui/commit/9550731cc17759f6862595b8cd849ae48695b5c1), [#27717](https://github.com/open-webui/open-webui/issues/27717) diff --git a/backend/open_webui/__init__.py b/backend/open_webui/__init__.py index e803cea46691..c56c875566aa 100644 --- a/backend/open_webui/__init__.py +++ b/backend/open_webui/__init__.py @@ -1,6 +1,6 @@ import base64 import os -import random +import secrets import sys from pathlib import Path from typing import Annotated @@ -45,7 +45,7 @@ def serve( if key_length < 1: raise ValueError('WEBUI_SECRET_KEY_LENGTH must be a positive integer') typer.echo(f'Generating a new secret key and saving it to {KEY_FILE}') - KEY_FILE.write_bytes(base64.b64encode(random.randbytes(key_length))) + KEY_FILE.write_bytes(base64.b64encode(secrets.token_bytes(key_length))) typer.echo(f'Loading WEBUI_SECRET_KEY from {KEY_FILE}') os.environ['WEBUI_SECRET_KEY'] = KEY_FILE.read_text() diff --git a/backend/open_webui/config.py b/backend/open_webui/config.py index 5765a26ab6b4..cc38281c4bfb 100644 --- a/backend/open_webui/config.py +++ b/backend/open_webui/config.py @@ -1302,6 +1302,8 @@ def reachable(host: str, port: int) -> bool: TAVILY_EXTRACT_DEPTH = os.getenv('TAVILY_EXTRACT_DEPTH', 'basic') +TAVILY_SEARCH_DEPTH = os.getenv('TAVILY_SEARCH_DEPTH', 'basic') + STAAN_API_KEY = os.getenv('STAAN_API_KEY', '') STAAN_MARKET = os.getenv('STAAN_MARKET', 'en-us') @@ -1675,43 +1677,13 @@ def reachable(host: str, port: int) -> bool: DEFAULT_PINNED_MODELS = os.getenv('DEFAULT_PINNED_MODELS', None) +# None uses the frontend's localized defaults; an empty list disables suggestions. try: - default_prompt_suggestions = JSONCodec.loads(os.getenv('DEFAULT_PROMPT_SUGGESTIONS', '[]')) + DEFAULT_PROMPT_SUGGESTIONS = JSONCodec.loads(os.getenv('DEFAULT_PROMPT_SUGGESTIONS', 'null')) except Exception as e: log.exception(f'Error loading DEFAULT_PROMPT_SUGGESTIONS: {e}') - default_prompt_suggestions = [] -if default_prompt_suggestions == []: - default_prompt_suggestions = [ - { - 'title': ['Help me study', 'vocabulary for a college entrance exam'], - 'content': "Help me study vocabulary: write a sentence for me to fill in the blank, and I'll try to pick the correct option.", - }, - { - 'title': ['Give me ideas', "for what to do with my kids' art"], - 'content': "What are 5 creative things I could do with my kids' art? I don't want to throw them away, but it's also so much clutter.", - }, - { - 'title': ['Tell me a fun fact', 'about the Roman Empire'], - 'content': 'Tell me a random fun fact about the Roman Empire', - }, - { - 'title': ['Show me a code snippet', "of a website's sticky header"], - 'content': "Show me a code snippet of a website's sticky header in CSS and JavaScript.", - }, - { - 'title': [ - 'Explain options trading', - "if I'm familiar with buying and selling stocks", - ], - 'content': "Explain options trading in simple terms if I'm familiar with buying and selling stocks.", - }, - { - 'title': ['Overcome procrastination', 'give me tips'], - 'content': 'Could you start by asking me about instances when I procrastinate the most and then give me some suggestions to overcome it?', - }, - ] + DEFAULT_PROMPT_SUGGESTIONS = None -DEFAULT_PROMPT_SUGGESTIONS = default_prompt_suggestions DEFAULT_PROMPT_SUGGESTIONS_I18N = {} try: @@ -3028,6 +3000,7 @@ def feishu_oauth_register(oauth: OAuth): 'web.search.sougou_api_sk': SOUGOU_API_SK, 'web.search.tavily_api_key': TAVILY_API_KEY, 'web.search.tavily_extract_depth': TAVILY_EXTRACT_DEPTH, + 'web.search.tavily_search_depth': TAVILY_SEARCH_DEPTH, 'web.search.staan_api_key': STAAN_API_KEY, 'web.search.staan_market': STAAN_MARKET, 'web.search.staan_max_snippets': STAAN_MAX_SNIPPETS, diff --git a/backend/open_webui/env.py b/backend/open_webui/env.py index 68c4272872a3..dc160e15925d 100644 --- a/backend/open_webui/env.py +++ b/backend/open_webui/env.py @@ -388,6 +388,14 @@ def parse_section(section): except ValueError: REDIS_RESPONSE_STREAM_TTL = 3600 +# Seconds a task survives without a heartbeat. 0 disables expiry. +try: + REDIS_TASK_TTL = int(os.getenv('REDIS_TASK_TTL', '300')) + if REDIS_TASK_TTL != 0 and REDIS_TASK_TTL < 60: + REDIS_TASK_TTL = 300 +except ValueError: + REDIS_TASK_TTL = 300 + REDIS_SENTINEL_HOSTS = os.getenv('REDIS_SENTINEL_HOSTS', '') REDIS_SENTINEL_PORT = os.getenv('REDIS_SENTINEL_PORT', '26379') @@ -485,6 +493,12 @@ def parse_section(section): WEBSOCKET_REDIS_URL = os.getenv('WEBSOCKET_REDIS_URL', REDIS_URL) WEBSOCKET_REDIS_CLUSTER = os.getenv('WEBSOCKET_REDIS_CLUSTER', str(REDIS_CLUSTER)).lower() == 'true' +# publishes room-targeted emits on per-room redis channels so instances skip +# messages for rooms without local members; must be identical across the fleet +# (toggle with a full restart, not a rolling one), set false for the previous +# shared-channel-only delivery +WEBSOCKET_REDIS_ROOM_CHANNELS = os.getenv('WEBSOCKET_REDIS_ROOM_CHANNELS', 'True').lower() == 'true' + websocket_redis_lock_timeout = os.getenv('WEBSOCKET_REDIS_LOCK_TIMEOUT', '60') try: @@ -1048,7 +1062,9 @@ def _parse_ssl_env(value: str) -> 'bool | _ssl.SSLContext': # Opt in to CPython's in-place string append optimization for streamed responses. # Off by default for a staged rollout. Only a host already out of memory can lose # text here; the default path (a full copy per chunk) raises there too. -ENABLE_CHAT_RESPONSE_STREAM_INPLACE_APPEND = os.getenv('ENABLE_CHAT_RESPONSE_STREAM_INPLACE_APPEND', 'False').lower() == 'true' +ENABLE_CHAT_RESPONSE_STREAM_INPLACE_APPEND = ( + os.getenv('ENABLE_CHAT_RESPONSE_STREAM_INPLACE_APPEND', 'False').lower() == 'true' +) # When enabled, uses a hardcoded extension-to-MIME dictionary as a last-resort # fallback when both mimetypes.guess_type() and file.meta.content_type fail to diff --git a/backend/open_webui/internal/db.py b/backend/open_webui/internal/db.py index 7fb0d17b1961..e2f523cfc233 100644 --- a/backend/open_webui/internal/db.py +++ b/backend/open_webui/internal/db.py @@ -348,15 +348,16 @@ def like(pattern, value, escape=None): if compiled is False: return False if compiled is None: - regex = [] + segments = [''] escaped = False for char in pattern: if escape and not escaped and char == escape: escaped = True continue - regex.append( - '.*' if not escaped and char == '%' else '.' if not escaped and char == '_' else re.escape(char) - ) + if not escaped and char == '%': + segments.append('') + else: + segments[-1] += '.' if not escaped and char == '_' else re.escape(char) escaped = False if escaped: compiled = False @@ -364,7 +365,11 @@ def like(pattern, value, escape=None): compiled_patterns.clear() compiled_patterns[key] = compiled return False - compiled = re.compile(''.join(regex), re.DOTALL) + # Atomic groups pin each middle segment to its first match, so '%' never backtracks. + regex = segments[0] + ''.join(f'(?>.*?{segment})' for segment in segments[1:-1]) + if len(segments) > 1: + regex += '.*' + segments[-1] + compiled = re.compile(regex, re.DOTALL) if len(compiled_patterns) >= 512: compiled_patterns.clear() compiled_patterns[key] = compiled diff --git a/backend/open_webui/main.py b/backend/open_webui/main.py index 3cf7619332f4..ecba6407f342 100644 --- a/backend/open_webui/main.py +++ b/backend/open_webui/main.py @@ -74,6 +74,7 @@ seed_registered_defaults, ) from open_webui.constants import ERROR_MESSAGES, TASKS +from open_webui.utils.recurrence import RecurrenceEvaluationTimeout from open_webui.env import ( USE_SLIM, AIOHTTP_CLIENT_SESSION_SSL, @@ -107,12 +108,14 @@ MAX_BODY_LOG_SIZE, # Redis REDIS_KEY_PREFIX, + REDIS_TASK_TTL, REDIS_URL, RESET_CONFIG_ON_START, SAFE_MODE, SCIM_TOKEN, VERSION, WEBSOCKET_HEARTBEAT_INTERVAL, + WEBSOCKET_MANAGER, # Admin Account Runtime Creation WEBUI_ADMIN_EMAIL, WEBUI_ADMIN_NAME, @@ -189,6 +192,7 @@ get_user_id_from_session_pool, periodic_session_pool_cleanup, periodic_usage_pool_cleanup, + redis_event_listener, ) from open_webui.socket.main import ( app as socket_app, @@ -200,6 +204,7 @@ list_task_ids_by_item_id, list_tasks, redis_task_command_listener, + redis_task_heartbeat, stop_item_tasks, stop_task, ) # Import from tasks.py @@ -232,6 +237,7 @@ normalize_chat_variables, ) from open_webui.utils.embeddings import generate_embeddings +from open_webui.utils.headers import get_headers_and_cookies from open_webui.utils.json_codec import JSONCodec from open_webui.utils.json_response import apply_orjson_http_json from open_webui.utils.logger import start_logger @@ -259,7 +265,7 @@ encrypt_data, get_oauth_client_info_with_dynamic_client_registration, get_oauth_client_info_with_static_credentials, - recover_static_oauth_client_metadata, + recover_oauth_client_metadata, resolve_oauth_client_info, ) from open_webui.utils.plugin import install_tool_and_function_dependencies @@ -386,6 +392,11 @@ async def lifespan(app: FastAPI): if app.state.redis is not None: app.state.redis_task_command_listener = asyncio.create_task(redis_task_command_listener(app)) + if REDIS_TASK_TTL > 0: + app.state.redis_task_heartbeat = asyncio.create_task(redis_task_heartbeat(app)) + + if WEBSOCKET_MANAGER == 'redis': + app.state.redis_event_listener = asyncio.create_task(redis_event_listener()) app.state.periodic_usage_pool_cleanup = asyncio.create_task(periodic_usage_pool_cleanup()) app.state.periodic_session_pool_cleanup = asyncio.create_task(periodic_session_pool_cleanup()) @@ -473,6 +484,12 @@ async def lifespan(app: FastAPI): if hasattr(app.state, 'redis_task_command_listener'): app.state.redis_task_command_listener.cancel() + if hasattr(app.state, 'redis_task_heartbeat'): + app.state.redis_task_heartbeat.cancel() + + if hasattr(app.state, 'redis_event_listener'): + app.state.redis_event_listener.cancel() + app.state.periodic_usage_pool_cleanup.cancel() app.state.periodic_session_pool_cleanup.cancel() app.state.scheduler_worker_loop.cancel() @@ -495,6 +512,12 @@ async def lifespan(app: FastAPI): lifespan=lifespan, ) + +@app.exception_handler(RecurrenceEvaluationTimeout) +async def recurrence_timeout_handler(request: Request, exc: RecurrenceEvaluationTimeout): + return JSONResponse(status_code=400, content={'detail': str(exc)}) + + # Used by readiness checks to gate traffic until startup work is done. app.state.startup_complete = False @@ -614,9 +637,7 @@ async def initialize_runtime_config(app: FastAPI): if server_id and auth_type in ('oauth_2.1', 'oauth_2.1_static'): try: oauth_client_info = resolve_oauth_client_info(tool_server_connection) - oauth_client_info = await recover_static_oauth_client_metadata( - tool_server_connection, oauth_client_info - ) + oauth_client_info = await recover_oauth_client_metadata(tool_server_connection, oauth_client_info) oauth_client_info = apply_connection_oauth_options(tool_server_connection, oauth_client_info) app.state.oauth_client_manager.add_client( f'mcp:{server_id}', @@ -974,14 +995,12 @@ async def unload_model(request: Request, form_data: ModelUnloadForm, user=Depend try: timeout = aiohttp.ClientTimeout(total=30) async with aiohttp.ClientSession(timeout=timeout, trust_env=True) as session: - headers = { - 'Content-Type': 'application/json', - **({'Authorization': f'Bearer {key}'} if key else {}), - } + headers, cookies = await get_headers_and_cookies(request, url, key, api_config, user=user) async with session.post( f'{url}/api/generate', data=payload, headers=headers, + cookies=cookies, ) as r: if not r.ok: errors.append({'url_idx': idx, 'error': await r.text()}) @@ -1015,14 +1034,12 @@ async def unload_model(request: Request, form_data: ModelUnloadForm, user=Depend try: timeout = aiohttp.ClientTimeout(total=30) async with aiohttp.ClientSession(timeout=timeout, trust_env=True) as session: - headers = { - 'Content-Type': 'application/json', - **({'Authorization': f'Bearer {key}'} if key else {}), - } + headers, cookies = await get_headers_and_cookies(request, base_url, key, api_config, user=user) async with session.post( f'{root_url}/models/unload', json={'model': actual_model}, headers=headers, + cookies=cookies, ) as r: if not r.ok: detail = await r.text() @@ -1470,7 +1487,9 @@ async def run_initial_title_generation(): asyncio.create_task(run_initial_title_generation()) else: # Existing chat — verify ownership - if not await Chats.is_chat_owner(chat_id, user.id) and user.role != 'admin': + if not await Chats.is_chat_owner(chat_id, user.id) and not ( + user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS + ): raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, detail=ERROR_MESSAGES.DEFAULT(), @@ -1625,7 +1644,8 @@ async def run_initial_title_generation(): async def process_chat(request, form_data, user, metadata, model, tasks=None): try: ctx = None - if metadata.get('assistant_message_id'): + # Saved chats load the message after approved tool calls run, so their results are kept + if metadata.get('assistant_message_id') and not is_saved_chat_id(metadata.get('chat_id')): ctx = await build_chat_response_context(request, form_data, user, model, metadata, tasks, []) form_data, metadata, events = await process_chat_payload(request, form_data, user, metadata, model) @@ -2058,7 +2078,7 @@ async def verify_chat_ownership(chat_id: str | None, user) -> None: detail='Channel chats are not supported on this endpoint', ) - if user.role != 'admin' and not await Chats.is_chat_owner(chat_id, user.id): + if not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS) and not await Chats.is_chat_owner(chat_id, user.id): raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, detail=ERROR_MESSAGES.DEFAULT(), @@ -2122,11 +2142,11 @@ async def list_tasks_by_chat_id_endpoint(request: Request, chat_id: str, user=De socket_id = get_temporary_chat_session_id(chat_id) if socket_id: owner_id = get_user_id_from_session_pool(socket_id) - if owner_id != user.id and user.role != 'admin': + if owner_id != user.id and not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS): return {'task_ids': []} else: chat = await Chats.get_chat_by_id(chat_id) - if chat is None or (chat.user_id != user.id and user.role != 'admin'): + if chat is None or (chat.user_id != user.id and not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS)): return {'task_ids': []} task_ids = await list_task_ids_by_item_id(request.app.state.redis, chat_id) @@ -2141,11 +2161,11 @@ async def stop_tasks_by_chat_id_endpoint(request: Request, chat_id: str, user=De chat = None if socket_id: owner_id = get_user_id_from_session_pool(socket_id) - if owner_id != user.id and user.role != 'admin': + if owner_id != user.id and not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS): raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=ERROR_MESSAGES.NOT_FOUND) else: chat = await Chats.get_chat_by_id(chat_id) - if chat is None or (chat.user_id != user.id and user.role != 'admin'): + if chat is None or (chat.user_id != user.id and not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS)): raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=ERROR_MESSAGES.NOT_FOUND) result = await stop_item_tasks(request.app.state.redis, chat_id) diff --git a/backend/open_webui/models/access_grants.py b/backend/open_webui/models/access_grants.py index 03a7ef72029e..49cb83a18039 100644 --- a/backend/open_webui/models/access_grants.py +++ b/backend/open_webui/models/access_grants.py @@ -686,8 +686,7 @@ async def get_users_with_access( Get all users who have the specified permission on a resource. Returns a list of UserModel instances. """ - from open_webui.models.groups import Groups - from open_webui.models.users import UserModel, Users + from open_webui.models.users import Users async with get_async_db_context(db) as db: result = await db.execute( @@ -699,27 +698,69 @@ async def get_users_with_access( ) grants = result.scalars().all() - # Check for public access - for grant in grants: - if grant.principal_type == 'user' and grant.principal_id == '*': - result = await Users.get_users(filter={'roles': ['!pending']}, db=db) - return result.get('users', []) - - user_ids_with_access = set() - - for grant in grants: - if grant.principal_type == 'user': - user_ids_with_access.add(grant.principal_id) - elif grant.principal_type == 'group': - group_user_ids = await Groups.get_group_user_ids_by_id(grant.principal_id, db=db) - if group_user_ids: - user_ids_with_access.update(group_user_ids) + user_ids_with_access = await self.get_user_ids_by_access_grants(grants, permission, db=db) if not user_ids_with_access: return [] return await Users.get_users_by_user_ids(list(user_ids_with_access), db=db) + async def get_user_ids_by_access_grants( + self, + access_grants: list[AccessGrantModel], + permission: str = 'read', + db: AsyncSession | None = None, + ) -> set[str]: + """Get user IDs with the specified permission, including public and group grants.""" + from open_webui.models.groups import Groups + from open_webui.models.users import Users + + async with get_async_db_context(db) as db: + user_ids = set() + group_ids = [] + for grant in access_grants: + if grant.permission != permission: + continue + if grant.principal_type == PRINCIPAL_TYPE_USER: + if grant.principal_id == WILDCARD_PRINCIPAL_ID: + result = await Users.get_users(filter={'roles': ['!pending']}, db=db) + return {user.id for user in result.get('users', [])} + user_ids.add(grant.principal_id) + elif grant.principal_type == PRINCIPAL_TYPE_GROUP: + group_ids.append(grant.principal_id) + + if group_ids: + group_user_ids = await Groups.get_group_user_ids_by_ids(group_ids, db=db) + for members in group_user_ids.values(): + user_ids.update(members) + return user_ids + + async def get_revoked_user_ids_by_resource( + self, + resource_type: str, + resource_id: str, + previous_access_grants: list[AccessGrantModel], + permission: str = 'read', + db: AsyncSession | None = None, + ) -> set[str]: + """Get user IDs that lost the specified permission after a resource's grants changed.""" + async with get_async_db_context(db) as db: + access_grants = await self.get_grants_by_resource(resource_type, resource_id, db=db) + previous_principals = { + (grant.principal_type, grant.principal_id) + for grant in previous_access_grants + if grant.permission == permission + } + principals = { + (grant.principal_type, grant.principal_id) for grant in access_grants if grant.permission == permission + } + if previous_principals <= principals or (PRINCIPAL_TYPE_USER, WILDCARD_PRINCIPAL_ID) in principals: + return set() + + previous_user_ids = await self.get_user_ids_by_access_grants(previous_access_grants, permission, db=db) + user_ids = await self.get_user_ids_by_access_grants(access_grants, permission, db=db) + return previous_user_ids - user_ids + def has_permission_filter( self, db, diff --git a/backend/open_webui/models/automations.py b/backend/open_webui/models/automations.py index 11a906670b85..954026cfc8c4 100644 --- a/backend/open_webui/models/automations.py +++ b/backend/open_webui/models/automations.py @@ -195,7 +195,7 @@ async def search_automations( stmt = stmt.filter( or_( Automation.name.ilike(f'%{query}%'), - *(data_text.ilike(f'%{variant}%') for variant in json_text_variants(query)), + *(data_text.icontains(variant, autoescape=True) for variant in json_text_variants(query)), ) ) @@ -312,6 +312,7 @@ async def claim_due(self, now_ns: int, limit: int = 10, db: Optional[AsyncSessio rows = result.scalars().all() from open_webui.utils.automations import next_run_ns + from open_webui.utils.recurrence import RecurrenceEvaluationTimeout # Batch-fetch user timezones so rescheduling respects each # user's local timezone instead of falling back to server time. @@ -323,13 +324,20 @@ async def claim_due(self, now_ns: int, limit: int = 10, db: Optional[AsyncSessio tz_result = await db.execute(select(User.id, User.timezone).where(User.id.in_(user_ids))) timezone_by_user_id = {uid: tz for uid, tz in tz_result.all()} + claimed = [] for row in rows: + try: + next_run_at = await next_run_ns(row.data.get('rrule', ''), tz=timezone_by_user_id.get(row.user_id)) + except RecurrenceEvaluationTimeout: + log.warning('Skipping automation %s: recurrence evaluation timed out', row.id) + continue row.last_run_at = now_ns - row.next_run_at = next_run_ns(row.data.get('rrule', ''), tz=timezone_by_user_id.get(row.user_id)) + row.next_run_at = next_run_at + claimed.append(row) await db.commit() - return [AutomationModel.model_validate(r) for r in rows] + return [AutomationModel.model_validate(r) for r in claimed] #################### diff --git a/backend/open_webui/models/calendar.py b/backend/open_webui/models/calendar.py index 3a3147a7ad03..fd7390fcea65 100644 --- a/backend/open_webui/models/calendar.py +++ b/backend/open_webui/models/calendar.py @@ -8,7 +8,7 @@ from open_webui.models.access_grants import AccessGrantModel, AccessGrants from open_webui.models.groups import Groups from open_webui.models.users import User, UserModel, UserResponse -from pydantic import BaseModel, ConfigDict, Field, field_validator +from pydantic import BaseModel, ConfigDict, Field from sqlalchemy import ( JSON, BigInteger, @@ -179,6 +179,20 @@ class CalendarUpdateForm(BaseModel): access_grants: Optional[list[dict]] = None +async def validate_calendar_rrule(value: Optional[str]) -> None: + if value: + from open_webui.utils.recurrence import rrule_interval_seconds + + try: + interval = await rrule_interval_seconds(value) + except ValueError: + raise + except Exception as e: + raise ValueError(ERROR_MESSAGES.AUTOMATION_INVALID_RRULE(e)) from e + if interval is not None and interval < MIN_CALENDAR_RRULE_INTERVAL_SECONDS: + raise ValueError(ERROR_MESSAGES.CALENDAR_RRULE_TOO_FREQUENT) + + class CalendarEventForm(BaseModel): calendar_id: str title: str @@ -193,22 +207,6 @@ class CalendarEventForm(BaseModel): meta: Optional[dict] = None attendees: Optional[list[dict]] = None - @field_validator('rrule') - @classmethod - def reject_sub_daily_rrule(cls, value: Optional[str]) -> Optional[str]: - if value: - from open_webui.utils.automations import rrule_interval_seconds - - try: - interval = rrule_interval_seconds(value) - except ValueError: - raise - except Exception as e: - raise ValueError(ERROR_MESSAGES.AUTOMATION_INVALID_RRULE(e)) - if interval is not None and interval < MIN_CALENDAR_RRULE_INTERVAL_SECONDS: - raise ValueError(ERROR_MESSAGES.CALENDAR_RRULE_TOO_FREQUENT) - return value - class CalendarEventUpdateForm(BaseModel): calendar_id: Optional[str] = None @@ -225,22 +223,6 @@ class CalendarEventUpdateForm(BaseModel): is_cancelled: Optional[bool] = None attendees: Optional[list[dict]] = None - @field_validator('rrule') - @classmethod - def reject_sub_daily_rrule(cls, value: Optional[str]) -> Optional[str]: - if value: - from open_webui.utils.automations import rrule_interval_seconds - - try: - interval = rrule_interval_seconds(value) - except ValueError: - raise - except Exception as e: - raise ValueError(ERROR_MESSAGES.AUTOMATION_INVALID_RRULE(e)) - if interval is not None and interval < MIN_CALENDAR_RRULE_INTERVAL_SECONDS: - raise ValueError(ERROR_MESSAGES.CALENDAR_RRULE_TOO_FREQUENT) - return value - class RSVPForm(BaseModel): status: str # 'accepted' | 'declined' | 'tentative' | 'pending' @@ -465,6 +447,7 @@ async def _to_event_model( async def insert_new_event( self, user_id: str, form_data: CalendarEventForm, db: Optional[AsyncSession] = None ) -> Optional[CalendarEventModel]: + await validate_calendar_rrule(form_data.rrule) async with get_async_db_context(db) as db: now = int(time.time_ns()) event = CalendarEvent( @@ -695,6 +678,7 @@ async def search_events( async def update_event_by_id( self, id: str, form_data: CalendarEventUpdateForm, db: Optional[AsyncSession] = None ) -> Optional[CalendarEventModel]: + await validate_calendar_rrule(form_data.rrule) async with get_async_db_context(db) as db: result = await db.execute(select(CalendarEvent).filter(CalendarEvent.id == id)) event = result.scalars().first() diff --git a/backend/open_webui/models/chat_messages.py b/backend/open_webui/models/chat_messages.py index baa45ab603a4..7ccbba7a87c1 100644 --- a/backend/open_webui/models/chat_messages.py +++ b/backend/open_webui/models/chat_messages.py @@ -1012,12 +1012,15 @@ async def get_hourly_message_counts_by_model( self, start_date: Optional[int] = None, end_date: Optional[int] = None, + group_id: Optional[str] = None, db: Optional[AsyncSession] = None, ) -> dict[str, dict[str, int]]: """Get message counts grouped by hour and model.""" async with get_async_db_context(db) as db: from datetime import datetime, timedelta + from open_webui.models.groups import GroupMember + stmt = select(ChatMessage.created_at, ChatMessage.model_id).filter( ChatMessage.role == 'assistant', ChatMessage.model_id.isnot(None), @@ -1027,6 +1030,9 @@ async def get_hourly_message_counts_by_model( stmt = stmt.filter(ChatMessage.created_at >= start_date) if end_date: stmt = stmt.filter(ChatMessage.created_at <= end_date) + if group_id: + group_users = select(GroupMember.user_id).filter(GroupMember.group_id == group_id).scalar_subquery() + stmt = stmt.filter(ChatMessage.user_id.in_(group_users)) result = await db.execute(stmt) results = result.all() diff --git a/backend/open_webui/models/chats.py b/backend/open_webui/models/chats.py index 46131dc8e9df..c5620f0cfef4 100644 --- a/backend/open_webui/models/chats.py +++ b/backend/open_webui/models/chats.py @@ -417,9 +417,6 @@ def _sanitize_chat_row(self, chat_item): """ Clean a Chat SQLAlchemy model's title + chat JSON, and return True if anything changed. - - The message write paths (upsert/status/delete) rely on this - leaving the blob clean and sanitize only the data they add. """ changed = False @@ -651,6 +648,7 @@ def _chat_import_form_to_chat_model(self, user_id: str, form_data: ChatImportFor 'current_message_id': form_data.current_message_id or self.get_current_message_id(form_data.chat), 'created_at': (form_data.created_at if form_data.created_at else int(time.time())), 'updated_at': (form_data.updated_at if form_data.updated_at else int(time.time())), + 'last_read_at': int(time.time()), } ) return chat @@ -1106,11 +1104,16 @@ async def get_message_by_id_and_message_id(self, id: str, message_id: str) -> di if messages_map and message_id in messages_map: return messages_map[message_id] - chat = await self.get_chat_by_id(id) - if chat is None: + # Messages the frontend saved straight into the chat blob have no chat_message row yet. + async with get_async_db_context() as session: + result = await session.execute(select(Chat.chat[('history', 'messages')]).filter_by(id=id)) + row = result.one_or_none() + + if row is None: return None - return chat.chat.get('history', {}).get('messages', {}).get(message_id, {}) + messages = row[0] or {} + return self._clean_null_bytes(messages.get(message_id, {})) async def get_message_metadata( self, @@ -1158,7 +1161,6 @@ async def upsert_message_to_chat_by_id_and_message_id( if chat_item is None: return None - self._sanitize_chat_row(chat_item) chat = chat_item.chat or {} self._repair_chat_current_id(chat) @@ -1166,7 +1168,7 @@ async def upsert_message_to_chat_by_id_and_message_id( saved_message = self.upsert_message_to_history(history, message_id, message) chat['history'] = history chat_item.chat = chat # chat is a fresh dict when the column was empty - chat_item.title = chat.get('title', 'New Chat') + chat_item.title = self._clean_null_bytes(chat.get('title', 'New Chat')) chat_item.current_message_id = self.get_current_message_id(chat) flag_modified(chat_item, 'chat') @@ -1204,7 +1206,6 @@ async def delete_message_from_chat_by_id_and_message_id(self, id: str, message_i if chat_item is None: return None - self._sanitize_chat_row(chat_item) chat = chat_item.chat or {} self._repair_chat_current_id(chat) @@ -1212,7 +1213,7 @@ async def delete_message_from_chat_by_id_and_message_id(self, id: str, message_i deleted_ids = self.delete_message_from_history(history, message_id) if not deleted_ids: chat_item.chat = chat - chat_item.title = chat.get('title', 'New Chat') + chat_item.title = self._clean_null_bytes(chat.get('title', 'New Chat')) chat_item.current_message_id = self.get_current_message_id(chat) flag_modified(chat_item, 'chat') await session.commit() @@ -1221,7 +1222,7 @@ async def delete_message_from_chat_by_id_and_message_id(self, id: str, message_i messages = history.get('messages') or {} chat['history'] = history chat_item.chat = chat - chat_item.title = chat.get('title', 'New Chat') + chat_item.title = self._clean_null_bytes(chat.get('title', 'New Chat')) chat_item.current_message_id = self.get_current_message_id(chat) flag_modified(chat_item, 'chat') chat_item.updated_at = int(time.time()) @@ -1251,7 +1252,6 @@ async def add_message_status_to_chat_by_id_and_message_id( if chat_item is None: return None - self._sanitize_chat_row(chat_item) chat = chat_item.chat or {} self._repair_chat_current_id(chat) history = chat.get('history', {}) @@ -1263,7 +1263,7 @@ async def add_message_status_to_chat_by_id_and_message_id( chat['history'] = history chat_item.chat = chat - chat_item.title = chat.get('title', 'New Chat') + chat_item.title = self._clean_null_bytes(chat.get('title', 'New Chat')) chat_item.current_message_id = self.get_current_message_id(chat) flag_modified(chat_item, 'chat') await session.commit() @@ -2522,6 +2522,8 @@ async def delete_chats_by_user_id(self, user_id: str, db: AsyncSession | None = async def delete_chats_by_user_id_and_folder_id( self, user_id: str, folder_id: str, db: AsyncSession | None = None ) -> bool: + from open_webui.models.shared_chats import SharedChat as SharedChatTable + try: async with get_async_db_context(db) as session: chat_ids_stmt = select(Chat.id).filter_by(user_id=user_id, folder_id=folder_id) @@ -2529,6 +2531,7 @@ async def delete_chats_by_user_id_and_folder_id( update(AutomationRun).filter(AutomationRun.chat_id.in_(chat_ids_stmt)).values(chat_id=None) ) await session.execute(delete(ChatMessage).filter(ChatMessage.chat_id.in_(chat_ids_stmt))) + await session.execute(delete(SharedChatTable).filter(SharedChatTable.chat_id.in_(chat_ids_stmt))) await session.execute(delete(Chat).filter_by(user_id=user_id, folder_id=folder_id)) await session.commit() @@ -2536,18 +2539,15 @@ async def delete_chats_by_user_id_and_folder_id( except Exception: return False - async def move_chats_by_user_id_and_folder_id( + async def move_chats_by_folder_id( self, - user_id: str, folder_id: str, new_folder_id: str | None, db: AsyncSession | None = None, ) -> bool: try: async with get_async_db_context(db) as session: - await session.execute( - update(Chat).filter_by(user_id=user_id, folder_id=folder_id).values(folder_id=new_folder_id) - ) + await session.execute(update(Chat).filter_by(folder_id=folder_id).values(folder_id=new_folder_id)) await session.commit() return True diff --git a/backend/open_webui/models/knowledge.py b/backend/open_webui/models/knowledge.py index 9361da9ad963..6322e1c1613d 100644 --- a/backend/open_webui/models/knowledge.py +++ b/backend/open_webui/models/knowledge.py @@ -625,6 +625,7 @@ async def search_files_by_id( db=db, ), breadcrumbs=await self.get_directory_breadcrumbs( + knowledge_id, filter.get('directory_id') if filter else None, db=db, ), @@ -908,6 +909,7 @@ async def get_directory_by_id( async def get_directory_breadcrumbs( self, + knowledge_id: str, directory_id: Optional[str], db: Optional[AsyncSession] = None, ) -> list[KnowledgeDirectoryModel]: @@ -922,7 +924,10 @@ async def get_directory_breadcrumbs( while current_id and current_id not in seen: seen.add(current_id) - result = await db.execute(select(KnowledgeDirectory).filter_by(id=current_id)) + # Scoped by knowledge base so a caller-supplied id cannot walk another one's tree. + result = await db.execute( + select(KnowledgeDirectory).filter_by(id=current_id, knowledge_id=knowledge_id) + ) directory = result.scalars().first() if not directory: break diff --git a/backend/open_webui/models/models.py b/backend/open_webui/models/models.py index 00a7a2650e4b..d48f1e9008a9 100755 --- a/backend/open_webui/models/models.py +++ b/backend/open_webui/models/models.py @@ -392,7 +392,9 @@ async def search_models( else: meta_text = func.lower(cast(Model.meta, String)) variants = json_text_variants(tag.lower()) - stmt = stmt.filter(or_(*(meta_text.like(f'%"{variant}"%') for variant in variants))) + stmt = stmt.filter( + or_(*(meta_text.contains(f'"{variant}"', autoescape=True) for variant in variants)) + ) order_by = filter.get('order_by') direction = filter.get('direction') @@ -622,16 +624,20 @@ async def sync_models( await db.execute(update(Model).filter_by(id=model.id).values(**model_data)) else: db.add(Model(**model_data)) - await AccessGrants.set_access_grants('model', model.id, model.access_grants, db=db) # Remove models that are no longer present for model in existing_models: if model.id not in new_model_ids: - await AccessGrants.revoke_all_access('model', model.id, db=db) await db.delete(model) await db.commit() + # Grants after the commit to avoid SQLite write-lock contention when session sharing is off + for model in models: + await AccessGrants.set_access_grants('model', model.id, model.access_grants, db=db) + for model_id in existing_ids - new_model_ids: + await AccessGrants.revoke_all_access('model', model_id, db=db) + result = await db.execute(select(Model)) all_models = result.scalars().all() model_ids = [model.id for model in all_models] diff --git a/backend/open_webui/models/prompts.py b/backend/open_webui/models/prompts.py index e6e07eacc79b..75cc93b5a831 100644 --- a/backend/open_webui/models/prompts.py +++ b/backend/open_webui/models/prompts.py @@ -346,7 +346,10 @@ async def search_prompts( # Fallback for dialects with no JSON array function: LIKE on the text. tags_text = func.lower(cast(Prompt.tags, String)) tag_clause = or_( - *(tags_text.like(f'%"{variant}"%') for variant in json_text_variants(tag_lower)) + *( + tags_text.contains(f'"{variant}"', autoescape=True) + for variant in json_text_variants(tag_lower) + ) ) tag_lower = None @@ -506,14 +509,16 @@ async def update_prompt_by_id( ) # Update prompt fields - prompt.name = form_data.name prompt.command = form_data.command - prompt.content = form_data.content - prompt.data = form_data.data or prompt.data - prompt.meta = form_data.meta or prompt.meta - if form_data.tags is not None: - prompt.tags = form_data.tags + if form_data.is_production: + prompt.name = form_data.name + prompt.content = form_data.content + prompt.data = form_data.data or prompt.data + prompt.meta = form_data.meta or prompt.meta + + if form_data.tags is not None: + prompt.tags = form_data.tags if form_data.access_grants is not None: await AccessGrants.set_access_grants('prompt', prompt.id, form_data.access_grants, db=session) @@ -531,7 +536,7 @@ async def update_prompt_by_id( 'command': prompt.command, 'data': form_data.data or {}, 'meta': form_data.meta or {}, - 'tags': prompt.tags or [], + 'tags': form_data.tags if form_data.tags is not None else (prompt.tags or []), 'access_grants': [grant.model_dump() for grant in current_access_grants], } diff --git a/backend/open_webui/retrieval/loaders/main.py b/backend/open_webui/retrieval/loaders/main.py index 74221f1aea68..a3a3d5b7dd6f 100644 --- a/backend/open_webui/retrieval/loaders/main.py +++ b/backend/open_webui/retrieval/loaders/main.py @@ -273,7 +273,7 @@ def load(self) -> list[Document]: f'{self.url}/v1/convert/file', files={ 'files': ( - self.file_path, + os.path.basename(self.file_path), f, self.mime_type or 'application/octet-stream', ) @@ -427,15 +427,17 @@ def _detect_text_encoding(self, file_path: str) -> str: 'gbk': 'gb18030', 'big5': 'big5', 'euckr': 'euc-kr', + 'cp949': 'cp949', 'eucjp': 'euc-jp', 'iso2022jp': 'euc-jp', - 'shiftjis': 'shift_jis', + 'shiftjis': 'cp932', + 'cp932': 'cp932', } # Build priority list: chardet-hinted codec first, then remaining CJK base_order = ['gb18030', 'big5', 'euc-kr', 'euc-jp'] hinted = _ENC_FAMILY.get(detected_enc) - if hinted and hinted in base_order: + if hinted: ordered = [hinted] + [e for e in base_order if e != hinted] else: ordered = base_order @@ -743,7 +745,7 @@ def _get_loader(self, filename: str, file_content_type: str, file_path: str): ) loader = TextLoader(file_path, encoding=self._detect_text_encoding(file_path)) elif file_ext in ['htm', 'html']: - loader = HTMLLoader(file_path, encoding='unicode_escape') + loader = HTMLLoader(file_path, encoding=self._detect_text_encoding(file_path)) elif file_ext == 'md': loader = TextLoader(file_path, encoding=self._detect_text_encoding(file_path)) elif file_content_type == 'application/epub+zip': diff --git a/backend/open_webui/retrieval/utils.py b/backend/open_webui/retrieval/utils.py index 055f75012295..58a4da1f1b82 100644 --- a/backend/open_webui/retrieval/utils.py +++ b/backend/open_webui/retrieval/utils.py @@ -30,6 +30,7 @@ AIOHTTP_CLIENT_SESSION_SSL, AIOHTTP_CLIENT_TIMEOUT, BYPASS_RETRIEVAL_ACCESS_CONTROL, + ENABLE_ADMIN_CHAT_ACCESS, ENABLE_FORWARD_USER_INFO_HEADERS, ENABLE_RETRIEVAL_UNSCOPED_COLLECTIONS, MPS_INFERENCE_LOCK, @@ -838,6 +839,7 @@ async def _fetch_collection(name: str): return name, await ASYNC_VECTOR_DB_CLIENT.get(collection_name=name) except Exception as e: log.exception(f'Failed to fetch collection {name}: {e}') + failed_collection_names.add(name) return name, None collection_results = dict(await asyncio.gather(*(_fetch_collection(name) for name in collection_names))) @@ -1461,7 +1463,9 @@ async def get_sources_from_items( elif item.get('type') == 'chat': # Chat Attached chat = await Chats.get_chat_by_id(item.get('id')) - has_read_access = bool(chat and (user.role == 'admin' or chat.user_id == user.id)) + has_read_access = bool( + chat and ((user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS) or chat.user_id == user.id) + ) if chat and not has_read_access: has_read_access = await AccessGrants.has_access( diff --git a/backend/open_webui/retrieval/vector/dbs/chroma.py b/backend/open_webui/retrieval/vector/dbs/chroma.py index 2c51aeca9374..5c5d9602e8d2 100755 --- a/backend/open_webui/retrieval/vector/dbs/chroma.py +++ b/backend/open_webui/retrieval/vector/dbs/chroma.py @@ -28,6 +28,8 @@ log = logging.getLogger(__name__) +GET_PAGE_SIZE = 10000 + class ChromaClient(VectorDBBase): def __init__(self): @@ -131,12 +133,18 @@ def get(self, collection_name: str) -> Optional[GetResult]: # Get all the items in the collection. collection = self.client.get_collection(name=collection_name, embedding_function=None) if collection: - result = collection.get() + ids, documents, metadatas = [], [], [] + # Unpaged get() exceeds SQLite's bind-variable limit on large collections + for offset in range(0, collection.count(), GET_PAGE_SIZE): + page = collection.get(limit=GET_PAGE_SIZE, offset=offset) + ids.extend(page['ids']) + documents.extend(page['documents']) + metadatas.extend(page['metadatas']) return GetResult( **{ - 'ids': [result['ids']], - 'documents': [result['documents']], - 'metadatas': [result['metadatas']], + 'ids': [ids], + 'documents': [documents], + 'metadatas': [metadatas], } ) return None diff --git a/backend/open_webui/retrieval/vector/dbs/qdrant.py b/backend/open_webui/retrieval/vector/dbs/qdrant.py index f8ed6007739b..d1b302b34f72 100644 --- a/backend/open_webui/retrieval/vector/dbs/qdrant.py +++ b/backend/open_webui/retrieval/vector/dbs/qdrant.py @@ -28,6 +28,7 @@ from qdrant_client.models import models NO_LIMIT = 999999999 +SCROLL_PAGE_SIZE = 1000 log = logging.getLogger(__name__) @@ -92,6 +93,24 @@ def _result_to_get_result(self, points) -> GetResult: } ) + def _scroll_points( + self, collection_name: str, scroll_filter: Optional[models.Filter] = None, limit: Optional[int] = None + ) -> list: + # Paged so a strict-mode max_query_limit does not reject the read + points = [] + offset = None + while True: + page_size = SCROLL_PAGE_SIZE if limit is None else min(SCROLL_PAGE_SIZE, limit - len(points)) + page, offset = self.client.scroll( + collection_name=f'{self.collection_prefix}_{collection_name}', + scroll_filter=scroll_filter, + limit=page_size, + offset=offset, + ) + points.extend(page) + if offset is None or len(points) == limit: + return points + def _create_collection(self, collection_name: str, dimension: int): collection_name_with_prefix = f'{self.collection_prefix}_{collection_name}' self.client.create_collection( @@ -180,32 +199,22 @@ def query(self, collection_name: str, filter: dict, limit: Optional[int] = None) if not self.has_collection(collection_name): return None try: - if limit is None: - limit = NO_LIMIT # otherwise qdrant would set limit to 10! - field_conditions = [] for key, value in filter.items(): field_conditions.append( models.FieldCondition(key=f'metadata.{key}', match=models.MatchValue(value=value)) ) - points = self.client.scroll( - collection_name=f'{self.collection_prefix}_{collection_name}', - scroll_filter=models.Filter(should=field_conditions), - limit=limit, - ) - return self._result_to_get_result(points[0]) + points = self._scroll_points(collection_name, models.Filter(should=field_conditions), limit) + return self._result_to_get_result(points) except Exception as e: log.exception(f"Error querying a collection '{collection_name}': {e}") return None def get(self, collection_name: str) -> Optional[GetResult]: # Get all the items in the collection. - points = self.client.scroll( - collection_name=f'{self.collection_prefix}_{collection_name}', - limit=NO_LIMIT, # otherwise qdrant would set limit to 10! - ) - return self._result_to_get_result(points[0]) + points = self._scroll_points(collection_name) + return self._result_to_get_result(points) def insert(self, collection_name: str, items: list[VectorItem]): # Insert the items into the collection, if the collection does not exist, it will be created. diff --git a/backend/open_webui/retrieval/vector/dbs/qdrant_multitenancy.py b/backend/open_webui/retrieval/vector/dbs/qdrant_multitenancy.py index 870cd2b02f0b..6bf71717e291 100644 --- a/backend/open_webui/retrieval/vector/dbs/qdrant_multitenancy.py +++ b/backend/open_webui/retrieval/vector/dbs/qdrant_multitenancy.py @@ -29,7 +29,7 @@ from qdrant_client.http.models import PointStruct from qdrant_client.models import models -NO_LIMIT = 999999999 +SCROLL_PAGE_SIZE = 1000 TENANT_ID_FIELD = 'tenant_id' DEFAULT_DIMENSION = 384 @@ -97,6 +97,21 @@ def _result_to_get_result(self, points) -> GetResult: metadatas.append(payload['metadata']) return GetResult(ids=[ids], documents=[documents], metadatas=[metadatas]) + def _scroll_points(self, collection_name: str, scroll_filter: models.Filter, limit: Optional[int] = None) -> List: + # Paged so a strict-mode max_query_limit does not reject the read + points, offset = [], None + while True: + page_size = SCROLL_PAGE_SIZE if limit is None else min(SCROLL_PAGE_SIZE, limit - len(points)) + page, offset = self.client.scroll( + collection_name=collection_name, + scroll_filter=scroll_filter, + limit=page_size, + offset=offset, + ) + points.extend(page) + if offset is None or len(points) == limit: + return points + def _get_collection_and_tenant_id(self, collection_name: str) -> Tuple[str, str]: """ Maps the traditional collection name to multi-tenant collection and tenant ID. @@ -287,17 +302,11 @@ def query(self, collection_name: str, filter: Dict[str, Any], limit: Optional[in if not self.client.collection_exists(collection_name=mt_collection): log.debug("Collection %s doesn't exist, query returns None", mt_collection) return None - if limit is None: - limit = NO_LIMIT tenant_filter = _tenant_filter(tenant_id) field_conditions = [_metadata_filter(k, '$eq', v) for k, v in filter.items()] combined_filter = models.Filter(must=[tenant_filter, *field_conditions]) - points = self.client.scroll( - collection_name=mt_collection, - scroll_filter=combined_filter, - limit=limit, - ) - return self._result_to_get_result(points[0]) + points = self._scroll_points(mt_collection, combined_filter, limit) + return self._result_to_get_result(points) def get(self, collection_name: str) -> Optional[GetResult]: """ @@ -310,12 +319,8 @@ def get(self, collection_name: str) -> Optional[GetResult]: log.debug("Collection %s doesn't exist, get returns None", mt_collection) return None tenant_filter = _tenant_filter(tenant_id) - points = self.client.scroll( - collection_name=mt_collection, - scroll_filter=models.Filter(must=[tenant_filter]), - limit=NO_LIMIT, - ) - return self._result_to_get_result(points[0]) + points = self._scroll_points(mt_collection, models.Filter(must=[tenant_filter])) + return self._result_to_get_result(points) def upsert(self, collection_name: str, items: List[VectorItem]): """ diff --git a/backend/open_webui/retrieval/web/searchapi.py b/backend/open_webui/retrieval/web/searchapi.py index bd4c292c1809..0e9f4b02fe33 100644 --- a/backend/open_webui/retrieval/web/searchapi.py +++ b/backend/open_webui/retrieval/web/searchapi.py @@ -26,19 +26,26 @@ def search_searchapi( engine = engine or 'google' payload = {'engine': engine, 'q': query, 'api_key': api_key} + if engine.startswith('google'): + payload['link'] = 'resolved' url = f'{url}?{urlencode(payload)}' - response = requests.request('GET', url) + response = requests.request('GET', url, timeout=30) + response.raise_for_status() json_response = response.json() - log.info('results from searchapi search: %s', json_response) + log.debug('results from searchapi search: %s', json_response) - results = sorted(json_response.get('organic_results', []), key=lambda x: x.get('position', 0)) + # top_stories entries carry no position, so the merged list keeps API order + results = [ + *json_response.get('organic_results', []), + *json_response.get('top_stories', []), + ] if filter_list: results = get_filtered_results(results, filter_list) return [ SearchResult( - link=result['link'], + link=result.get('link', ''), title=result.get('title'), snippet=result.get('snippet'), ) diff --git a/backend/open_webui/retrieval/web/staan.py b/backend/open_webui/retrieval/web/staan.py index 8326a0782c57..c1af2b5b2a83 100644 --- a/backend/open_webui/retrieval/web/staan.py +++ b/backend/open_webui/retrieval/web/staan.py @@ -30,7 +30,7 @@ def search_staan( 'Accept': 'application/json', 'Authorization': f'Bearer {api_key}', } - params = {'q': query, 'market': market, 'count': count} + params = {'q': query, 'market': market} if max_snippets: params['extra_snippets'] = 'true' diff --git a/backend/open_webui/retrieval/web/tavily.py b/backend/open_webui/retrieval/web/tavily.py index 7bf15a1e627f..a0403787b593 100644 --- a/backend/open_webui/retrieval/web/tavily.py +++ b/backend/open_webui/retrieval/web/tavily.py @@ -14,6 +14,7 @@ def search_tavily( query: str, count: int, filter_list: list[str] | None = None, + search_depth: str = 'basic', # **kwargs, ) -> list[SearchResult]: """Search using Tavily's Search API and return the results as a list of SearchResult objects. @@ -22,6 +23,7 @@ def search_tavily( api_key (str): A Tavily Search API key query (str): The query to search for count (int): The maximum number of results to return + search_depth (str): Tavily search depth Returns: A list of SearchResult objects. @@ -31,7 +33,7 @@ def search_tavily( 'Content-Type': 'application/json', 'Authorization': f'Bearer {api_key}', } - data = {'query': query, 'max_results': count} + data = {'query': query, 'max_results': count, 'search_depth': search_depth} response = requests.post(url, headers=headers, json=data) response.raise_for_status() diff --git a/backend/open_webui/routers/analytics.py b/backend/open_webui/routers/analytics.py index d9cf8e0a6139..7d57ab3262ba 100644 --- a/backend/open_webui/routers/analytics.py +++ b/backend/open_webui/routers/analytics.py @@ -207,7 +207,9 @@ async def get_daily_stats( ): """Get message counts grouped by model for time-series chart.""" if granularity == 'hourly': - counts = await ChatMessages.get_hourly_message_counts_by_model(start_date=start_date, end_date=end_date, db=db) + counts = await ChatMessages.get_hourly_message_counts_by_model( + start_date=start_date, end_date=end_date, group_id=group_id, db=db + ) else: counts = await ChatMessages.get_daily_message_counts_by_model( start_date=start_date, end_date=end_date, group_id=group_id, db=db diff --git a/backend/open_webui/routers/auths.py b/backend/open_webui/routers/auths.py index 16ae0422e82b..0ca2f518f840 100644 --- a/backend/open_webui/routers/auths.py +++ b/backend/open_webui/routers/auths.py @@ -9,6 +9,7 @@ import uuid from ssl import CERT_NONE, CERT_REQUIRED, PROTOCOL_TLS +import jwt from aiohttp import BasicAuth, ClientSession from fastapi import APIRouter, Depends, HTTPException, Request, status from fastapi.responses import JSONResponse, Response @@ -75,6 +76,7 @@ verify_password, ) from open_webui.utils.groups import apply_default_group_assignment +from open_webui.utils.json_codec import JSONCodec from open_webui.utils.misc import parse_duration, validate_email_format from open_webui.utils.rate_limit import RateLimiter from pydantic import BaseModel, StrictStr, field_validator @@ -1463,6 +1465,9 @@ class OAuthConfigForm(BaseModel): def _format_oauth_form_value(field: str, value): + if field == 'OAUTH_BLOCKED_GROUPS' and isinstance(value, list): + # Preserve commas in group names and regex patterns when the form is saved. + return JSONCodec.dumps(value) if field in OAUTH_COMMA_LIST_FIELDS and isinstance(value, list): return ','.join(str(item) for item in value) return value @@ -1747,12 +1752,20 @@ async def token_exchange( detail='User not found. Please sign in via the web interface first.', ) + # The provider's userinfo endpoint has already accepted this token. + # Keep an empty dict for opaque tokens so exchange role checks still apply. + token_claims = {} + try: + token_claims = jwt.decode(form_data.token, options={'verify_signature': False}) + except jwt.PyJWTError as e: + log.debug('Token exchange: cannot decode token claims: %s', e) + user = await oauth_manager.update_user_role_from_oauth( request=request, user=user, user_data=user_data, provider=provider, - access_token=form_data.token, + token_claims=token_claims, db=db, ) if await Config.get('oauth.enable_group_mapping'): @@ -1761,6 +1774,7 @@ async def token_exchange( user=user, user_data=user_data, default_permissions=await Config.get('user.permissions'), + token_claims=token_claims, db=db, ) diff --git a/backend/open_webui/routers/automations.py b/backend/open_webui/routers/automations.py index caf8879e2899..4b44a8b8ed21 100644 --- a/backend/open_webui/routers/automations.py +++ b/backend/open_webui/routers/automations.py @@ -87,7 +87,7 @@ async def check_automation_limits(request, user, rrule_str: str, db, is_create: if min_interval: min_interval = int(min_interval) if min_interval > 0: - interval = rrule_interval_seconds(rrule_str) + interval = await rrule_interval_seconds(rrule_str) if interval is not None and interval < min_interval: raise HTTPException( status_code=status.HTTP_400_BAD_REQUEST, @@ -150,7 +150,7 @@ async def enrich_automation(automation: AutomationModel, db: AsyncSession, tz: s return AutomationResponse( **automation.model_dump(), last_run=last_run, - next_runs=next_n_runs_ns(automation.data['rrule'], tz=tz), + next_runs=await next_n_runs_ns(automation.data['rrule'], tz=tz), ) @@ -216,7 +216,7 @@ async def create_new_automation( await check_automation_folder_access(form_data.folder_id, user, db) await check_automation_channel_access(form_data, user, db) try: - validate_rrule(form_data.data.rrule, tz=user.timezone) + await validate_rrule(form_data.data.rrule, tz=user.timezone) except ValueError as e: raise HTTPException( status_code=status.HTTP_400_BAD_REQUEST, @@ -226,7 +226,7 @@ async def create_new_automation( await check_automation_limits(request, user, form_data.data.rrule, db, is_create=True) tz = user.timezone - automation = await Automations.insert(user.id, form_data, next_run_ns(form_data.data.rrule, tz=tz), db=db) + automation = await Automations.insert(user.id, form_data, await next_run_ns(form_data.data.rrule, tz=tz), db=db) response = await enrich_automation(automation, db, tz=tz) await publish_event( request, @@ -276,7 +276,7 @@ async def update_automation_by_id( await check_automation_channel_access(form_data, user, db) try: - validate_rrule(form_data.data.rrule, tz=user.timezone) + await validate_rrule(form_data.data.rrule, tz=user.timezone) except ValueError as e: raise HTTPException( status_code=status.HTTP_400_BAD_REQUEST, @@ -286,7 +286,7 @@ async def update_automation_by_id( await check_automation_limits(request, user, form_data.data.rrule, db, is_create=False) tz = user.timezone - updated = await Automations.update_by_id(id, form_data, next_run_ns(form_data.data.rrule, tz=tz), db=db) + updated = await Automations.update_by_id(id, form_data, await next_run_ns(form_data.data.rrule, tz=tz), db=db) response = await enrich_automation(updated, db, tz=tz) await publish_event( request, @@ -313,7 +313,7 @@ async def toggle_automation_by_id( await check_automations_permission(request, user) automation = await Automations.get_by_id(id, db=db) check_automation_access(automation, user) - toggled = await Automations.toggle(id, next_run_ns(automation.data['rrule'], tz=user.timezone), db=db) + toggled = await Automations.toggle(id, await next_run_ns(automation.data['rrule'], tz=user.timezone), db=db) response = await enrich_automation(toggled, db, tz=user.timezone) await publish_event( request, diff --git a/backend/open_webui/routers/calendar.py b/backend/open_webui/routers/calendar.py index f95be48e8401..86d982480322 100644 --- a/backend/open_webui/routers/calendar.py +++ b/backend/open_webui/routers/calendar.py @@ -273,7 +273,10 @@ async def get_events( async def create_event(request: Request, form_data: CalendarEventForm, user: UserModel = Depends(get_verified_user)): await check_calendar_permission(request, user) await _check_calendar_access(form_data.calendar_id, user, 'write') - event = await CalendarEvents.insert_new_event(user.id, form_data) + try: + event = await CalendarEvents.insert_new_event(user.id, form_data) + except ValueError as e: + raise HTTPException(status_code=422, detail=str(e)) from e await publish_event( request, EVENTS.CALENDAR_EVENT_CREATED, @@ -325,7 +328,10 @@ async def update_event( if form_data.calendar_id is not None and form_data.calendar_id != event.calendar_id: await _check_calendar_access(form_data.calendar_id, user, 'write') - updated = await CalendarEvents.update_event_by_id(event_id, form_data) + try: + updated = await CalendarEvents.update_event_by_id(event_id, form_data) + except ValueError as e: + raise HTTPException(status_code=422, detail=str(e)) from e if not updated: raise HTTPException(status_code=500, detail='Failed to update') await publish_event( diff --git a/backend/open_webui/routers/channels.py b/backend/open_webui/routers/channels.py index de25739c92ca..c9d38bf2ced3 100644 --- a/backend/open_webui/routers/channels.py +++ b/backend/open_webui/routers/channels.py @@ -40,6 +40,7 @@ emit_to_users, enter_room_for_users, get_user_ids_from_room, + leave_room_for_users, sio, ) from open_webui.utils.access_control import filter_allowed_access_grants, has_permission @@ -687,6 +688,8 @@ async def remove_members_by_id( try: deleted = await Channels.remove_members_from_channel(channel.id, form_data.user_ids, db=db) + if channel.type in ['group', 'dm']: + await leave_room_for_users(f'channel:{channel.id}', form_data.user_ids) await publish_event( request, @@ -731,8 +734,18 @@ async def update_channel_by_id( 'sharing.public_channels', ) + previous_access_grants = channel.access_grants + try: channel = await Channels.update_channel_by_id(id, form_data, db=db) + # Group and DM channels use membership instead of access grants. + if form_data.access_grants is not None and channel.type not in ['group', 'dm']: + revoked_user_ids = await AccessGrants.get_revoked_user_ids_by_resource( + 'channel', id, previous_access_grants, db=db + ) + revoked_user_ids.discard(channel.user_id) + await leave_room_for_users(f'channel:{id}', list(revoked_user_ids)) + await publish_event( request, EVENTS.CHANNEL_UPDATED, @@ -769,6 +782,7 @@ async def delete_channel_by_id( try: await Channels.delete_channel_by_id(id, db=db) + await sio.close_room(f'channel:{id}') await publish_event( request, EVENTS.CHANNEL_DELETED, diff --git a/backend/open_webui/routers/chats.py b/backend/open_webui/routers/chats.py index ed339a601aa7..e6194d0f6448 100644 --- a/backend/open_webui/routers/chats.py +++ b/backend/open_webui/routers/chats.py @@ -680,7 +680,7 @@ async def export_single_chat_stats( ) # Verify the chat belongs to the user (unless admin) - if chat.user_id != user.id and user.role != 'admin': + if chat.user_id != user.id and not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS): raise HTTPException( status_code=status.HTTP_401_UNAUTHORIZED, detail=ERROR_MESSAGES.ACCESS_PROHIBITED, @@ -716,8 +716,10 @@ async def delete_all_user_chats( detail=ERROR_MESSAGES.ACCESS_PROHIBITED, ) + tag_ids = [tag.id for tag in await Tags.get_tags_by_user_id(user.id, db=db)] result = await Chats.delete_chats_by_user_id(user.id, db=db) if result: + await Chats.delete_orphan_tags_for_user(tag_ids, user.id, db=db) await publish_event( request, EVENTS.CHAT_DELETED_ALL, @@ -1129,8 +1131,14 @@ async def archive_all_chats( async def unarchive_all_chats( request: Request, user=Depends(get_verified_user), db: AsyncSession = Depends(get_async_session) ): + tag_ids = { + tag_id + for chat in await Chats.get_archived_chats_by_user_id(user.id, db=db) + for tag_id in chat.meta.get('tags', []) + } result = await Chats.unarchive_all_chats_by_user_id(user.id, db=db) if result: + await Tags.ensure_tags_exist(list(tag_ids), user.id, db=db) await publish_event(request, EVENTS.CHAT_UNARCHIVED, actor=user, subject_id=user.id, subject_type='user') return result @@ -1420,7 +1428,7 @@ async def update_chat_message_by_id( detail=ERROR_MESSAGES.ACCESS_PROHIBITED, ) - if chat.user_id != user.id and user.role != 'admin': + if chat.user_id != user.id and not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS): raise HTTPException( status_code=status.HTTP_401_UNAUTHORIZED, detail=ERROR_MESSAGES.ACCESS_PROHIBITED, @@ -1481,7 +1489,7 @@ async def delete_chat_message_by_id( detail=ERROR_MESSAGES.ACCESS_PROHIBITED, ) - if chat.user_id != user.id and user.role != 'admin': + if chat.user_id != user.id and not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS): raise HTTPException( status_code=status.HTTP_401_UNAUTHORIZED, detail=ERROR_MESSAGES.ACCESS_PROHIBITED, @@ -1529,7 +1537,7 @@ async def send_chat_message_event_by_id( detail=ERROR_MESSAGES.ACCESS_PROHIBITED, ) - if chat.user_id != user.id and user.role != 'admin': + if chat.user_id != user.id and not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS): raise HTTPException( status_code=status.HTTP_401_UNAUTHORIZED, detail=ERROR_MESSAGES.ACCESS_PROHIBITED, @@ -1576,6 +1584,8 @@ async def delete_chat_by_id( # not be reachable for a chat the caller may not delete. if user.role == 'admin': chat = await Chats.get_chat_by_id(id, db=db) + if chat and chat.user_id != user.id and not ENABLE_ADMIN_CHAT_ACCESS: + chat = None else: if not await has_permission(user.id, 'chat.delete', await Config.get('user.permissions')): raise HTTPException( @@ -1840,21 +1850,9 @@ async def clone_shared_chat_by_id( ): await require_chat_import_permission(request, user, db) - chat = await Chats.get_chat_by_share_id(id, db=db) - - # Fallback: admins can also access any chat directly by chat ID - if not chat and user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS: - chat = await Chats.get_chat_by_id(id, db=db) - - if not chat: - raise HTTPException( - status_code=status.HTTP_401_UNAUTHORIZED, - detail=ERROR_MESSAGES.NOT_FOUND, - ) - # Enforce access grants (owner and admins bypass) shared = await SharedChats.get_by_id(id, db=db) - if shared and user.role != 'admin' and shared.user_id != user.id: + if shared and not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS) and shared.user_id != user.id: has_grant = await is_open_shared_chat(shared, db=db) or await AccessGrants.has_access( user_id=user.id, resource_type='shared_chat', @@ -1868,6 +1866,18 @@ async def clone_shared_chat_by_id( detail=ERROR_MESSAGES.ACCESS_PROHIBITED, ) + chat = await Chats.get_chat_by_share_id(id, db=db) if shared else None + + # Fallback: admins can also access any chat directly by chat ID + if not chat and user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS: + chat = await Chats.get_chat_by_id(id, db=db) + + if not chat: + raise HTTPException( + status_code=status.HTTP_401_UNAUTHORIZED, + detail=ERROR_MESSAGES.NOT_FOUND, + ) + updated_chat = { **chat.chat, 'originalChatId': chat.id, @@ -2034,7 +2044,7 @@ async def update_shared_chat_access_by_id( user=Depends(get_verified_user), db: AsyncSession = Depends(get_async_session), ): - if user.role == 'admin': + if user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS: chat = await Chats.get_chat_by_id(id, db=db) else: chat = await Chats.get_chat_by_id_and_user_id(id, user.id, db=db) @@ -2070,7 +2080,7 @@ async def get_shared_chat_access_by_id( user=Depends(get_verified_user), db: AsyncSession = Depends(get_async_session), ): - if user.role == 'admin': + if user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS: chat = await Chats.get_chat_by_id(id, db=db) else: chat = await Chats.get_chat_by_id_and_user_id(id, user.id, db=db) diff --git a/backend/open_webui/routers/configs.py b/backend/open_webui/routers/configs.py index 38227dd2fc90..35f1b23cf467 100644 --- a/backend/open_webui/routers/configs.py +++ b/backend/open_webui/routers/configs.py @@ -22,7 +22,7 @@ get_discovery_urls, get_oauth_client_info_with_dynamic_client_registration, get_oauth_client_info_with_static_credentials, - recover_static_oauth_client_metadata, + recover_oauth_client_metadata, resolve_oauth_client_info, ) from open_webui.utils.tools import ( @@ -273,7 +273,7 @@ async def set_tool_servers_config( if auth_type in ('oauth_2.1', 'oauth_2.1_static') and server_id: try: oauth_client_info = resolve_oauth_client_info(connection) - oauth_client_info = await recover_static_oauth_client_metadata(connection, oauth_client_info) + oauth_client_info = await recover_oauth_client_metadata(connection, oauth_client_info) oauth_client_info = apply_connection_oauth_options(connection, oauth_client_info) request.app.state.oauth_client_manager.add_client( f'{server_type}:{server_id}', @@ -810,7 +810,7 @@ class PromptSuggestion(BaseModel): class SetDefaultSuggestionsForm(BaseModel): - suggestions: list[PromptSuggestion] + suggestions: list[PromptSuggestion] | None i18n: dict[str, Any] | None = None diff --git a/backend/open_webui/routers/evaluations.py b/backend/open_webui/routers/evaluations.py index 00c1517c88eb..9dd8413077d4 100644 --- a/backend/open_webui/routers/evaluations.py +++ b/backend/open_webui/routers/evaluations.py @@ -265,7 +265,7 @@ async def get_leaderboard( return LeaderboardResponse(entries=entries) -@router.get('/leaderboard/{model_id}/history', response_model=ModelHistoryResponse) +@router.get('/leaderboard/{model_id:path}/history', response_model=ModelHistoryResponse) async def get_model_history( model_id: str, days: int = 30, diff --git a/backend/open_webui/routers/files.py b/backend/open_webui/routers/files.py index c2a429a3db3e..650249849f06 100644 --- a/backend/open_webui/routers/files.py +++ b/backend/open_webui/routers/files.py @@ -224,6 +224,15 @@ async def _process_handler(db_session): f'{knowledge_id}: user {user.id} lacks write access' ) else: + directory_id = file_metadata.get('directory_id') or None + if directory_id: + directory = await Knowledges.get_directory_by_id(directory_id, db=db_session) + if not directory or directory.knowledge_id != knowledge_id: + log.warning( + 'Ignoring directory %s: not a directory of knowledge %s', directory_id, knowledge_id + ) + directory_id = None + # Keep the generic file status stream open until the # KB-specific vector write and durable link both finish. await Files.update_file_data_by_id(file_item.id, {'status': 'processing'}, db=db_session) @@ -237,7 +246,7 @@ async def _process_handler(db_session): knowledge_id=knowledge_id, file_id=file_item.id, user_id=user.id, - directory_id=file_metadata.get('directory_id'), + directory_id=directory_id, db=db_session, ) if not knowledge_file: diff --git a/backend/open_webui/routers/folders.py b/backend/open_webui/routers/folders.py index 9653a6d9b830..cb8c9953cf42 100644 --- a/backend/open_webui/routers/folders.py +++ b/backend/open_webui/routers/folders.py @@ -162,6 +162,16 @@ async def create_folder( detail=ERROR_MESSAGES.DEFAULT('Folder already exists'), ) + if ( + form_data.data + and 'files' in form_data.data + and not await can_read_all_folder_files(form_data.data['files'], user, db=db) + ): + raise HTTPException( + status_code=status.HTTP_403_FORBIDDEN, + detail=ERROR_MESSAGES.ACCESS_PROHIBITED, + ) + # Check if creating a subfolder in a shared folder if form_data.parent_id: parent = await Folders.get_folder_by_id(form_data.parent_id, db=db) @@ -202,16 +212,6 @@ async def create_folder( detail=ERROR_MESSAGES.DEFAULT('Error creating folder'), ) - if ( - form_data.data - and 'files' in form_data.data - and not await can_read_all_folder_files(form_data.data['files'], user, db=db) - ): - raise HTTPException( - status_code=status.HTTP_403_FORBIDDEN, - detail=ERROR_MESSAGES.ACCESS_PROHIBITED, - ) - try: folder = await Folders.insert_new_folder(user.id, form_data, form_data.parent_id, db=db) await publish_event( @@ -287,7 +287,12 @@ async def get_shared_folders( ############################ -@router.get('/{id}', response_model=None) +class FolderResponse(FolderModel): + access_grants: list[dict] = [] + write_access: bool = False + + +@router.get('/{id}', response_model=FolderResponse) async def get_folder_by_id( request: Request, id: str, user=Depends(get_verified_user), db: AsyncSession = Depends(get_async_session) ): @@ -295,13 +300,21 @@ async def get_folder_by_id( folder = await Folders.get_folder_by_id_and_user_id(id, user.id, db=db) if folder: grants = await AccessGrants.get_grants_by_resource('folder', id, db=db) - return {**folder.model_dump(), 'access_grants': [g.model_dump() for g in grants]} + return FolderResponse( + **folder.model_dump(), + access_grants=[g.model_dump() for g in grants], + write_access=True, + ) # Check shared access folder = await Folders.get_folder_by_id(id, db=db) if folder and (user.role == 'admin' or await _has_folder_access(user.id, folder, 'read', db)): grants = await AccessGrants.get_grants_by_resource('folder', id, db=db) - return {**folder.model_dump(), 'access_grants': [g.model_dump() for g in grants]} + return FolderResponse( + **folder.model_dump(), + access_grants=[g.model_dump() for g in grants], + write_access=user.role == 'admin' or await _has_folder_access(user.id, folder, 'write', db), + ) raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, @@ -358,6 +371,15 @@ async def update_folder_name_by_id( detail=ERROR_MESSAGES.ACCESS_PROHIBITED, ) + # Editors send back the owner's existing entries, so only new ones are checked against the editor. + existing_files = (folder.data or {}).get('files') or [] + added_files = [entry for entry in form_data.data['files'] or [] if entry not in existing_files] + if not await can_read_all_folder_files(added_files, user, db=db): + raise HTTPException( + status_code=status.HTTP_403_FORBIDDEN, + detail=ERROR_MESSAGES.ACCESS_PROHIBITED, + ) + try: folder = await Folders.update_folder_by_id_and_user_id(id, folder.user_id, form_data, db=db) await publish_event( @@ -704,8 +726,8 @@ async def delete_folder_by_id( for folder_id in folder_ids: if delete_contents: await Chats.delete_chats_by_user_id_and_folder_id(folder_owner_id, folder_id, db=db) - else: - await Chats.move_chats_by_user_id_and_folder_id(folder_owner_id, folder_id, None, db=db) + + await Chats.move_chats_by_folder_id(folder_id, None, db=db) # Clean up access grants for this folder await AccessGrants.revoke_all_access('folder', folder_id, db=db) diff --git a/backend/open_webui/routers/images.py b/backend/open_webui/routers/images.py index f7ea78962297..c894f2c65148 100644 --- a/backend/open_webui/routers/images.py +++ b/backend/open_webui/routers/images.py @@ -509,8 +509,9 @@ async def get_image_data(data: str, headers=None, trusted_base_url: str | None = mime_type = header.split(';')[0].lstrip('data:') img_data = base64.b64decode(encoded) else: - mime_type = 'image/png' img_data = base64.b64decode(data) + with Image.open(io.BytesIO(img_data)) as image: + mime_type = Image.MIME.get(image.format, 'image/png') return img_data, mime_type except Exception as e: log.exception(f'Error loading image data: {e}') @@ -520,7 +521,7 @@ async def get_image_data(data: str, headers=None, trusted_base_url: str | None = async def upload_image(request, image_data, content_type, metadata, user, db=None): if image_data is None or content_type is None: raise ValueError('Failed to retrieve image data from the generation backend') - image_format = mimetypes.guess_extension(content_type) + image_format = IMAGE_FILE_EXTENSIONS.get(content_type.lower()) or mimetypes.guess_extension(content_type) or '.png' file = UploadFile( file=io.BytesIO(image_data), filename=f'generated-image{image_format}', # will be converted to a unique ID on upload_file @@ -667,7 +668,9 @@ async def image_generations( if image_url := image.get('url', None): image_data, content_type = await get_image_data( image_url, - {k: v for k, v in headers.items() if k != 'Content-Type'}, + {k: v for k, v in headers.items() if k != 'Content-Type'} + if _is_same_origin(image_url, image_config.IMAGES_OPENAI_API_BASE_URL) + else None, ) else: image_data, content_type = await get_image_data(image['b64_json']) @@ -918,11 +921,8 @@ async def load_url_image(data): if data.startswith('http://') or data.startswith('https://'): parsed = urlparse(data) - if ( - parsed.netloc == urlparse(str(request.base_url)).netloc - and parsed.path.startswith('/api/v1/files/') - and '/content' in parsed.path - ): + # Fetching /api/v1/files/{id}/content over the network would be unauthenticated. + if parsed.path.startswith('/api/v1/files/') and '/content' in parsed.path: return await load_url_image(parsed.path) # Validate URL to prevent SSRF attacks against local/private networks. @@ -1046,7 +1046,9 @@ async def load_url_image(data): if image_url := image.get('url', None): image_data, content_type = await get_image_data( image_url, - {k: v for k, v in headers.items() if k != 'Content-Type'}, + {k: v for k, v in headers.items() if k != 'Content-Type'} + if _is_same_origin(image_url, image_config.IMAGES_EDIT_OPENAI_API_BASE_URL) + else None, ) else: image_data, content_type = await get_image_data(image['b64_json']) diff --git a/backend/open_webui/routers/knowledge.py b/backend/open_webui/routers/knowledge.py index 43ec07a53c2b..84008db22155 100644 --- a/backend/open_webui/routers/knowledge.py +++ b/backend/open_webui/routers/knowledge.py @@ -151,6 +151,24 @@ def external_knowledge_error(): ) +async def _verify_directory_in_knowledge( + id: str, + directory_id: str | None, + db: AsyncSession, + detail: str = ERROR_MESSAGES.NOT_FOUND, +): + """Verify a caller-supplied directory belongs to the knowledge base in the URL. Unset means the root level.""" + if not directory_id: + return None + + directory = await Knowledges.get_directory_by_id(directory_id, db=db) + if not directory or directory.knowledge_id != id: + raise HTTPException( + status_code=status.HTTP_404_NOT_FOUND, + detail=detail, + ) + + @router.get('/', response_model=KnowledgeAccessListResponse) async def get_knowledge_bases( page: int | None = 1, @@ -1437,6 +1455,8 @@ async def add_file_to_knowledge_by_id( detail=ERROR_MESSAGES.ACCESS_PROHIBITED, ) + await _verify_directory_in_knowledge(id, form_data.directory_id, db, detail='Target directory not found.') + file = await Files.get_file_by_id(form_data.file_id, db=db) if not file: raise HTTPException( @@ -2049,6 +2069,9 @@ async def add_files_to_knowledge_batch( detail=ERROR_MESSAGES.ACCESS_PROHIBITED, ) + for directory_id in {form.directory_id for form in form_data if form.directory_id}: + await _verify_directory_in_knowledge(id, directory_id, db, detail='Target directory not found.') + # Batch-fetch all files to avoid N+1 queries log.info('files/batch/add - %s files', len(form_data)) file_ids = [form.file_id for form in form_data] @@ -2237,6 +2260,8 @@ async def create_knowledge_directory( ): await _verify_knowledge_write_access(id, user, db) + await _verify_directory_in_knowledge(id, form_data.parent_id, db, detail='Parent directory not found.') + directory = await Knowledges.create_directory( knowledge_id=id, name=form_data.name, @@ -2269,14 +2294,11 @@ async def update_knowledge_directory( db: AsyncSession = Depends(get_async_session), ): await _verify_knowledge_write_access(id, user, db) + await _verify_directory_in_knowledge(id, dir_id, db) - # Verify directory belongs to this knowledge base - directory = await Knowledges.get_directory_by_id(dir_id, db=db) - if not directory or directory.knowledge_id != id: - raise HTTPException( - status_code=status.HTTP_404_NOT_FOUND, - detail=ERROR_MESSAGES.NOT_FOUND, - ) + # '__unset__' leaves the parent alone, None moves the directory to the root + if form_data.parent_id not in (None, '__unset__'): + await _verify_directory_in_knowledge(id, form_data.parent_id, db, detail='Parent directory not found.') result = await Knowledges.update_directory( directory_id=dir_id, @@ -2309,14 +2331,7 @@ async def delete_knowledge_directory( db: AsyncSession = Depends(get_async_session), ): await _verify_knowledge_write_access(id, user, db) - - # Verify directory belongs to this knowledge base - directory = await Knowledges.get_directory_by_id(dir_id, db=db) - if not directory or directory.knowledge_id != id: - raise HTTPException( - status_code=status.HTTP_404_NOT_FOUND, - detail=ERROR_MESSAGES.NOT_FOUND, - ) + await _verify_directory_in_knowledge(id, dir_id, db) # Collect before delete_directory drops the KnowledgeFile rows files = [] if move_files else await Knowledges.get_files_by_id_and_directory_id(id, dir_id, db=db) @@ -2375,14 +2390,7 @@ async def move_file_in_knowledge( detail=ERROR_MESSAGES.NOT_FOUND, ) - # If target directory is set, verify it belongs to this knowledge base - if form_data.directory_id: - directory = await Knowledges.get_directory_by_id(form_data.directory_id, db=db) - if not directory or directory.knowledge_id != id: - raise HTTPException( - status_code=status.HTTP_404_NOT_FOUND, - detail='Target directory not found.', - ) + await _verify_directory_in_knowledge(id, form_data.directory_id, db, detail='Target directory not found.') success = await Knowledges.move_file_to_directory( knowledge_id=id, diff --git a/backend/open_webui/routers/models.py b/backend/open_webui/routers/models.py index f9437b5b87b1..e8426b1dae03 100644 --- a/backend/open_webui/routers/models.py +++ b/backend/open_webui/routers/models.py @@ -27,7 +27,7 @@ ) from open_webui.events import EVENTS, publish_event from open_webui.internal.db import get_async_session -from open_webui.models.access_grants import AccessGrants +from open_webui.models.access_grants import AccessGrants, normalize_access_grants from open_webui.models.config import Config from open_webui.models.files import Files from open_webui.models.groups import Groups @@ -62,6 +62,8 @@ def add_chat_variables_schema(model_dict: dict) -> dict: schema = get_chat_variables_schema(system) if schema: model_dict.setdefault('meta', {})['chat_variables_schema'] = schema + elif isinstance(model_dict.get('meta'), dict): + model_dict['meta'].pop('chat_variables_schema', None) return model_dict @@ -983,13 +985,29 @@ async def update_model_by_id( form_data.meta.background_image_url = model.meta.background_image_url await _verify_background_image(form_data.meta.background_image_url, user, db, model.meta.background_image_url) - form_data.access_grants = await filter_allowed_access_grants( - await Config.get('user.permissions'), - user.id, - user.role, - form_data.access_grants, - 'sharing.public_models', - ) + if form_data.access_grants is not None: + # The editor resends every stored grant, so re-checking them would strip sharing this user cannot re-create. + existing_access_grants = { + (grant.principal_type, grant.principal_id, grant.permission) for grant in model.access_grants + } + submitted_access_grants_map = { + (grant['principal_type'], grant['principal_id'], grant['permission']): grant + for grant in normalize_access_grants(form_data.access_grants) + } + preserved_access_grants = [ + grant for key, grant in submitted_access_grants_map.items() if key in existing_access_grants + ] + new_access_grants = [ + grant for key, grant in submitted_access_grants_map.items() if key not in existing_access_grants + ] + + form_data.access_grants = preserved_access_grants + await filter_allowed_access_grants( + await Config.get('user.permissions'), + user.id, + user.role, + new_access_grants, + 'sharing.public_models', + ) model = await Models.update_model_by_id(form_data.id, ModelForm(**form_data.model_dump()), db=db) if model: diff --git a/backend/open_webui/routers/notes.py b/backend/open_webui/routers/notes.py index a84dd3f94eea..e44a311f4582 100644 --- a/backend/open_webui/routers/notes.py +++ b/backend/open_webui/routers/notes.py @@ -11,7 +11,7 @@ from open_webui.constants import ERROR_MESSAGES from open_webui.events import EVENTS, publish_event from open_webui.internal.db import get_async_session -from open_webui.models.access_grants import AccessGrants +from open_webui.models.access_grants import AccessGrantModel, AccessGrants from open_webui.models.chats import ChatForm, ChatResponse, Chats from open_webui.models.config import Config from open_webui.models.groups import Groups @@ -23,7 +23,7 @@ NoteUserResponse, ) from open_webui.models.users import UserResponse, Users -from open_webui.socket.main import sio +from open_webui.socket.main import leave_room_for_users, sio from open_webui.utils.access_control import ( filter_allowed_access_grants, has_permission, @@ -46,6 +46,23 @@ def _truncate_note_data(data: Optional[dict], max_length: int = 1000) -> Optiona return {'content': {'md': md[:max_length]}} +async def leave_note_rooms_for_revoked_users( + note: NoteModel, previous_access_grants: list[AccessGrantModel], db: AsyncSession | None = None +): + revoked_user_ids = await AccessGrants.get_revoked_user_ids_by_resource( + 'note', note.id, previous_access_grants, db=db + ) + revoked_user_ids.discard(note.user_id) + if not revoked_user_ids: + return + + users = await Users.get_users_by_user_ids(list(revoked_user_ids), db=db) + # Admins retain access to notes regardless of grants. + user_ids = [user.id for user in users if user.role != 'admin'] + for room in [f'note:{note.id}', f'doc_note:{note.id}']: + await leave_room_for_users(room, user_ids) + + ############################ # GetNotes ############################ @@ -564,8 +581,13 @@ async def update_note_by_id( db=db, ) + previous_access_grants = note.access_grants + try: note = await Notes.update_note_by_id(id, form_data, db=db) + if form_data.access_grants is not None: + await leave_note_rooms_for_revoked_users(note, previous_access_grants, db=db) + pinned_note_ids = await Notes.get_pinned_note_ids(user.id, db=db) note.is_pinned = note.id in pinned_note_ids @@ -644,6 +666,7 @@ async def update_note_access_by_id( ) await AccessGrants.set_access_grants('note', id, form_data.access_grants, db=db) + await leave_note_rooms_for_revoked_users(note, note.access_grants, db=db) note = await Notes.get_note_by_id(id, db=db) pinned_note_ids = await Notes.get_pinned_note_ids(user.id, db=db) @@ -744,6 +767,9 @@ async def delete_note_by_id( try: note = await Notes.delete_note_by_id(id, db=db) + for room in [f'note:{id}', f'doc_note:{id}']: + await sio.close_room(room) + await publish_event( request, EVENTS.NOTE_DELETED, diff --git a/backend/open_webui/routers/ollama.py b/backend/open_webui/routers/ollama.py index b283db9ae822..d354986d37cd 100644 --- a/backend/open_webui/routers/ollama.py +++ b/backend/open_webui/routers/ollama.py @@ -22,8 +22,6 @@ AIOHTTP_CLIENT_TIMEOUT_MODEL_LIST, AIOHTTP_FILE_STREAM_CHUNK_SIZE, BYPASS_MODEL_ACCESS_CONTROL, - ENABLE_FORWARD_USER_INFO_HEADERS, - FORWARD_SESSION_INFO_HEADER_CHAT_ID, MODELS_CACHE_TTL, REDIS_KEY_PREFIX, ) @@ -36,7 +34,7 @@ from open_webui.models.users import UserModel from open_webui.utils.access_control import check_model_access from open_webui.utils.auth import get_admin_user, get_verified_user -from open_webui.utils.headers import get_custom_headers, include_user_info_headers +from open_webui.utils.headers import get_headers_and_cookies from open_webui.utils.json_codec import JSONCodec from open_webui.utils.misc import calculate_sha256 from open_webui.utils.model_ids import strip_provider_model_prefix @@ -67,24 +65,21 @@ def _clean_proxy_headers(raw_headers) -> dict: async def send_get_request( - url: str, - key: str | None = None, - user: UserModel | None = None, + request: Request = None, + url=None, + key=None, + user: UserModel = None, + config=None, ): """Issue a GET request to an Ollama backend and return JSON, or *None* on failure.""" try: session = await get_session() - headers: dict = { - 'Content-Type': 'application/json', - } - if key: - headers['Authorization'] = f'Bearer {key}' - if ENABLE_FORWARD_USER_INFO_HEADERS and user: - headers = include_user_info_headers(headers, user) + headers, cookies = await get_headers_and_cookies(request, url, key, config, user=user) async with session.get( url, headers=headers, + cookies=cookies, ssl=AIOHTTP_CLIENT_SESSION_SSL, timeout=_MODEL_LIST_TIMEOUT, ) as r: @@ -114,25 +109,14 @@ async def send_request( try: session = await get_session() - headers = { - 'Content-Type': 'application/json', - **({'Authorization': f'Bearer {key}'} if key else {}), - } - - if ENABLE_FORWARD_USER_INFO_HEADERS and user: - headers = include_user_info_headers(headers, user, request=request) - if metadata and metadata.get('chat_id'): - headers[FORWARD_SESSION_INFO_HEADER_CHAT_ID] = metadata.get('chat_id') - - # Custom per-connection headers last so admin-set headers take precedence. - if api_config and api_config.get('headers'): - headers.update(await get_custom_headers(api_config['headers'], user, metadata, request=request)) + headers, cookies = await get_headers_and_cookies(request, url, key, api_config, metadata, user=user) r = await session.request( method, url, data=payload, headers=headers, + cookies=cookies, ssl=AIOHTTP_CLIENT_SESSION_SSL, timeout=get_client_timeout(stream=stream), ) @@ -249,24 +233,26 @@ class ConnectionVerificationForm(BaseModel): url: str key: str | None = None + config: dict | None = None + @router.post('/verify') async def verify_connection( + request: Request, form_data: ConnectionVerificationForm, user=Depends(get_admin_user), ): """Verify that an Ollama backend at *form_data.url* is reachable.""" try: session = await get_session() - headers: dict = {} - if form_data.key: - headers['Authorization'] = f'Bearer {form_data.key}' - if ENABLE_FORWARD_USER_INFO_HEADERS and user: - headers = include_user_info_headers(headers, user) + headers, cookies = await get_headers_and_cookies( + request, form_data.url, form_data.key, form_data.config, user=user + ) async with session.get( f'{form_data.url}/api/version', headers=headers, + cookies=cookies, ssl=AIOHTTP_CLIENT_SESSION_SSL, timeout=_MODEL_LIST_TIMEOUT, ) as r: @@ -403,9 +389,11 @@ async def get_all_models(request: Request, user: UserModel | None = None): for idx, url in enumerate(base_urls): api_config = resolve_api_config(api_configs, idx, url) if not api_config: - tasks.append(send_get_request(f'{url}/api/tags', user=user)) + tasks.append(send_get_request(request, f'{url}/api/tags', user=user)) elif api_config.get('enable', True): - tasks.append(send_get_request(f'{url}/api/tags', api_config.get('key'), user=user)) + tasks.append( + send_get_request(request, f'{url}/api/tags', api_config.get('key'), user=user, config=api_config) + ) else: tasks.append(asyncio.ensure_future(asyncio.sleep(0, None))) @@ -495,9 +483,15 @@ async def get_ollama_tags( if url_idx is None: result = await get_all_models(request, user=user) else: - url = (await Config.get('ollama.base_urls', []))[url_idx] - key = get_api_key(url_idx, url, (await Config.get('ollama.api_configs', {}))) - result = await send_request(f'{url}/api/tags', 'GET', key=key, user=user) + url, api_config, key = await get_ollama_connection(url_idx) + result = await send_request( + f'{url}/api/tags', + 'GET', + key=key, + user=user, + api_config=api_config, + request=request, + ) if user.role == 'user' and not BYPASS_MODEL_ACCESS_CONTROL: result['models'] = await get_filtered_models(result, user) @@ -524,9 +518,11 @@ async def get_ollama_loaded_models( continue api_config = resolve_api_config(api_configs, idx, url) if not api_config: - tasks.append(send_get_request(f'{url}/api/ps', user=user)) + tasks.append(send_get_request(request, f'{url}/api/ps', user=user)) elif api_config.get('enable', True): - tasks.append(send_get_request(f'{url}/api/ps', api_config.get('key'), user=user)) + tasks.append( + send_get_request(request, f'{url}/api/ps', api_config.get('key'), user=user, config=api_config) + ) else: tasks.append(asyncio.ensure_future(asyncio.sleep(0, None))) @@ -559,8 +555,15 @@ async def get_ollama_versions( return {'version': False} if url_idx is not None: - url = (await Config.get('ollama.base_urls', []))[url_idx] - return await send_request(f'{url}/api/version', 'GET') + url, api_config, key = await get_ollama_connection(url_idx) + return await send_request( + f'{url}/api/version', + 'GET', + key=key, + user=user, + api_config=api_config, + request=request, + ) # Fan-out to every enabled backend tasks = [] @@ -570,7 +573,9 @@ async def get_ollama_versions( (await Config.get('ollama.api_configs', {})).get(url, {}), ) if api_config.get('enable', True): - tasks.append(send_get_request(f'{url}/api/version', api_config.get('key'))) + tasks.append( + send_get_request(request, f'{url}/api/version', api_config.get('key'), user=user, config=api_config) + ) raw = await asyncio.gather(*tasks) valid = [r for r in raw if r is not None] @@ -634,6 +639,8 @@ async def unload_model( payload=JSONCodec.dumps(payload), key=key, user=user, + api_config=api_config, + request=request, ) results.append({'url_idx': idx, 'success': True, 'response': res}) except Exception as e: @@ -663,17 +670,19 @@ async def pull_model( form_data = form_data.model_dump(exclude_none=True) form_data['model'] = form_data.get('model', form_data.get('name')) - url = (await Config.get('ollama.base_urls', []))[url_idx] + url, api_config, key = await get_ollama_connection(url_idx) log.info('url: %s', url) # Admins may pull from any registry return await send_request( f'{url}/api/pull', payload=JSONCodec.dumps({**form_data, 'insecure': True}), - key=get_api_key(url_idx, url, (await Config.get('ollama.api_configs', {}))), + key=key, user=user, stream=True, passthrough=True, + api_config=api_config, + request=request, ) @@ -704,16 +713,18 @@ async def push_model( raise HTTPException(status_code=400, detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.model)) url_idx = models[form_data.model]['urls'][0] - url = (await Config.get('ollama.base_urls', []))[url_idx] + url, api_config, key = await get_ollama_connection(url_idx) log.debug('url: %s', url) return await send_request( f'{url}/api/push', payload=form_data.model_dump_json(exclude_none=True).encode(), - key=get_api_key(url_idx, url, (await Config.get('ollama.api_configs', {}))), + key=key, user=user, stream=True, passthrough=True, + api_config=api_config, + request=request, ) @@ -738,15 +749,17 @@ async def create_model( raise HTTPException(status_code=503, detail=ERROR_MESSAGES.OLLAMA_API_DISABLED) log.debug('form_data: %s', form_data) - url = (await Config.get('ollama.base_urls', []))[url_idx] + url, api_config, key = await get_ollama_connection(url_idx) return await send_request( f'{url}/api/create', payload=form_data.model_dump_json(exclude_none=True).encode(), - key=get_api_key(url_idx, url, (await Config.get('ollama.api_configs', {}))), + key=key, user=user, stream=True, passthrough=True, + api_config=api_config, + request=request, ) @@ -776,14 +789,15 @@ async def copy_model( raise HTTPException(status_code=400, detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.source)) url_idx = models[form_data.source]['urls'][0] - url = (await Config.get('ollama.base_urls', []))[url_idx] - key = get_api_key(url_idx, url, (await Config.get('ollama.api_configs', {}))) + url, api_config, key = await get_ollama_connection(url_idx) await send_request( f'{url}/api/copy', payload=form_data.model_dump_json(exclude_none=True).encode(), key=key, user=user, + api_config=api_config, + request=request, ) await publish_event( request, @@ -818,8 +832,7 @@ async def delete_model( raise HTTPException(status_code=400, detail=ERROR_MESSAGES.MODEL_NOT_FOUND(model)) url_idx = models[model]['urls'][0] - url = (await Config.get('ollama.base_urls', []))[url_idx] - key = get_api_key(url_idx, url, (await Config.get('ollama.api_configs', {}))) + url, api_config, key = await get_ollama_connection(url_idx) await send_request( f'{url}/api/delete', @@ -827,6 +840,8 @@ async def delete_model( payload=JSONCodec.dumps(payload), key=key, user=user, + api_config=api_config, + request=request, ) await publish_event( request, @@ -861,14 +876,15 @@ async def show_model_info( raise HTTPException(status_code=400, detail=ERROR_MESSAGES.MODEL_NOT_FOUND(model)) url_idx = random.choice(models[model]['urls']) - url = (await Config.get('ollama.base_urls', []))[url_idx] - key = get_api_key(url_idx, url, (await Config.get('ollama.api_configs', {}))) + url, api_config, key = await get_ollama_connection(url_idx) return await send_request( f'{url}/api/show', payload=JSONCodec.dumps(payload), key=key, user=user, + api_config=api_config, + request=request, ) @@ -922,6 +938,8 @@ async def embed( payload=form_data.model_dump_json(exclude_none=True).encode(), key=key, user=user, + api_config=api_config, + request=request, ) @@ -973,6 +991,8 @@ async def embeddings( payload=form_data.model_dump_json(exclude_none=True).encode(), key=key, user=user, + api_config=api_config, + request=request, ) @@ -1030,6 +1050,8 @@ async def generate_completion( user=user, stream=True, passthrough=True, + api_config=api_config, + request=request, ) @@ -1501,8 +1523,15 @@ async def get_openai_models( model_list = await get_all_models(request, user=user) raw_models = model_list['models'] else: - url = (await Config.get('ollama.base_urls', []))[url_idx] - model_list = await send_request(f'{url}/api/tags', 'GET') + url, api_config, key = await get_ollama_connection(url_idx) + model_list = await send_request( + f'{url}/api/tags', + 'GET', + key=key, + user=user, + api_config=api_config, + request=request, + ) raw_models = model_list.get('models', []) now_ts = int(time.time()) @@ -1552,6 +1581,8 @@ async def download_file_stream( file_url: str, file_path: str, file_name: str, + ollama_headers: dict, + ollama_cookies: dict, chunk_size: int = AIOHTTP_FILE_STREAM_CHUNK_SIZE, ): """Stream a model file download from *file_url*, then push the blob to Ollama.""" @@ -1590,7 +1621,12 @@ async def blob_chunks(): async with session.post( blob_url, data=blob_chunks(), - headers={'Content-Length': str(blob_size)}, + headers={ + **ollama_headers, + 'Content-Type': 'application/octet-stream', + 'Content-Length': str(blob_size), + }, + cookies=ollama_cookies, ssl=AIOHTTP_CLIENT_SESSION_SSL, timeout=aiohttp.ClientTimeout(total=30), ) as blob_resp: @@ -1617,15 +1653,17 @@ async def download_model( detail='Invalid file_url. Only URLs from allowed hosts are permitted.', ) - url = (await Config.get('ollama.base_urls', []))[url_idx if url_idx is not None else 0] + url, api_config, key = await get_ollama_connection(url_idx if url_idx is not None else 0) file_name = parse_huggingface_url(form_data.url) if not file_name: return None + headers, cookies = await get_headers_and_cookies(request, url, key, api_config, user=user) + file_path = os.path.join(UPLOAD_DIR, file_name) return StreamingResponse( - download_file_stream(url, form_data.url, file_path, file_name), + download_file_stream(url, form_data.url, file_path, file_name, headers, cookies), ) @@ -1638,7 +1676,8 @@ async def upload_model( user=Depends(get_admin_user), ): """Upload a local model file, push it as a blob, and create the model in Ollama.""" - ollama_url = (await Config.get('ollama.base_urls', []))[url_idx if url_idx is not None else 0] + ollama_url, api_config, key = await get_ollama_connection(url_idx if url_idx is not None else 0) + headers, cookies = await get_headers_and_cookies(request, ollama_url, key, api_config, user=user) filename = os.path.basename(file.filename) file_path = os.path.join(UPLOAD_DIR, filename) @@ -1680,7 +1719,12 @@ async def blob_chunks(): async with session.post( blob_url, data=blob_chunks(), - headers={'Content-Length': str(total_size)}, + headers={ + **headers, + 'Content-Type': 'application/octet-stream', + 'Content-Length': str(total_size), + }, + cookies=cookies, ssl=AIOHTTP_CLIENT_SESSION_SSL, timeout=get_client_timeout(), ) as resp: @@ -1702,7 +1746,8 @@ async def blob_chunks(): async with session.post( f'{ollama_url}/api/create', - headers={'Content-Type': 'application/json'}, + headers=headers, + cookies=cookies, data=JSONCodec.dumps(create_payload), ssl=AIOHTTP_CLIENT_SESSION_SSL, timeout=get_client_timeout(), diff --git a/backend/open_webui/routers/openai.py b/backend/open_webui/routers/openai.py index 63676d166e91..b8a75c2bfb39 100644 --- a/backend/open_webui/routers/openai.py +++ b/backend/open_webui/routers/openai.py @@ -10,7 +10,6 @@ import aiofiles import aiohttp from aiocache import cached -from azure.identity import DefaultAzureCredential, get_bearer_token_provider from fastapi import APIRouter, Depends, HTTPException, Request, status from fastapi.responses import ( FileResponse, @@ -28,7 +27,6 @@ BYPASS_MODEL_ACCESS_CONTROL, ENABLE_FORWARD_USER_INFO_HEADERS, ENABLE_OPENAI_API_PASSTHROUGH, - FORWARD_SESSION_INFO_HEADER_CHAT_ID, MODELS_CACHE_TTL, REDIS_KEY_PREFIX, ) @@ -42,7 +40,7 @@ from open_webui.utils.access_control import check_model_access, has_connection_access, has_permission from open_webui.utils.anthropic import ANTHROPIC_VERSION, get_anthropic_models, is_anthropic_url from open_webui.utils.auth import get_admin_user, get_verified_user -from open_webui.utils.headers import get_custom_headers, include_user_info_headers +from open_webui.utils.headers import get_headers_and_cookies, include_user_info_headers from open_webui.utils.json_codec import JSONCodec from open_webui.utils.misc import convert_logit_bias_input_to_json from open_webui.utils.model_ids import strip_provider_model_prefix @@ -152,87 +150,6 @@ def openai_reasoning_model_handler(payload): return payload -async def get_headers_and_cookies( - request: Request, - url, - key=None, - config=None, - metadata: dict | None = None, - user: UserModel = None, -): - cookies = getattr(request, 'cookies', {}) if config.get('forward_cookies', False) else {} - headers = { - 'Content-Type': 'application/json', - **( - { - # LICENSE covers this Open WebUI upstream metadata identifier. - # Do not alter, remove, obscure, or replace it except as LICENSE permits: - # https://docs.openwebui.com/license. - 'HTTP-Referer': 'https://openwebui.com/', - 'X-Title': 'Open WebUI', - } - if 'openrouter.ai' in url - else {} - ), - } - - if ENABLE_FORWARD_USER_INFO_HEADERS and user: - headers = include_user_info_headers(headers, user, request=request) - if metadata and metadata.get('chat_id'): - headers[FORWARD_SESSION_INFO_HEADER_CHAT_ID] = metadata.get('chat_id') - - token = None - auth_type = config.get('auth_type') - - if auth_type == 'bearer' or auth_type is None: - # Default to bearer if not specified - token = f'{key}' - elif auth_type == 'none': - token = None - elif auth_type == 'session': - token = request.state.token.credentials - elif auth_type == 'system_oauth': - oauth_token = None - try: - if request.cookies.get('oauth_session_id', None): - oauth_token = await request.app.state.oauth_manager.get_oauth_token( - user.id, - request.cookies.get('oauth_session_id', None), - ) - except Exception as e: - log.error(f'Error getting OAuth token: {e}') - - if oauth_token: - token = f'{oauth_token.get("access_token", "")}' - - elif auth_type in ('azure_ad', 'microsoft_entra_id'): - token = get_microsoft_entra_id_access_token() - - if token: - headers['Authorization'] = f'Bearer {token}' - - if config.get('headers') and isinstance(config.get('headers'), dict): - custom_headers = await get_custom_headers(config.get('headers'), user, metadata, request=request) - headers.update(custom_headers) - - return headers, cookies - - -def get_microsoft_entra_id_access_token(): - """ - Get Microsoft Entra ID access token using DefaultAzureCredential for Azure OpenAI. - Returns the token string or None if authentication fails. - """ - try: - token_provider = get_bearer_token_provider( - DefaultAzureCredential(), 'https://cognitiveservices.azure.com/.default' - ) - return token_provider() - except Exception as e: - log.error(f'Error getting Microsoft Entra ID access token: {e}') - return None - - ########################################## # # API routes diff --git a/backend/open_webui/routers/retrieval.py b/backend/open_webui/routers/retrieval.py index c36b0b8a53b5..401eeb079ad4 100644 --- a/backend/open_webui/routers/retrieval.py +++ b/backend/open_webui/routers/retrieval.py @@ -59,6 +59,7 @@ SENTENCE_TRANSFORMERS_CROSS_ENCODER_SIGMOID_ACTIVATION_FUNCTION, SENTENCE_TRANSFORMERS_MODEL_KWARGS, USE_SLIM, + USER_AGENT, ) from open_webui.events import EVENTS, publish_event from open_webui.internal.db import get_async_db, get_async_session @@ -386,6 +387,7 @@ def get_rf( 'STAAN_MAX_SNIPPETS': 'web.search.staan_max_snippets', 'TAVILY_API_KEY': 'web.search.tavily_api_key', 'TAVILY_EXTRACT_DEPTH': 'web.search.tavily_extract_depth', + 'TAVILY_SEARCH_DEPTH': 'web.search.tavily_search_depth', 'TEXT_SPLITTER': 'rag.text_splitter', 'TIKA_SERVER_URL': 'rag.tika_server_url', 'TIKA_SERVER_VERSION': 'rag.tika_server_version', @@ -772,6 +774,7 @@ async def get_rag_config(request: Request, user=Depends(get_admin_user)): 'FIRECRAWL_API_BASE_URL': config.FIRECRAWL_API_BASE_URL, 'FIRECRAWL_TIMEOUT': config.FIRECRAWL_TIMEOUT, 'TAVILY_EXTRACT_DEPTH': config.TAVILY_EXTRACT_DEPTH, + 'TAVILY_SEARCH_DEPTH': config.TAVILY_SEARCH_DEPTH, 'EXTERNAL_WEB_SEARCH_URL': config.EXTERNAL_WEB_SEARCH_URL, 'EXTERNAL_WEB_SEARCH_API_KEY': config.EXTERNAL_WEB_SEARCH_API_KEY, 'EXTERNAL_WEB_LOADER_URL': config.EXTERNAL_WEB_LOADER_URL, @@ -855,6 +858,7 @@ class WebConfig(BaseModel): FIRECRAWL_API_BASE_URL: str | None = None FIRECRAWL_TIMEOUT: str | None = None TAVILY_EXTRACT_DEPTH: str | None = None + TAVILY_SEARCH_DEPTH: str | None = None EXTERNAL_WEB_SEARCH_URL: str | None = None EXTERNAL_WEB_SEARCH_API_KEY: str | None = None EXTERNAL_WEB_LOADER_URL: str | None = None @@ -1385,6 +1389,7 @@ async def update_rag_config(request: Request, form_data: ConfigForm, user=Depend config.EXTERNAL_WEB_LOADER_URL = form_data.web.EXTERNAL_WEB_LOADER_URL config.EXTERNAL_WEB_LOADER_API_KEY = form_data.web.EXTERNAL_WEB_LOADER_API_KEY config.TAVILY_EXTRACT_DEPTH = form_data.web.TAVILY_EXTRACT_DEPTH + config.TAVILY_SEARCH_DEPTH = form_data.web.TAVILY_SEARCH_DEPTH config.YOUTUBE_LOADER_LANGUAGE = form_data.web.YOUTUBE_LOADER_LANGUAGE config.YOUTUBE_LOADER_PROXY_URL = form_data.web.YOUTUBE_LOADER_PROXY_URL request.app.state.YOUTUBE_LOADER_TRANSLATION = form_data.web.YOUTUBE_LOADER_TRANSLATION @@ -1534,6 +1539,7 @@ async def update_rag_config(request: Request, form_data: ConfigForm, user=Depend 'FIRECRAWL_API_BASE_URL': config.FIRECRAWL_API_BASE_URL, 'FIRECRAWL_TIMEOUT': config.FIRECRAWL_TIMEOUT, 'TAVILY_EXTRACT_DEPTH': config.TAVILY_EXTRACT_DEPTH, + 'TAVILY_SEARCH_DEPTH': config.TAVILY_SEARCH_DEPTH, 'EXTERNAL_WEB_SEARCH_URL': config.EXTERNAL_WEB_SEARCH_URL, 'EXTERNAL_WEB_SEARCH_API_KEY': config.EXTERNAL_WEB_SEARCH_API_KEY, 'EXTERNAL_WEB_LOADER_URL': config.EXTERNAL_WEB_LOADER_URL, @@ -1935,10 +1941,10 @@ async def process_file( The session is committed before external API calls, and updates use a fresh session. """ config = await get_retrieval_config() - if user.role == 'admin': - file = await Files.get_file_by_id(form_data.file_id, db=db) - else: - file = await Files.get_file_by_id_and_user_id(form_data.file_id, user.id, db=db) + file = await Files.get_file_by_id(form_data.file_id, db=db) + if file and file.user_id != user.id and user.role != 'admin': + if not await has_access_to_file(file.id, 'write', user, db=db): + file = None if file: try: @@ -2251,9 +2257,11 @@ async def _fetch_url(url: str, max_size_mb: int | str | None) -> dict: except (TypeError, ValueError): max_bytes = None + headers = {'User-Agent': USER_AGENT} if USER_AGENT else None + async with get_ssrf_safe_session() as session: async with session.get( - url, ssl=AIOHTTP_CLIENT_SESSION_SSL, allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS + url, headers=headers, ssl=AIOHTTP_CLIENT_SESSION_SSL, allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS ) as response: response.raise_for_status() @@ -2373,7 +2381,16 @@ async def process_url( } config = await get_retrieval_config() - url_result = await _fetch_url(form_data.url, config.FILE_MAX_SIZE) + try: + url_result = await _fetch_url(form_data.url, config.FILE_MAX_SIZE) + except HTTPException: + raise + except Exception as e: + log.exception(e) + raise HTTPException( + status_code=status.HTTP_400_BAD_REQUEST, + detail=ERROR_MESSAGES.DEFAULT(e, f'Could not read content from {form_data.url}'), + ) if url_result['kind'] == 'web': result = await process_web(request, form_data, process=process, user=user) @@ -2454,6 +2471,13 @@ async def process_web( detail=ERROR_MESSAGES.DEFAULT(e, f'Could not read content from {form_data.url}'), ) + # web loaders swallow fetch errors and return no documents + if not docs: + raise HTTPException( + status_code=status.HTTP_400_BAD_REQUEST, + detail=ERROR_MESSAGES.DEFAULT(f'Could not read content from {form_data.url}'), + ) + try: log.debug('text_content: %s', content) @@ -2702,6 +2726,7 @@ async def search_web(request: Request, engine: str, query: str, user=None) -> li query, config.WEB_SEARCH_RESULT_COUNT, config.WEB_SEARCH_DOMAIN_FILTER_LIST, + search_depth=config.TAVILY_SEARCH_DEPTH, ) else: raise Exception('No TAVILY_API_KEY found in environment variables') diff --git a/backend/open_webui/routers/terminals.py b/backend/open_webui/routers/terminals.py index e57314bfde86..6b5cd6e1d3da 100644 --- a/backend/open_webui/routers/terminals.py +++ b/backend/open_webui/routers/terminals.py @@ -5,6 +5,7 @@ * /{server_id}/{path:path} — proxy request to terminal server """ +import asyncio import logging import posixpath from urllib.parse import unquote @@ -18,17 +19,17 @@ from open_webui.models.config import Config from open_webui.models.groups import Groups from open_webui.utils.access_control import has_connection_access -from open_webui.utils.auth import get_verified_user +from open_webui.utils.auth import get_verified_user, get_verified_user_by_token from open_webui.utils.headers import bearer_auth_header, normalize_bearer_token from open_webui.utils.json_codec import JSONCodec from open_webui.utils.terminals import ( TERMINAL_CONTEXT_HEADER, get_terminal_server_url, is_terminal_orchestrator, + terminal_chat_uploads, terminal_context_available, terminal_context_config, terminal_context_id, - terminal_chat_uploads, terminal_contexts, ) from starlette.background import BackgroundTask @@ -262,10 +263,6 @@ async def _resolve_authenticated_connection(ws: WebSocket, server_id: str): Returns ``(user, connection, chat_id, token)`` on success, or ``None`` after closing *ws* with an appropriate error code. """ - import asyncio - - from open_webui.utils.auth import get_verified_user_by_token - # First-message authentication try: raw = await asyncio.wait_for(ws.receive_text(), timeout=10.0) @@ -274,13 +271,28 @@ async def _resolve_authenticated_connection(ws: WebSocket, server_id: str): await ws.close(code=4001, reason='Expected auth message') return None token = payload.get('token', '') + except (TimeoutError, JSONCodec.JSONDecodeError): + await ws.close(code=4001, reason='Auth timeout or invalid payload') + return None + except Exception: + await ws.close(code=4001, reason='Invalid token') + return None + + result = await _resolve_terminal_access(ws, server_id, token) + if result is None: + return None + user, connection = result + chat_id = payload.get('chat_id', '') + return user, connection, chat_id if isinstance(chat_id, str) else '', token + + +async def _resolve_terminal_access(ws: WebSocket, server_id: str, token: str): + """Resolve current access for both the handshake and an open terminal session.""" + try: user = await get_verified_user_by_token(token, getattr(ws.app.state, 'redis', None)) if user is None: await ws.close(code=4001, reason='Invalid token') return None - except (asyncio.TimeoutError, JSONCodec.JSONDecodeError): - await ws.close(code=4001, reason='Auth timeout or invalid payload') - return None except Exception: await ws.close(code=4001, reason='Invalid token') return None @@ -297,16 +309,14 @@ async def _resolve_authenticated_connection(ws: WebSocket, server_id: str): await ws.close(code=4003, reason='Terminal server disabled') return None - user_group_ids = {group.id for group in await Groups.get_groups_by_member_id(user.id)} - if not await has_connection_access(user, connection, user_group_ids): + if not await has_connection_access(user, connection): await ws.close(code=4003, reason='Access denied') return None - chat_id = payload.get('chat_id', '') if not terminal_context_available(connection, 'chat'): await ws.close(code=4003, reason='Terminal server is not available in chats') return None - return user, connection, chat_id if isinstance(chat_id, str) else '', token + return user, connection @router.websocket('/{server_id}/api/terminals/{session_id}') @@ -366,7 +376,6 @@ async def ws_terminal( headers=upstream_headers, ssl=AIOHTTP_CLIENT_SESSION_SSL, ) as upstream: - import asyncio import json as _json # First-message auth to upstream terminal server @@ -419,20 +428,30 @@ async def _upstream_to_client(): except Exception: pass - # End the proxy as soon as either direction finishes (e.g. a - # graceful upstream CLOSE) and cancel the sibling, which would + async def _watch_access(): + try: + while True: + # Poll current state so revocation also works across workers. + await asyncio.sleep(10) + if await _resolve_terminal_access(ws, server_id, token) is None: + return + except Exception: + log.exception('Terminal access recheck failed') + + # End the proxy as soon as any task finishes (e.g. a + # graceful upstream CLOSE) and cancel the rest, which would # otherwise hang on a blocked ws.receive() until the browser leaves. tasks = [ asyncio.create_task(_client_to_upstream()), asyncio.create_task(_upstream_to_client()), + asyncio.create_task(_watch_access()), ] - _done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED) - for task in pending: - task.cancel() - try: - await task - except asyncio.CancelledError: - pass + try: + await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED) + finally: + for task in tasks: + task.cancel() + await asyncio.gather(*tasks, return_exceptions=True) except Exception as e: log.exception('Terminal WebSocket proxy error: %s', e) finally: diff --git a/backend/open_webui/routers/tools.py b/backend/open_webui/routers/tools.py index a814eb30f367..12c3771420ae 100644 --- a/backend/open_webui/routers/tools.py +++ b/backend/open_webui/routers/tools.py @@ -27,7 +27,7 @@ ) from open_webui.utils.access_control import ( filter_allowed_access_grants, - has_access, + has_connection_access, has_permission, ) from open_webui.utils.auth import get_admin_user, get_verified_user @@ -102,7 +102,7 @@ async def get_tools( ) # OpenAPI Tool Servers - server_access_grants = {} + server_connections = {} for server in await get_tool_servers(request): server_idx = server.get('idx', 0) connections = await Config.get('tool_server.connections', []) @@ -113,10 +113,8 @@ async def get_tools( ) continue connection = connections[server_idx] - server_config = connection.get('config', {}) - server_id = f'server:{server.get("id")}' - server_access_grants[server_id] = server_config.get('access_grants', []) + server_connections[server_id] = connection tools.append( ToolUserResponse( @@ -149,10 +147,8 @@ async def get_tools( user.id, f'mcp:{server_id}' ) - server_config = server.get('config') or {} - tool_id = f'server:mcp:{info.get("id")}' - server_access_grants[tool_id] = server_config.get('access_grants', []) + server_connections[tool_id] = server tools.append( ToolUserResponse( @@ -181,12 +177,10 @@ async def get_tools( tool for tool in tools if not str(tool.id).startswith('server:') - or await has_access( - user.id, - 'read', - server_access_grants.get(str(tool.id), []), + or await has_connection_access( + user, + server_connections[str(tool.id)], user_group_ids, - db=db, ) ] diff --git a/backend/open_webui/socket/main.py b/backend/open_webui/socket/main.py index d4ac18f655a1..f157ed2f81ba 100644 --- a/backend/open_webui/socket/main.py +++ b/backend/open_webui/socket/main.py @@ -6,6 +6,7 @@ import random import sys import time +from contextlib import suppress from typing import Any import pycrdt as Y @@ -23,6 +24,7 @@ WEBSOCKET_REDIS_CLUSTER, WEBSOCKET_REDIS_LOCK_TIMEOUT, WEBSOCKET_REDIS_OPTIONS, + WEBSOCKET_REDIS_ROOM_CHANNELS, WEBSOCKET_REDIS_URL, WEBSOCKET_SENTINEL_HOSTS, WEBSOCKET_SENTINEL_PORT, @@ -37,18 +39,25 @@ from open_webui.models.folders import Folders from open_webui.models.notes import Notes, NoteUpdateForm from open_webui.models.users import UserNameResponse, Users -from open_webui.socket.utils import RedisDict, RedisLock, YdocManager -from open_webui.tasks import create_task, stop_item_tasks +from open_webui.socket.redis_room_channels import AsyncRedisRoomChannelManager +from open_webui.socket.utils import CachedRedisDict, RedisDict, RedisLock, YdocManager +from open_webui.tasks import ( + REDIS_PUBSUB_MAX_RECONNECT_INTERVAL, + REDIS_PUBSUB_RECONNECT_INTERVAL, + create_task, + stop_item_tasks, +) from open_webui.utils.access_control import has_permission from open_webui.utils.auth import get_verified_user_by_token from open_webui.utils.chat_id import is_saved_chat_id -from open_webui.utils.json_codec import SOCKETIO_JSON +from open_webui.utils.json_codec import SOCKETIO_JSON, JSONCodec, dumps_bytes from open_webui.utils.misc import get_output_text from open_webui.utils.redis import ( build_sentinel_url, get_redis_connection, get_sentinels_from_env, ) +from redis.exceptions import RedisError from socketio.packet import Packet logging.basicConfig(stream=sys.stdout, level=GLOBAL_LOG_LEVEL) @@ -86,7 +95,8 @@ def reconstruct_binary(cls, data: Any, attachments: list[bytes]): if sentinel_hosts else WEBSOCKET_REDIS_URL ) - redis_manager = socketio.AsyncRedisManager(ws_redis_url, redis_options=WEBSOCKET_REDIS_OPTIONS, json=SOCKETIO_JSON) + manager_class = AsyncRedisRoomChannelManager if WEBSOCKET_REDIS_ROOM_CHANNELS else socketio.AsyncRedisManager + redis_manager = manager_class(ws_redis_url, redis_options=WEBSOCKET_REDIS_OPTIONS, json=SOCKETIO_JSON) sio = socketio.AsyncServer( cors_allowed_origins=SOCKETIO_CORS_ORIGINS, async_mode='asgi', @@ -133,12 +143,11 @@ def reconstruct_binary(cls, data: Any, attachments: list[bytes]): async_mode=True, ) - MODELS = RedisDict( + MODELS = CachedRedisDict( f'{REDIS_KEY_PREFIX}:models', redis_url=WEBSOCKET_REDIS_URL, redis_sentinels=ws_sentinels, redis_cluster=WEBSOCKET_REDIS_CLUSTER, - cache_set_signature=True, ) SESSION_POOL = RedisDict( @@ -190,6 +199,11 @@ def reconstruct_binary(cls, data: Any, attachments: list[bytes]): redis_key_prefix=f'{REDIS_KEY_PREFIX}:ydoc:documents', ) +REDIS_EVENT_CHANNEL = f'{REDIS_KEY_PREFIX}:direct_completion' + +EVENT_QUEUES: dict[str, asyncio.Queue] = {} +EVENT_PUBLISH_LOCK = asyncio.Lock() + def get_session_pool_batches(): """All session pool entries, in bounded batches for the Redis backing.""" @@ -332,8 +346,20 @@ def get_session_ids_from_room(room): def get_session_ids_by_user_id(user_id: str) -> list[str]: """Get known session IDs for a user across the local rooms and shared session pool.""" - session_ids = set(get_session_ids_from_room(f'user:{user_id}')) - session_ids.update(sid for sid, entry in SESSION_POOL.items() if entry and entry.get('id') == user_id) + return get_session_ids_by_user_ids([user_id]) + + +def get_session_ids_by_user_ids(user_ids: list[str]) -> list[str]: + """Get known session IDs for users across the local rooms and shared session pool.""" + if not user_ids: + return [] + + user_ids = set(user_ids) + session_ids = set() + for user_id in user_ids: + session_ids.update(get_session_ids_from_room(f'user:{user_id}')) + for batch in get_session_pool_batches(): + session_ids.update(sid for sid, user in batch if user and user.get('id') in user_ids) return list(session_ids) @@ -374,6 +400,15 @@ async def enter_room_for_users(room: str, user_ids: list[str]): log.debug('Failed to make users %s join room %s: %s', user_ids, room, e) +async def leave_room_for_users(room: str, user_ids: list[str]): + """Make all sessions of each user leave a room, including sessions on other workers.""" + for sid in get_session_ids_by_user_ids(user_ids): + try: + await sio.leave_room(sid, room) + except Exception as e: + log.debug('Failed to make session %s leave room %s: %s', sid, room, e) + + async def disconnect_user_sessions(user_id: str): """Disconnect all Socket.IO sessions belonging to a user. @@ -828,33 +863,34 @@ async def yjs_document_update(sid, data): log.warning(f'User {user.get("id")} does not have write access to note {note_id}. Rejecting update.') return - user_id = data.get('user_id', sid) + update = data.get('update') # List of bytes from frontend - update = data['update'] # List of bytes from frontend + if update: + user_id = data.get('user_id', sid) - await YDOC_MANAGER.append_to_updates( - document_id=document_id, - update=update, # Convert list of bytes to bytes - ) + await YDOC_MANAGER.append_to_updates( + document_id=document_id, + update=update, # Convert list of bytes to bytes + ) - # Broadcast update to all other users in the document - await sio.emit( - 'ydoc:document:update', - { - 'document_id': document_id, - 'user_id': user_id, - 'update': update, - 'socket_id': sid, # Add socket_id to match frontend filtering - }, - room=f'doc_{document_id}', - skip_sid=sid, - ) + # Broadcast update to all other users in the document + await sio.emit( + 'ydoc:document:update', + { + 'document_id': document_id, + 'user_id': user_id, + 'update': update, + 'socket_id': sid, # Add socket_id to match frontend filtering + }, + room=f'doc_{document_id}', + skip_sid=sid, + ) async def debounced_save(): await asyncio.sleep(0.5) await document_save_handler(document_id, data.get('data', {}), user) - if data.get('data'): + if document_id.startswith('note:') and data.get('data'): # Only drop the pending save when a new one takes its place. # Updates without a content snapshot (the resync a client sends # after rejoining a document) would otherwise cancel the pending @@ -948,6 +984,62 @@ async def disconnect(sid, reason=None): # print(f"Unknown session ID {sid} disconnected") +async def redis_event_listener() -> None: + """Route events received over Redis to their local queues.""" + reconnect_interval = REDIS_PUBSUB_RECONNECT_INTERVAL + + while True: + pubsub = None + try: + # RedisCluster can't route a pubsub subscribe until initialize() fills its slot cache. + await REDIS.initialize() + + pubsub = REDIS.pubsub() + await pubsub.subscribe(REDIS_EVENT_CHANNEL) + reconnect_interval = REDIS_PUBSUB_RECONNECT_INTERVAL + + async for message in pubsub.listen(): + if message['type'] != 'message': + continue + event = JSONCodec.loads(message['data']) + queue = EVENT_QUEUES.get(event['channel']) + if queue is not None: + await queue.put(event['data']) + log.warning('Redis event listener stopped. Retrying.') + except asyncio.CancelledError: + raise + except Exception: + log.exception('Redis event listener failed. Retrying.') + finally: + if pubsub: + with suppress(Exception): + await pubsub.aclose() + + await asyncio.sleep(reconnect_interval) + reconnect_interval = min(reconnect_interval * 2, REDIS_PUBSUB_MAX_RECONNECT_INTERVAL) + + +@sio.on('*') +async def socket_event_handler(event: Any, sid: str, *args: Any) -> None: + """Route user-owned stream events to a local queue or another worker.""" + if not isinstance(event, str) or event.count(':') != 2 or not args: + return + + user = await get_socket_session_user(sid) + if not user or user.get('id') != event.split(':', 1)[0]: + return + + queue = EVENT_QUEUES.get(event) + if queue is not None: + await queue.put(args[0]) + elif WEBSOCKET_MANAGER == 'redis': + try: + async with EVENT_PUBLISH_LOCK: + await REDIS.publish(REDIS_EVENT_CHANNEL, dumps_bytes({'channel': event, 'data': args[0]})) + except RedisError as e: + log.debug('Failed to relay socket event %s: %s', event, e) + + async def _make_channel_emitter(request_info): """Event emitter that routes pipeline output to a channel message. @@ -1015,7 +1107,8 @@ async def __channel_emitter__(event_data): state['output'] = copy.deepcopy(output) now = time.time() - if done or (now - state['last_emit_at']) >= THROTTLE_INTERVAL: + # Tool boundaries must publish all results before waiting on the next model response. + if done or data.get('flush') or (now - state['last_emit_at']) >= THROTTLE_INTERVAL: state['last_emit_at'] = now await _emit_channel_update(content, done, output if isinstance(output, list) else None) diff --git a/backend/open_webui/socket/redis_room_channels.py b/backend/open_webui/socket/redis_room_channels.py new file mode 100644 index 000000000000..715ccc1ea92d --- /dev/null +++ b/backend/open_webui/socket/redis_room_channels.py @@ -0,0 +1,82 @@ +"""Per-room redis channels let instances skip the decode and packet encode for rooms with no local members.""" + +import asyncio + +from socketio import AsyncRedisManager + + +class AsyncRedisRoomChannelManager(AsyncRedisManager): + name = 'aioredisroomchannel' + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self._local_room_channels = set() + + # collision-free while namespaces contain no '#' (socket.io default '/'); rooms may contain '#' + def _room_channel(self, namespace, room): + return f'{self.channel}#{namespace}#{room}'.encode() + + def basic_enter_room(self, sid, namespace, room, eio_sid=None): + super().basic_enter_room(sid, namespace, room, eio_sid=eio_sid) + if room is not None: + self._local_room_channels.add(self._room_channel(namespace, room)) + + def basic_leave_room(self, sid, namespace, room): + super().basic_leave_room(sid, namespace, room) + if room is not None and room not in self.rooms.get(namespace, {}): + self._local_room_channels.discard(self._room_channel(namespace, room)) + + async def _publish(self, data): + if data.get('method') == 'emit' and isinstance(data.get('room'), str): + channel = self._room_channel(data['namespace'], data['room']) + else: + channel = self.channel + _, error = self._get_redis_module_and_error() + for retries_left in range(1, -1, -1): # 2 attempts + try: + if not self.connected: + self._redis_connect() + return await self.redis.publish(channel, self.json.dumps(data)) + except error as exc: + if retries_left > 0: + self._get_logger().error('Cannot publish to redis... retrying', extra={'redis_exception': str(exc)}) + self.connected = False + else: + self._get_logger().error( + 'Cannot publish to redis... giving up', extra={'redis_exception': str(exc)} + ) + break + + async def _redis_listen_with_retries(self): + _, error = self._get_redis_module_and_error() + retry_sleep = 1 + subscribed = False + while True: + try: + if not subscribed: + self._redis_connect() + await self.pubsub.subscribe(self.channel) + await self.pubsub.psubscribe(f'{self.channel}#*') + retry_sleep = 1 + async for message in self.pubsub.listen(): + yield message + except error as exc: + self._get_logger().error( + f'Cannot receive from redis... retrying in {retry_sleep} secs', + extra={'redis_exception': str(exc)}, + ) + subscribed = False + await asyncio.sleep(retry_sleep) + retry_sleep *= 2 + if retry_sleep > 60: + retry_sleep = 60 + + async def _listen(self): + main_channel = self.channel.encode() + async for message in self._redis_listen_with_retries(): + if 'data' not in message: + continue + if (message['type'] == 'message' and message['channel'] == main_channel) or ( + message['type'] == 'pmessage' and message['channel'] in self._local_room_channels + ): + yield message['data'] diff --git a/backend/open_webui/socket/utils.py b/backend/open_webui/socket/utils.py index 049052a1e12f..f976a94e77c8 100644 --- a/backend/open_webui/socket/utils.py +++ b/backend/open_webui/socket/utils.py @@ -134,7 +134,8 @@ def delete_many(self, *keys): """Delete fields in one HDEL; no keys is a no-op (HDEL rejects an empty field list).""" if keys: self.redis.hdel(self.name, *keys) - self._last_signature = None + if self._signature_name: + self.redis.delete(self._signature_name) def set(self, mapping: dict): if not mapping: @@ -149,10 +150,14 @@ def set(self, mapping: dict): digest.update(b'\0') digest.update(serialized[key].encode()) digest.update(b'\0') - signature = digest.hexdigest() + content_digest = digest.hexdigest() - if self._signature_name and self.redis.get(self._signature_name) == signature: - return + if self._signature_name: + stored_signature = self.redis.get(self._signature_name) + if stored_signature and stored_signature.startswith(f'{content_digest}:'): + return + # Cleared first so readers refetch while the hash is being rewritten. + self.redis.delete(self._signature_name) # Fetch existing keys before writing so we know which ones to remove. # HKEYS is cheap — it transfers only short key strings, not large JSON values. @@ -168,7 +173,7 @@ def set(self, mapping: dict): self.redis.hdel(self.name, *keys_to_remove) if self._signature_name: - self.redis.set(self._signature_name, signature) + self.redis.set(self._signature_name, f'{content_digest}:{uuid.uuid4().hex}') def get(self, key, default=None): try: @@ -196,6 +201,43 @@ def setdefault(self, key, default=None): return self[key] +class CachedRedisDict(RedisDict): + """Answers reads from a per-worker cache of the hash, refetched whenever its signature changes.""" + + def __init__(self, name: str, redis_url: str, redis_sentinels: list = [], redis_cluster: bool = False): + super().__init__(name, redis_url, redis_sentinels, redis_cluster, cache_set_signature=True) + self._cache: dict = {} + self._cached_signature: str | None = None + + def _refresh_cache(self) -> dict: + stored_signature = self.redis.get(self._signature_name) + if stored_signature is None or stored_signature != self._cached_signature: + self._cache = self.redis.hgetall(self.name) + self._cached_signature = stored_signature + return self._cache + + def __getitem__(self, key): + value = self._refresh_cache().get(key) + if value is None: + raise KeyError(key) + return JSONCodec.loads(value) + + def __contains__(self, key): + return key in self._refresh_cache() + + def __len__(self): + return len(self._refresh_cache()) + + def keys(self): + return list(self._refresh_cache().keys()) + + def values(self): + return [JSONCodec.loads(v) for v in self._refresh_cache().values()] + + def items(self): + return [(k, JSONCodec.loads(v)) for k, v in self._refresh_cache().items()] + + class YdocManager: COMPACTION_THRESHOLD = 500 diff --git a/backend/open_webui/storage/provider.py b/backend/open_webui/storage/provider.py index c8236416de1d..919900f75254 100644 --- a/backend/open_webui/storage/provider.py +++ b/backend/open_webui/storage/provider.py @@ -168,7 +168,9 @@ def get_file(self, file_path: str) -> str: try: s3_key = self._extract_s3_key(file_path) local_file_path = self._get_local_file_path(s3_key) - self.s3_client.download_file(self.bucket_name, s3_key, local_file_path) + # download_file's temp name caps characters, not bytes, so non-ASCII names can exceed NAME_MAX + with open(local_file_path, 'wb') as local_file: + self.s3_client.download_fileobj(self.bucket_name, s3_key, local_file) return local_file_path except ClientError as e: raise RuntimeError(f'Error downloading file from S3: {e}') diff --git a/backend/open_webui/tasks.py b/backend/open_webui/tasks.py index 322cd32e60dc..05f622ef06f1 100644 --- a/backend/open_webui/tasks.py +++ b/backend/open_webui/tasks.py @@ -6,7 +6,7 @@ from redis.asyncio import Redis -from open_webui.env import REDIS_KEY_PREFIX, REDIS_RESPONSE_STREAM_TTL +from open_webui.env import REDIS_KEY_PREFIX, REDIS_RESPONSE_STREAM_TTL, REDIS_TASK_TTL from open_webui.utils.json_codec import JSONCodec, dumps_bytes log = logging.getLogger(__name__) @@ -66,13 +66,28 @@ async def redis_task_command_listener(app): reconnect_interval = min(reconnect_interval * 2, REDIS_PUBSUB_MAX_RECONNECT_INTERVAL) +async def redis_task_heartbeat(app): + redis: Redis = app.state.redis + while True: + await asyncio.sleep(REDIS_TASK_TTL / 4) + try: + pipe = redis.pipeline(transaction=False) + for task_id in list(tasks): + # EXPIRE cannot recreate a task already removed by cleanup. + pipe.expire(f'{REDIS_TASKS_KEY}:{task_id}', REDIS_TASK_TTL) + await pipe.execute() + except Exception: + log.exception('Redis task heartbeat failed') + + ### ------------------------------ ### REDIS-ENABLED HANDLERS ### ------------------------------ async def redis_save_task(redis: Redis, task_id: str, item_id: str | None): - pipe = redis.pipeline() + pipe = redis.pipeline(transaction=False) + pipe.set(f'{REDIS_TASKS_KEY}:{task_id}', '1', ex=REDIS_TASK_TTL or None) pipe.hset(REDIS_TASKS_KEY, task_id, item_id or '') if item_id: pipe.sadd(f'{REDIS_ITEM_TASKS_KEY}:{item_id}', task_id) @@ -80,25 +95,36 @@ async def redis_save_task(redis: Redis, task_id: str, item_id: str | None): async def redis_cleanup_task(redis: Redis, task_id: str, item_id: str | None): - pipe = redis.pipeline() + pipe = redis.pipeline(transaction=False) + pipe.delete(f'{REDIS_TASKS_KEY}:{task_id}') pipe.hdel(REDIS_TASKS_KEY, task_id) pipe.hdel(REDIS_RESPONSE_STREAMS_KEY, task_id) if item_id: pipe.srem(f'{REDIS_ITEM_TASKS_KEY}:{item_id}', task_id) - await pipe.execute() - # Remove the set key entirely if no tasks remain for this item - if await redis.scard(f'{REDIS_ITEM_TASKS_KEY}:{item_id}') == 0: - await redis.delete(f'{REDIS_ITEM_TASKS_KEY}:{item_id}') - else: - await pipe.execute() + await pipe.execute() -async def redis_list_tasks(redis: Redis) -> list[str]: - return list(await redis.hkeys(REDIS_TASKS_KEY)) +async def redis_list_tasks(redis: Redis, item_id: str | None = None) -> list[str]: + task_ids = list( + await redis.smembers(f'{REDIS_ITEM_TASKS_KEY}:{item_id}') + if item_id is not None + else await redis.hkeys(REDIS_TASKS_KEY) + ) + if not task_ids or REDIS_TASK_TTL == 0: + return task_ids + pipe = redis.pipeline(transaction=False) + for task_id in task_ids: + pipe.exists(f'{REDIS_TASKS_KEY}:{task_id}') -async def redis_list_item_tasks(redis: Redis, item_id: str) -> list[str]: - return list(await redis.smembers(f'{REDIS_ITEM_TASKS_KEY}:{item_id}')) + active = [] + for task_id, exists in zip(task_ids, await pipe.execute()): + if exists: + active.append(task_id) + else: + task_item_id = item_id if item_id is not None else await redis.hget(REDIS_TASKS_KEY, task_id) + await redis_cleanup_task(redis, task_id, task_item_id or None) + return active async def redis_send_command(redis: Redis, command: dict): @@ -166,7 +192,7 @@ async def list_task_ids_by_item_id(redis, id): List all tasks associated with a specific ID. """ if redis: - return await redis_list_item_tasks(redis, id) + return await redis_list_tasks(redis, id) return list(item_tasks.get(id, [])) diff --git a/backend/open_webui/tools/builtin.py b/backend/open_webui/tools/builtin.py index 8dedd75c1362..c1d1b164d626 100644 --- a/backend/open_webui/tools/builtin.py +++ b/backend/open_webui/tools/builtin.py @@ -3484,6 +3484,7 @@ async def view_skill( __request__: Request = None, __user__: dict = None, __metadata__: dict = None, + __event_call__: callable = None, ) -> str: """ Load the full instructions of a skill by its id from the available skills manifest. @@ -3504,7 +3505,9 @@ async def view_skill( from open_webui.utils.terminals import get_terminal_skill skill_name = unquote(id.removeprefix(terminal_skill_prefix)) - skill = await get_terminal_skill(__request__, __user__, __metadata__ or {}, skill_name) + skill = await get_terminal_skill( + __request__, __user__, __metadata__ or {}, skill_name, {'__event_call__': __event_call__} + ) if not skill: return JSONCodec.dumps({'error': f"Skill '{id}' not found"}) return JSONCodec.dumps(skill, ensure_ascii=False) @@ -3768,7 +3771,7 @@ async def create_automation( # Validate the RRULE try: - validate_rrule(rrule, tz=user.timezone) + await validate_rrule(rrule, tz=user.timezone) except ValueError as e: return JSONCodec.dumps({'error': f'Invalid schedule: {e}'}) @@ -3794,7 +3797,7 @@ async def create_automation( is_active=True, ) - automation = await Automations.insert(user_id, form, next_run_ns(rrule, tz=tz)) + automation = await Automations.insert(user_id, form, await next_run_ns(rrule, tz=tz)) return JSONCodec.dumps( { @@ -3805,7 +3808,7 @@ async def create_automation( 'model_id': model_id, 'target': automation.data.get('target'), 'is_active': automation.is_active, - 'next_runs': next_n_runs_ns(rrule, tz=tz), + 'next_runs': await next_n_runs_ns(rrule, tz=tz), }, ensure_ascii=False, ) @@ -3876,7 +3879,7 @@ async def update_automation( # Validate RRULE if changed if rrule is not None: try: - validate_rrule(new_rrule, tz=user.timezone) + await validate_rrule(new_rrule, tz=user.timezone) except ValueError as e: return JSONCodec.dumps({'error': f'Invalid schedule: {e}'}) @@ -3898,7 +3901,7 @@ async def update_automation( is_active=automation.is_active, ) - updated = await Automations.update_by_id(automation_id, form, next_run_ns(new_rrule, tz=tz)) + updated = await Automations.update_by_id(automation_id, form, await next_run_ns(new_rrule, tz=tz)) return JSONCodec.dumps( { @@ -3909,7 +3912,7 @@ async def update_automation( 'model_id': new_model_id, 'target': updated.data.get('target'), 'is_active': updated.is_active, - 'next_runs': next_n_runs_ns(new_rrule, tz=tz), + 'next_runs': await next_n_runs_ns(new_rrule, tz=tz), }, ensure_ascii=False, ) @@ -3977,7 +3980,7 @@ async def list_automations( 'rrule': rrule, 'is_active': item.is_active, 'last_run_at': item.last_run_at, - 'next_runs': next_n_runs_ns(rrule, tz=user.timezone if user else None), + 'next_runs': await next_n_runs_ns(rrule, tz=user.timezone if user else None), } ) @@ -4024,7 +4027,7 @@ async def toggle_automation( rrule = automation.data.get('rrule', '') toggled = await Automations.toggle( automation_id, - next_run_ns(rrule, tz=user.timezone if user else None), + await next_run_ns(rrule, tz=user.timezone if user else None), ) return JSONCodec.dumps( diff --git a/backend/open_webui/utils/access_control/__init__.py b/backend/open_webui/utils/access_control/__init__.py index 0853597b13ce..4418fc45a30c 100644 --- a/backend/open_webui/utils/access_control/__init__.py +++ b/backend/open_webui/utils/access_control/__init__.py @@ -179,6 +179,40 @@ async def has_connection_access( return await has_access(user.id, 'read', access_grants, user_group_ids) +async def has_arena_model_access( + user: UserModel, + model: dict, + user_group_ids: set[str] | None = None, + db: AsyncSession | None = None, +) -> bool: + """ + Check if a user can access an arena model based on ``info.meta.access_grants``. + + Arena models are config-driven (not DB-owned). Empty grants are private to + admins, matching Open Terminal / ``has_connection_access``. + + - Admin with BYPASS_ADMIN_ACCESS_CONTROL → always allowed + - Missing, None, or empty access_grants → private, admin-only + - access_grants has entries → delegates to ``has_access`` + """ + from open_webui.config import BYPASS_ADMIN_ACCESS_CONTROL + + if user.role == 'admin' and BYPASS_ADMIN_ACCESS_CONTROL: + return True + + access_grants = ((model.get('info') or {}).get('meta') or {}).get('access_grants', []) + if not access_grants: + # No grants configured → private, admin-only: admins must keep access + # to arena models only they can configure, even when they do not bypass + # access control globally. + return user.role == 'admin' + + if user_group_ids is None: + user_group_ids = {group.id for group in await Groups.get_groups_by_member_id(user.id, db=db)} + + return await has_access(user.id, 'read', access_grants, user_group_ids, db=db) + + def migrate_access_control(data: dict, ac_key: str = 'access_control', grants_key: str = 'access_grants') -> None: """ Auto-migrate a config dict in-place from legacy access_control dict to access_grants list. diff --git a/backend/open_webui/utils/access_control/files.py b/backend/open_webui/utils/access_control/files.py index 66cf3d44841d..f4859acb9584 100644 --- a/backend/open_webui/utils/access_control/files.py +++ b/backend/open_webui/utils/access_control/files.py @@ -84,9 +84,7 @@ async def has_access_to_file( # Check if the file is directly attached to a shared workspace model (per the ownership # note above, model write is conferred only for files the model owner owns). - model_owners = await Models.get_model_owner_ids_by_file_id( - file.id, db=db, include_background=access_type == 'read' - ) + model_owners = await Models.get_model_owner_ids_by_file_id(file.id, db=db, include_background=access_type == 'read') if access_type != 'read': model_owners = {model_id: owner_id for model_id, owner_id in model_owners.items() if owner_id == file.user_id} if user.id in model_owners.values(): diff --git a/backend/open_webui/utils/anthropic.py b/backend/open_webui/utils/anthropic.py index 36237bdaeb81..bd2cea2e61d7 100644 --- a/backend/open_webui/utils/anthropic.py +++ b/backend/open_webui/utils/anthropic.py @@ -432,8 +432,8 @@ def convert_anthropic_to_openai_payload( else: openai_payload[param] = anthropic_payload[param] - # Tools conversion: Anthropic → OpenAI - if 'tools' in anthropic_payload: + # Tools conversion: Anthropic → OpenAI (backends reject an empty tools array) + if anthropic_payload.get('tools'): openai_tools = [] for tool in anthropic_payload['tools']: openai_tools.append( @@ -452,7 +452,7 @@ def convert_anthropic_to_openai_payload( openai_payload['tools'] = openai_tools # tool_choice - if 'tool_choice' in anthropic_payload: + if 'tool_choice' in anthropic_payload and 'tools' in openai_payload: tool_choice = anthropic_payload['tool_choice'] if isinstance(tool_choice, dict): tool_choice_type = tool_choice.get('type', 'auto') @@ -616,6 +616,7 @@ async def openai_stream_to_anthropic_stream(openai_stream_generator, model: str server_tool_use = None service_tier = None stop_reason = 'end_turn' + error_message = None # Track content blocks with a running index. # Each text block or tool_use block gets its own index. @@ -671,6 +672,13 @@ async def openai_stream_to_anthropic_stream(openai_stream_generator, model: str except (JSONCodec.JSONDecodeError, TypeError): continue + error = data.get('error') + if error: + error_message = ( + error.get('message') if isinstance(error, dict) else error + ) or 'Chat completion stream failed' + break + usage_data = data.get('usage') if isinstance(usage_data, dict): cache_creation = usage_data.get('cache_creation_input_tokens') @@ -904,8 +912,18 @@ async def openai_stream_to_anthropic_stream(openai_stream_generator, model: str } stop_reason = stop_reason_map.get(finish_reason, 'end_turn') + if error_message: + break + except Exception as e: log.error(f'Error in Anthropic stream conversion: {e}') + error_message = 'Chat completion stream failed' + + # Skip message_stop so a failed stream is not reported as complete. + if error_message: + error_event = {'type': 'error', 'error': {'type': 'api_error', 'message': error_message}} + yield f'event: error\ndata: {JSONCodec.dumps(error_event)}\n\n'.encode() + return # Close any open thinking block if thinking_block_open: diff --git a/backend/open_webui/utils/automations.py b/backend/open_webui/utils/automations.py index 7df4c399aa56..be5d6868f1df 100644 --- a/backend/open_webui/utils/automations.py +++ b/backend/open_webui/utils/automations.py @@ -20,12 +20,10 @@ import os import random import time -from datetime import datetime, timedelta +from datetime import timedelta from typing import Optional from uuid import uuid4 -from zoneinfo import ZoneInfo -from dateutil.rrule import HOURLY, MINUTELY, SECONDLY, rruleset, rrulestr from fastapi import Request from fastapi.security import HTTPAuthorizationCredentials from open_webui.constants import ERROR_MESSAGES @@ -39,6 +37,14 @@ from open_webui.models.users import Users from open_webui.utils.auth import create_token from open_webui.utils.misc import parse_duration +from open_webui.utils.models import get_all_models +from open_webui.utils.recurrence import ( + _resolve_tz, + next_n_runs_ns, + next_run_ns, + rrule_interval_seconds, + validate_rrule, +) from open_webui.utils.task import prompt_template from open_webui.utils.terminals import get_terminal_server_url from starlette.datastructures import Headers @@ -50,153 +56,6 @@ CALENDAR_ALERT_LOOKAHEAD_MINUTES = int(os.getenv('CALENDAR_ALERT_LOOKAHEAD_MINUTES', '10')) -#################### -# RRULE Helpers -#################### - - -def _resolve_tz(tz: str = None) -> Optional[ZoneInfo]: - """Safely resolve a timezone string to ZoneInfo. - - Returns None (→ server-local fallback) when *tz* is empty, None, - or an unrecognised IANA key. Logs a warning on bad keys so - misconfiguration is visible in the server logs. - """ - if not tz: - return None - try: - return ZoneInfo(tz) - except (KeyError, Exception): - log.warning('Unknown timezone %r — falling back to server time', tz) - return None - - -def _parse_rule(s: str, now: Optional[datetime] = None): - """Parse RRULE with clock-aligned DTSTART for sub-daily frequencies. - - SECONDLY/MINUTELY/HOURLY rules use a fixed epoch DTSTART (2000-01-01 00:00) - so intervals snap to clock boundaries (e.g. every 5min = :00, :05, :10). - """ - upper = s.upper() - if 'EXRULE' in upper: - raise ValueError('EXRULE is not supported in recurrence rules') - - parsed = rrulestr(s, ignoretz=True) - rules = parsed._rrule if isinstance(parsed, rruleset) else [parsed] - if len(rules) > 1: - raise ValueError('only one RRULE is supported per recurrence rule') - - rule = rules[0] - start = rule._dtstart.replace(tzinfo=None) - anchor = now or datetime.now() - lines = s.splitlines() - stripped = '\n'.join(line for line in lines if not line.upper().startswith('DTSTART')) or s - has_dtstart = any(line.upper().startswith('DTSTART') for line in lines) - step = { - SECONDLY: timedelta(seconds=rule._interval), - MINUTELY: timedelta(minutes=rule._interval), - HOURLY: timedelta(hours=rule._interval), - }.get(rule._freq) - - if step is None: - if not rule._dtstart.tzinfo: - return parsed - return rrulestr(stripped, dtstart=start, ignoretz=True) - - if rule._interval < 1: - raise ValueError('RRULE INTERVAL must be a positive integer') - dtstart = None - if has_dtstart: - emitted = ((anchor - start) // step) if anchor > start else 0 - emitted *= len(rule._byminute or (0,)) * len(rule._bysecond or (0,)) - if emitted <= 100_000: - if rule._dtstart.tzinfo: - dtstart = start - else: - return parsed - if not has_dtstart or dtstart is None: - epoch = datetime(2000, 1, 1) - dtstart = epoch + ((anchor - epoch) // step) * step - - return rrulestr(stripped, dtstart=dtstart, ignoretz=True) - - -def validate_rrule(s: str, tz: str = None) -> None: - """Raise ValueError if the RRULE is malformed or exhausted. - - When *tz* is provided the "now" reference uses the user's local - clock so that near-future schedules are not incorrectly rejected - on servers whose system clock is ahead (e.g. UTC vs US timezones). - """ - upper = s.upper() - if 'COUNT=' in upper and 'DTSTART' not in upper: - raise ValueError(ERROR_MESSAGES.AUTOMATION_COUNT_REQUIRES_DTSTART) - zi = _resolve_tz(tz) - now = datetime.now(zi).replace(tzinfo=None) if zi else datetime.now() - try: - rule = _parse_rule(s, now) - except Exception as e: - raise ValueError(ERROR_MESSAGES.AUTOMATION_INVALID_RRULE(e)) - if rule.after(now) is None: - raise ValueError(ERROR_MESSAGES.AUTOMATION_NO_FUTURE_RUNS) - - -def next_run_ns(s: str, tz: str = None) -> Optional[int]: - """Next occurrence as epoch nanoseconds, respecting user timezone.""" - zi = _resolve_tz(tz) - now = datetime.now(zi) if zi else datetime.now() - now_naive = now.replace(tzinfo=None) - dt = _parse_rule(s, now_naive).after(now_naive) - if dt is None: - return None - if zi: - dt = dt.replace(tzinfo=zi) - return int(dt.timestamp() * 1_000_000_000) - - -def next_n_runs_ns(s: str, n: int = 5, tz: str = None) -> list[int]: - """Compute next N occurrences for UI preview. - - Uses the user's timezone for the starting "now" so that the - preview matches the user's local clock (same as next_run_ns). - """ - zi = _resolve_tz(tz) - result = [] - now = datetime.now(zi).replace(tzinfo=None) if zi else datetime.now() - rule = _parse_rule(s, now) - dt = now - for _ in range(n): - dt = rule.after(dt) - if not dt: - break - if zi: - dt_tz = dt.replace(tzinfo=zi) - result.append(int(dt_tz.timestamp() * 1_000_000_000)) - else: - result.append(int(dt.timestamp() * 1_000_000_000)) - return result - - -def rrule_interval_seconds(s: str) -> Optional[int]: - """Approximate interval between recurrences in seconds. - - Returns None for one-shot (COUNT=1) schedules or rules - with fewer than two future occurrences. - """ - if 'COUNT=1' in s: - return None - s = '\n'.join(line for line in s.splitlines() if not line.upper().startswith('DTSTART')) or s - now = datetime.now() - rule = _parse_rule(s, now) - first = rule.after(now) - if first is None: - return None - second = rule.after(first) - if second is None: - return None - return int((second - first).total_seconds()) - - ############################ # Worker Loop ############################ @@ -532,6 +391,9 @@ async def execute_automation(app, automation: AutomationModel) -> None: expires_delta=expires_delta or timedelta(hours=1), ) + if not app.state.MODELS: + await get_all_models(_build_request(app, token=token), user=user) + target = automation.data.get('target') or {} if target.get('type') == 'channel': await _execute_channel_automation(app, automation, user, prompt, model_id, token) diff --git a/backend/open_webui/utils/calendar.py b/backend/open_webui/utils/calendar.py index 1ce590203a7a..0406db38f6af 100644 --- a/backend/open_webui/utils/calendar.py +++ b/backend/open_webui/utils/calendar.py @@ -44,7 +44,7 @@ def to_local_datetime(timestamp_ns: int) -> dt.datetime: original_start_ns = event_dict['start_at'] original_start = to_local_datetime(original_start_ns) - rule_str = '\n'.join(line for line in rrule_str.splitlines() if not line.upper().startswith('DTSTART')) or rrule_str + rule_str = '\n'.join(part for part in rrule_str.split() if not part.upper().startswith('DTSTART')) or rrule_str try: # Anchor to the event's real start so day-of-week / day-of-month are correct diff --git a/backend/open_webui/utils/chat.py b/backend/open_webui/utils/chat.py index 0b64a52165c0..c97985dde4ac 100644 --- a/backend/open_webui/utils/chat.py +++ b/backend/open_webui/utils/chat.py @@ -23,9 +23,9 @@ process_pipeline_outlet_filter, ) from open_webui.socket.main import ( + EVENT_QUEUES, get_event_call, get_event_emitter, - sio, ) from open_webui.utils.filter import ( get_filter_functions, @@ -33,7 +33,7 @@ ) from open_webui.utils.json_codec import JSONCodec from open_webui.utils.models import check_model_access, get_all_models -from open_webui.utils.payload import convert_payload_openai_to_ollama +from open_webui.utils.payload import apply_system_prompt_to_body, convert_payload_openai_to_ollama from open_webui.utils.response import ( convert_response_ollama_to_openai, convert_streaming_response_ollama_to_openai, @@ -71,19 +71,8 @@ async def generate_direct_chat_completion( logging.info('WebSocket channel: %s', channel) if form_data.get('stream'): - q = asyncio.Queue() - - async def message_listener(sid, data): - """ - Handle received socket messages and push them into the queue. - """ - await q.put(data) - - def remove_message_listener(): - sio.handlers['/'].pop(channel, None) - - # Register the listener - sio.on(channel, message_listener) + queue = asyncio.Queue() + EVENT_QUEUES[channel] = queue # Start processing chat completion in background try: @@ -103,16 +92,15 @@ def remove_message_listener(): status = res.get('status', False) except BaseException: - remove_message_listener() + EVENT_QUEUES.pop(channel, None) raise if status: # Define a generator to stream responses async def event_generator(): - nonlocal q try: while True: - data = await q.get() # Wait for new messages + data = await queue.get() # Wait for new messages if isinstance(data, dict): if 'done' in data and data['done']: break # Stop streaming when 'done' is received @@ -127,16 +115,16 @@ async def event_generator(): log.debug('Error in event generator: %s', e) pass finally: - remove_message_listener() + EVENT_QUEUES.pop(channel, None) # Define a background task to run the event generator async def background(): - remove_message_listener() + EVENT_QUEUES.pop(channel, None) # Return the streaming response return StreamingResponse(event_generator(), media_type='text/event-stream', background=background) else: - remove_message_listener() + EVENT_QUEUES.pop(channel, None) raise Exception(str(res)) else: res = await event_caller( @@ -293,6 +281,14 @@ async def stream_wrapper(stream): # Below does not require bypass_filter because this is the only route the uses this function and it is already bypassing the filter return await generate_function_chat_completion(request, form_data, user=user, models=models) if model.get('owned_by') == 'ollama': + # Apply before Ollama conversion so tool follow-ups keep the model system prompt + if not bypass_system_prompt: + model_info = await Models.get_model_by_id(form_data['model']) + if model_info: + system = model_info.params.model_dump().get('system') + form_data = await apply_system_prompt_to_body(system, form_data, metadata, user) + request.state.bypass_system_prompt = True + # Using /ollama/api/chat endpoint form_data = convert_payload_openai_to_ollama(form_data) response = await generate_ollama_chat_completion( diff --git a/backend/open_webui/utils/files.py b/backend/open_webui/utils/files.py index f982da167554..e9c926e5f859 100644 --- a/backend/open_webui/utils/files.py +++ b/backend/open_webui/utils/files.py @@ -75,9 +75,16 @@ async def get_image_base64_from_url(url: str, user=None) -> Optional[str]: # rebinding DNS answer that passed validate_url cannot reach an internal address. async with get_ssrf_safe_session() as session: async with session.get( - url, ssl=AIOHTTP_CLIENT_SESSION_SSL, allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS + url, + ssl=AIOHTTP_CLIENT_SESSION_SSL, + allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS, + headers={'Accept-Encoding': 'identity'}, ) as response: response.raise_for_status() + # Accept-Encoding is only a request; the sender can still compress and pick our decompressed size. + encodings = response.headers.getall('Content-Encoding', ()) + if any(encoding.lower() not in ('', 'identity') for encoding in encodings): + return None image_data = bytearray() total = 0 async for chunk in response.content.iter_chunked(64 * 1024): diff --git a/backend/open_webui/utils/headers.py b/backend/open_webui/utils/headers.py index 91cd0888348f..75368a43efda 100644 --- a/backend/open_webui/utils/headers.py +++ b/backend/open_webui/utils/headers.py @@ -5,7 +5,10 @@ from urllib.parse import quote import jwt +from fastapi import Request from open_webui.env import ( + ENABLE_FORWARD_USER_INFO_HEADERS, + FORWARD_SESSION_INFO_HEADER_CHAT_ID, FORWARD_USER_INFO_HEADER_AUTH_TYPE, FORWARD_USER_INFO_HEADER_JWT, FORWARD_USER_INFO_HEADER_JWT_EXPIRES_SECONDS, @@ -161,3 +164,87 @@ def parse_custom_headers( parsed_headers[key] = quote(value, safe=punctuation + ' \t') return parsed_headers + + +async def get_headers_and_cookies( + request: Request, + url, + key=None, + config=None, + metadata: dict | None = None, + user=None, +): + config = config or {} + cookies = getattr(request, 'cookies', {}) if config.get('forward_cookies', False) else {} + headers = { + 'Content-Type': 'application/json', + **( + { + # LICENSE covers this Open WebUI upstream metadata identifier. + # Do not alter, remove, obscure, or replace it except as LICENSE permits: + # https://docs.openwebui.com/license. + 'HTTP-Referer': 'https://openwebui.com/', + 'X-Title': 'Open WebUI', + } + if 'openrouter.ai' in url + else {} + ), + } + + if ENABLE_FORWARD_USER_INFO_HEADERS and user: + headers = include_user_info_headers(headers, user, request=request) + if metadata and metadata.get('chat_id'): + headers[FORWARD_SESSION_INFO_HEADER_CHAT_ID] = metadata.get('chat_id') + + token = None + auth_type = config.get('auth_type') + + if auth_type == 'bearer' or auth_type is None: + # Default to bearer if not specified + token = key + elif auth_type == 'none': + token = None + elif auth_type == 'session': + token = request.state.token.credentials + elif auth_type == 'system_oauth': + oauth_token = None + try: + if request.cookies.get('oauth_session_id', None): + oauth_token = await request.app.state.oauth_manager.get_oauth_token( + user.id, + request.cookies.get('oauth_session_id', None), + ) + except Exception as e: + log.error(f'Error getting OAuth token: {e}') + + if oauth_token: + token = f'{oauth_token.get("access_token", "")}' + + elif auth_type in ('azure_ad', 'microsoft_entra_id'): + token = get_microsoft_entra_id_access_token() + + if token: + headers['Authorization'] = f'Bearer {token}' + + if config.get('headers') and isinstance(config.get('headers'), dict): + custom_headers = await get_custom_headers(config.get('headers'), user, metadata, request=request) + headers.update(custom_headers) + + return headers, cookies + + +def get_microsoft_entra_id_access_token(): + """ + Get Microsoft Entra ID access token using DefaultAzureCredential for Azure OpenAI. + Returns the token string or None if authentication fails. + """ + from azure.identity import DefaultAzureCredential, get_bearer_token_provider + + try: + token_provider = get_bearer_token_provider( + DefaultAzureCredential(), 'https://cognitiveservices.azure.com/.default' + ) + return token_provider() + except Exception as e: + log.error(f'Error getting Microsoft Entra ID access token: {e}') + return None diff --git a/backend/open_webui/utils/images/comfyui.py b/backend/open_webui/utils/images/comfyui.py index 11a0ebb8febf..6447a69f88b9 100644 --- a/backend/open_webui/utils/images/comfyui.py +++ b/backend/open_webui/utils/images/comfyui.py @@ -91,6 +91,7 @@ async def _ws_get_images(ws, workflow, client_id, base_url, api_key): node_output = history['outputs'][node_id] if node_id in workflow and workflow[node_id].get('class_type') in [ 'SaveImage', + 'SaveImageAdvanced', 'PreviewImage', ]: if 'images' in node_output: diff --git a/backend/open_webui/utils/memory.py b/backend/open_webui/utils/memory.py index 96eb28a6e1ca..1308c44dd823 100644 --- a/backend/open_webui/utils/memory.py +++ b/backend/open_webui/utils/memory.py @@ -8,6 +8,7 @@ from fastapi import HTTPException from open_webui.models.config import Config from open_webui.models.memories import Memories +from open_webui.utils.access_control import has_permission from open_webui.utils.json_codec import JSONCodec from open_webui.utils.misc import add_or_update_system_message, get_content_from_message @@ -428,10 +429,12 @@ async def review_memory_after_turn( return config = await Config.get_many( + 'memories.enable', 'memories.background_review.enable', 'memories.review_interval_turns', + 'user.permissions', ) - if not config.get('memories.background_review.enable'): + if not config.get('memories.enable') or not config.get('memories.background_review.enable'): return try: @@ -443,6 +446,10 @@ async def review_memory_after_turn( if user_turns == 0 or user_turns % interval != 0: return + # features is client-supplied; re-check the permission the memory routes enforce. + if user.role != 'admin' and not await has_permission(user.id, 'features.memories', config.get('user.permissions')): + return + task = asyncio.create_task( _review_memory( request=request, diff --git a/backend/open_webui/utils/middleware.py b/backend/open_webui/utils/middleware.py index ad784658c8fa..6a3e4b6f5b3e 100644 --- a/backend/open_webui/utils/middleware.py +++ b/backend/open_webui/utils/middleware.py @@ -115,6 +115,7 @@ get_last_user_message_item, get_message_list, get_output_text, + get_paired_tool_call_ids, get_response_error_detail, get_reasoning_details, get_system_message, @@ -509,6 +510,7 @@ def get_citation_source_from_tool_result( }, 'document': [], 'metadata': [], + 'distances': [], } sources_by_file[key]['document'].append(content) @@ -520,6 +522,8 @@ def get_citation_source_from_tool_result( **({'note_id': note_id} if note_id else {}), } ) + if 'distance' in chunk: + sources_by_file[key]['distances'].append(chunk['distance']) # Return all grouped sources as a list if sources_by_file: @@ -933,6 +937,9 @@ def handle_responses_streaming_event( error = data.get('response', {}).get('error', {}) return current_output, {'error': error} + elif event_type == 'error': + return current_output, {'error': data} + else: return current_output, None @@ -1195,7 +1202,6 @@ async def process_tool_result( 'chat_id': metadata.get('chat_id', None), 'message_id': metadata.get('message_id', None), 'session_id': metadata.get('session_id', None), - 'result': item, }, user, ) @@ -1768,19 +1774,10 @@ async def get_image_urls(delta_images, request, metadata, user) -> list[str]: return image_urls -async def add_file_context(messages: list, chat_id: str, user) -> list: +def add_file_context(messages: list) -> list: """ Add file URLs to messages for native function calling. """ - if not is_saved_chat_id(chat_id): - return messages - - chat = await Chats.get_chat_by_id_and_user_id(chat_id, user.id) - if not chat: - return messages - - history = chat.chat.get('history', {}) - stored_messages = get_message_list(history.get('messages', {}), history.get('currentId')) def format_file_tag(file): # Every file reaching here has a url or a chat id, so id is always set. @@ -1793,20 +1790,13 @@ def format_file_tag(file): attrs += f' name="{file["name"]}"' return f'' - # Pair only user-role messages from both lists to avoid misalignment. - # After process_messages_with_output(), assistant messages with tool calls - # are expanded into multiple messages (assistant + tool results), making - # the payload message list longer than the stored message list. A naive - # positional zip() would pair user messages with wrong stored messages, - # causing later images to lose their file context (see #21878). - user_messages = [m for m in messages if m.get('role') == 'user'] - stored_user_messages = [m for m in stored_messages if m.get('role') == 'user'] - - for message, stored_message in zip(user_messages, stored_user_messages): + for message in messages: + if message.get('role') != 'user': + continue # Chat references carry no url - they are addressed by id via view_chat. attached_files = [ file - for file in stored_message.get('files', []) + for file in message.get('files', []) if (file.get('url') and not file.get('url').startswith('data:')) or (file.get('type') == 'chat' and file.get('id')) ] @@ -2237,7 +2227,9 @@ async def load_messages_from_db(chat_id: str, message_id: str) -> Optional[list[ return [ {k: v for k, v in msg.items() if k in MESSAGE_REPLAY_KEYS} for msg in db_messages - if not (msg.get('role') == 'assistant' and msg.get('error') and not msg.get('content') and not msg.get('output')) + if not ( + msg.get('role') == 'assistant' and msg.get('error') and not msg.get('content') and not msg.get('output') + ) ] @@ -2269,6 +2261,7 @@ def strip_reasoning_details(output: list) -> list: def process_messages_with_output( messages: list[dict], reasoning_format: str | None = None, + include_file_context: bool = False, ) -> list[dict]: """ Process messages with OR-aligned output items for LLM consumption. @@ -2294,33 +2287,25 @@ def process_messages_with_output( continue clean_message = dict(message) - for key in ('id', 'files', 'output', 'model', 'contextSummary', 'context_summary', 'usage'): + for key in ('id', 'output', 'model', 'contextSummary', 'context_summary', 'usage'): clean_message.pop(key, None) processed.append(clean_message) + if include_file_context: + add_file_context(processed) + for message in processed: + message.pop('files', None) + return processed def sanitize_tool_pairs(messages: list[dict]) -> list[dict]: - tool_result_ids = { - message.get('tool_call_id') - for message in messages - if message.get('role') == 'tool' and message.get('tool_call_id') - } - - tool_call_ids = { - tool_call.get('id') - for message in messages - for tool_call in (message.get('tool_calls') or []) - if message.get('role') == 'assistant' and tool_call.get('id') - } + paired_ids_by_message = get_paired_tool_call_ids(messages) sanitized = [] - for message in messages: + for message, paired_ids in zip(messages, paired_ids_by_message): if message.get('role') == 'assistant' and message.get('tool_calls'): - kept = [ - tool_call for tool_call in message.get('tool_calls') or [] if tool_call.get('id') in tool_result_ids - ] + kept = [tool_call for tool_call in message.get('tool_calls') or [] if tool_call.get('id') in paired_ids] if kept: sanitized.append({**message, 'tool_calls': kept}) else: @@ -2329,14 +2314,12 @@ def sanitize_tool_pairs(messages: list[dict]) -> list[dict]: clean.pop('reasoning_items', None) if clean.get('content'): sanitized.append(clean) - elif message.get('role') != 'tool' or message.get('tool_call_id') in tool_call_ids: + elif message.get('role') != 'tool' or message.get('tool_call_id') in paired_ids: sanitized.append(message) return sanitized - - async def connect_mcp_server( request, server_id: str, @@ -2439,6 +2422,17 @@ async def process_chat_payload(request, form_data, user, metadata, model): # which the frontend strips, causing tool calls to be merged into content. chat_id = metadata.get('chat_id') user_message_id = metadata.get('user_message_id') + payload_tools = form_data.get('tools', None) # snapshot before filters + chat = await Chats.get_chat_by_id(chat_id) if is_saved_chat_id(chat_id) else None + is_note_chat = bool(chat and (chat.meta or {}).get('internal') is True and (chat.meta or {}).get('type') == 'note') + use_builtin_tools = is_note_chat or ( + bool(metadata.get('session_id')) + and metadata.get('params', {}).get('function_calling') != 'legacy' + and (model.get('info', {}).get('meta', {}).get('capabilities') or {}).get('builtin_tools', True) + ) + metadata['include_file_context'] = bool( + chat and chat.user_id == user.id and payload_tools is None and use_builtin_tools + ) if is_saved_chat_id(chat_id) and user_message_id: db_messages = await load_messages_from_db(chat_id, user_message_id) @@ -2475,8 +2469,6 @@ async def process_chat_payload(request, form_data, user, metadata, model): if f.get('url') ], ] - # Strip files field — it's been incorporated into content - message.pop('files', None) if regeneration_prompt: form_data['messages'].append({'role': 'user', 'content': regeneration_prompt}) @@ -2512,6 +2504,10 @@ async def process_chat_payload(request, form_data, user, metadata, model): except Exception: log.exception('Context compaction failed; continuing with full chat history') + metadata['context_start_message_id'] = next( + (message.get('id') for message in form_data.get('messages', []) if message.get('role') != 'system'), None + ) + # Process messages with OR-aligned output items for clean LLM messages for message in form_data.get('messages', []): output = message.get('output') @@ -2523,6 +2519,7 @@ async def process_chat_payload(request, form_data, user, metadata, model): form_data['messages'] = process_messages_with_output( form_data.get('messages', []), reasoning_format=get_reasoning_format(model), + include_file_context=metadata['include_file_context'], ) form_data['messages'] = sanitize_tool_pairs(form_data['messages']) @@ -2644,7 +2641,6 @@ async def process_chat_payload(request, form_data, user, metadata, model): form_data['files'] = files variables = form_data.pop('variables', None) - payload_tools = form_data.get('tools', None) # snapshot before filters # Process the form_data through the pipeline try: @@ -2777,12 +2773,6 @@ async def process_chat_payload(request, form_data, user, metadata, model): available_skills = [] terminal_skills = [] view_skill_ids = [] - chat = None - if is_saved_chat_id(metadata.get('chat_id')): - chat = await Chats.get_chat_by_id(metadata['chat_id']) - - is_note_chat = bool(chat and (chat.meta or {}).get('internal') is True and (chat.meta or {}).get('type') == 'note') - if is_note_chat: note_id = (chat.meta or {}).get('note_id') note = await Notes.get_note_by_id(note_id) if note_id else None @@ -2806,20 +2796,12 @@ async def process_chat_payload(request, form_data, user, metadata, model): if note_files: files = [*(files or []), *note_files] - use_builtin_tools = is_note_chat or ( - bool(metadata.get('session_id')) - and metadata.get('params', {}).get('function_calling') != 'legacy' - and (model.get('info', {}).get('meta', {}).get('capabilities') or {}).get('builtin_tools', True) - ) - if skill_ids or use_builtin_tools: - import aiohttp - from open_webui.env import AIOHTTP_CLIENT_SESSION_TOOL_SERVER_SSL, AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER_DATA from open_webui.models.skills import Skills as SkillsModel from open_webui.utils.terminals import ( format_terminal_skill_context, format_terminal_skill_manifest_entry, - get_terminal_request_info, + get_terminal_json, get_terminal_skill, ) @@ -2853,45 +2835,34 @@ async def process_chat_payload(request, form_data, user, metadata, model): f'{skill.description or ""}\n\n' ) - terminal_request = ( - await get_terminal_request_info(request, user, metadata, extra_params) if terminal_id or terminal_skill_ids else None + listed = ( + await get_terminal_json(request, user, metadata, '/skills', extra_params) + if terminal_id or terminal_skill_ids + else None ) - listed_terminal_skills = [] - if terminal_request: - terminal_base_url, terminal_headers, terminal_cookies = terminal_request - timeout = aiohttp.ClientTimeout(total=AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER_DATA) - async with aiohttp.ClientSession(timeout=timeout, trust_env=True) as session: - async with session.get( - f'{terminal_base_url.rstrip("/")}/skills', - headers=terminal_headers, - cookies=terminal_cookies, - ssl=AIOHTTP_CLIENT_SESSION_TOOL_SERVER_SSL, - ) as response: - if response.status == 200: - listed = await response.json() - listed_terminal_skills = listed if isinstance(listed, list) else [] - - if terminal_id and use_builtin_tools: - terminal_skills = listed_terminal_skills - elif terminal_skill_ids: - terminal_skill_map = {skill['id']: skill for skill in listed_terminal_skills} - terminal_skills = [skill for sid in terminal_skill_ids if (skill := terminal_skill_map.get(sid))] - - for skill in terminal_skills: - sid = skill['id'] - if sid in mentioned_skill_ids or not use_builtin_tools: - skill_name = unquote(sid.removeprefix(terminal_skill_prefix)) - loaded = await get_terminal_skill(request, user.model_dump(), metadata, skill_name, extra_params) - if loaded: - form_data['messages'] = add_or_update_system_message( - format_terminal_skill_context(loaded), - form_data['messages'], - append=True, - ) - else: - view_skill_ids.append(sid) - skill_manifest += format_terminal_skill_manifest_entry(skill) + listed_terminal_skills = listed if isinstance(listed, list) else [] + + if terminal_id and use_builtin_tools: + terminal_skills = listed_terminal_skills + elif terminal_skill_ids: + terminal_skill_map = {skill['id']: skill for skill in listed_terminal_skills} + terminal_skills = [skill for sid in terminal_skill_ids if (skill := terminal_skill_map.get(sid))] + + for skill in terminal_skills: + sid = skill['id'] + if sid in mentioned_skill_ids or not use_builtin_tools: + skill_name = unquote(sid.removeprefix(terminal_skill_prefix)) + loaded = await get_terminal_skill(request, user.model_dump(), metadata, skill_name, extra_params) + if loaded: + form_data['messages'] = add_or_update_system_message( + format_terminal_skill_context(loaded), + form_data['messages'], + append=True, + ) + else: + view_skill_ids.append(sid) + skill_manifest += format_terminal_skill_manifest_entry(skill) if skill_manifest: form_data['messages'] = add_or_update_system_message( @@ -3062,6 +3033,8 @@ async def tool_function(**kwargs): for tool_server in direct_tool_servers: if tool_server.get('is_terminal') is True and not terminal_capability: continue + # Copy so the pops below keep metadata intact for sub-agents and approval resumes + tool_server = dict(tool_server) system_prompt = tool_server.pop('system_prompt', None) if system_prompt: form_data['messages'] = add_or_update_system_message( @@ -3093,10 +3066,6 @@ async def tool_function(**kwargs): # Only inject when the request originates from the UI (identified by session_id). # API callers don't expect hidden tools; they can explicitly request tools via tool_ids. if use_builtin_tools: - # Add file context to user messages - chat_id = metadata.get('chat_id') - form_data['messages'] = await add_file_context(form_data.get('messages', []), chat_id, user) - if (model.get('info', {}).get('meta', {}).get('builtinTools') or {}).get('knowledge', True): from html import escape @@ -3483,6 +3452,8 @@ async def drain_approved_tool_calls(request, form_data, user, model, metadata) - output_parts.append({'type': 'input_image', 'image_url': image_url}) else: display_files.append(file_item) + if file_item.get('type') == 'image' and file_item.get('url'): + output_parts.append({'type': 'input_image', 'image_url': file_item['url']}) output.append( { @@ -3556,6 +3527,11 @@ async def drain_approved_tool_calls(request, form_data, user, model, metadata) - assistant_message = await Chats.get_message_by_id_and_message_id(chat_id, message_id) if assistant_message: db_messages.append({k: v for k, v in assistant_message.items() if k in MESSAGE_REPLAY_KEYS}) + context_start_message_id = metadata.get('context_start_message_id') + start_index = next( + (index for index, message in enumerate(db_messages) if message.get('id') == context_start_message_id), 0 + ) + db_messages = db_messages[start_index:] for message in db_messages: output = message.get('output') # reasoning_details can be model/provider-bound, so only replay them @@ -3567,9 +3543,11 @@ async def drain_approved_tool_calls(request, form_data, user, model, metadata) - ): message['output'] = strip_reasoning_details(output) + system_message = get_system_message(form_data.get('messages', [])) form_data['messages'] = process_messages_with_output( - db_messages, + [system_message, *db_messages] if system_message else db_messages, reasoning_format=get_reasoning_format(model), + include_file_context=metadata.get('include_file_context', False), ) form_data['messages'] = sanitize_tool_pairs(form_data['messages']) @@ -3624,7 +3602,7 @@ async def pause_for_tool_approval(chat_id: str, message_id: str, output: list[di if not has_pending_approval: item['status'] = 'pending' has_pending_approval = True - elif item.get('status') == 'in_progress': + elif item.get('status') in {'in_progress', 'completed'}: item['status'] = 'queued' await Chats.upsert_message_to_chat_by_id_and_message_id( @@ -3860,7 +3838,7 @@ async def background_tasks_handler(ctx): if isinstance(content, str): content = re.sub( - r']*>.*?<\/details>|!\[.*?\]\(.*?\)', + r']*>(?:(?!|!\[[^\[\]]*\]\([^()]*\)', '', content, flags=re.S | re.I, @@ -3991,7 +3969,7 @@ async def background_tasks_handler(ctx): await event_emitter( { 'type': 'chat:title', - 'data': message.get('content', user_message), + 'data': title, } ) @@ -4035,7 +4013,7 @@ async def background_tasks_handler(ctx): await review_memory_after_turn( request=request, user=user, - model=ctx['model'], + model=ctx.get('model'), metadata=metadata, form_data=form_data, assistant_message=ctx.get('assistant_message') or {}, @@ -4561,7 +4539,7 @@ def get_tag_boundaries(item, text, scanned_length): last_type = output[-1].get('type', '') if output else '' - if last_type == 'message': + if last_type == 'message' and output[-1].get('_tag_type') != content_type: # Use the output item's own text for tag detection item = output[-1] item_text = get_last_text(output) @@ -4691,14 +4669,14 @@ def get_tag_boundaries(item, text, scanned_length): # Strip start and end tags from content start_tag_pattern = _start_tag_pattern(start_tag) - block_content = re.sub(start_tag_pattern, '', block_content).strip() + block_content = re.sub(start_tag_pattern, '', block_content) end_tag_pattern = rf'{re.escape(end_tag)}' end_tag_regex = re.compile(end_tag_pattern, re.DOTALL) split_content = end_tag_regex.split(block_content, maxsplit=1) block_content = split_content[0].strip() if split_content else '' - leftover_content = split_content[1].strip() if len(split_content) > 1 else '' + leftover_content = split_content[1].lstrip() if len(split_content) > 1 else '' if block_content: # Update the item with final content @@ -4974,6 +4952,14 @@ def get_response_data_with_full_output_index(response_data: dict): **response_data, 'output_index': response_data['output_index'] + len(prior_output), } + if prior_output and isinstance(response_data.get('response'), dict): + # response.output is this round only; the response.completed reducer drops earlier rounds + return { + **response_data, + 'response': { + key: value for key, value in response_data['response'].items() if key != 'output' + }, + } return response_data async def flush_pending_delta_data(threshold: int = 0): @@ -5117,8 +5103,8 @@ async def emit_response_completion_event(response_data: dict, stream_output: lis 'data': data, } ) - # Check for Responses API events (type field starts with "response.") - elif data.get('type', '').startswith('response.'): + # Check for Responses API events + elif data.get('type', '').startswith('response.') or data.get('type', '') == 'error': response_data_type = data.get('type', '') response_data_is_delta = response_data_type.endswith('.delta') output, response_metadata = handle_responses_streaming_event(data, output) @@ -5177,6 +5163,20 @@ async def emit_response_completion_event(response_data: dict, stream_output: lis response_metadata['usage'] = usage if response_metadata.get('error'): + log.error( + 'Provider returned error (streaming): %s', response_metadata['error'] + ) + if save_to_chat: + try: + await Chats.upsert_message_to_chat_by_id_and_message_id( + metadata['chat_id'], + metadata['message_id'], + { + 'error': {'content': response_metadata['error']}, + }, + ) + except Exception: + pass await event_emitter( { 'type': 'chat:completion', @@ -5808,6 +5808,7 @@ async def emit_response_completion_event(response_data: dict, stream_output: lis if responses_api_tool_calls: tool_calls.append(_split_tool_calls(responses_api_tool_calls)) + output_start = len(prior_output) try: await stream_body_handler(response, form_data) finally: @@ -5839,6 +5840,42 @@ async def emit_response_completion_event(response_data: dict, stream_output: lis get_content_from_message(original_system_message) if original_system_message else None ) + async def emit_output(): + # Channels publish whole messages; Continue can merge into the preceding item. + snapshot = continuing or (metadata.get('chat_id') or '').startswith('channel:') + frontend_output = [] + for item in full_output() if snapshot else full_output()[output_start:]: + if item.get('type') == 'function_call_output': + # input_image parts are for the LLM only, via convert_output_to_messages + item = { + **item, + 'output': [ + part for part in item.get('output', []) if part.get('type') != 'input_image' + ], + } + frontend_output.append(item) + + if snapshot: + await event_emitter( + { + 'type': 'chat:completion', + 'data': {'output': frontend_output, 'flush': True}, + } + ) + return + + for output_index, item in enumerate(frontend_output, start=output_start): + await event_emitter( + { + 'type': 'response:completion', + 'data': { + 'type': 'response.output_item.done', + 'output_index': output_index, + 'item': item, + }, + } + ) + while tool_calls and ( max_tool_call_iterations is None or tool_call_iterations < max_tool_call_iterations ): @@ -5906,14 +5943,7 @@ async def emit_response_completion_event(response_data: dict, stream_output: lis ) return - await event_emitter( - { - 'type': 'chat:completion', - 'data': { - 'output': full_output(), - }, - } - ) + await emit_output() tools = metadata.get('tools', {}) @@ -6085,8 +6115,7 @@ async def execute_tool_call(tool_call): ) result_status_by_call_id[result.get('tool_call_id', '')] = local_output_status - # Separate image data URIs (for LLM via input_image) from - # other files (for frontend display via files attribute). + # Data-URI images: LLM only. File-URL images: LLM and frontend. Other files: frontend. display_files = [] for file_item in result.get('files', []): if file_item.get('type') == 'image' and file_item.get('url', '').startswith('data:'): @@ -6094,8 +6123,9 @@ async def execute_tool_call(tool_call): image_url = await store_tool_result_image(request, file_item['url'], metadata, user) output_parts.append({'type': 'input_image', 'image_url': image_url}) else: - # Frontend display (MCP images, audio, etc.) display_files.append(file_item) + if file_item.get('type') == 'image' and file_item.get('url'): + output_parts.append({'type': 'input_image', 'image_url': file_item['url']}) output.append( { @@ -6180,25 +6210,7 @@ async def execute_tool_call(tool_call): ) tool_call_sources.clear() - # Strip input_image parts (large base64 data URIs) from the - # output sent to the frontend — they're only for LLM consumption - # via convert_output_to_messages. - frontend_output = [] - for item in full_output(): - if item.get('type') == 'function_call_output': - parts = item.get('output', []) - if any(p.get('type') == 'input_image' for p in parts): - item = {**item, 'output': [p for p in parts if p.get('type') != 'input_image']} - frontend_output.append(item) - - await event_emitter( - { - 'type': 'chat:completion', - 'data': { - 'output': frontend_output, - }, - } - ) + await emit_output() try: new_form_data = { @@ -6297,6 +6309,7 @@ async def execute_tool_call(tool_call): if not msg_parts or (len(msg_parts) == 1 and not msg_parts[0].get('text', '').strip()): prior_output.pop() output = [] + output_start = len(prior_output) await stream_body_handler(res, new_form_data) output = full_output() prior_output = [] diff --git a/backend/open_webui/utils/misc.py b/backend/open_webui/utils/misc.py index ad959d3bc790..5c5dcce3a6c4 100644 --- a/backend/open_webui/utils/misc.py +++ b/backend/open_webui/utils/misc.py @@ -261,6 +261,26 @@ def get_output_text(output: list | None) -> str: return '\n'.join(texts) +def get_paired_tool_call_ids(messages: list[dict]) -> list[set[str]]: + """Tool call ids answered within each message's assistant-plus-tool-results block.""" + paired_ids_by_message = [set() for _ in messages] + for index, message in enumerate(messages): + if message.get('role') != 'assistant' or not message.get('tool_calls'): + continue + + block_end = index + 1 + while block_end < len(messages) and messages[block_end].get('role') == 'tool': + block_end += 1 + + requested_ids = {tool_call.get('id') for tool_call in message['tool_calls'] if tool_call.get('id')} + completed_ids = {tool_message.get('tool_call_id') for tool_message in messages[index + 1 : block_end]} + paired_ids = requested_ids & completed_ids + for block_index in range(index, block_end): + paired_ids_by_message[block_index] = paired_ids + + return paired_ids_by_message + + def reconcile_tool_pairs(messages: list[dict]) -> list[dict]: """Drop unpaired tool_use / tool_result from a reconstructed conversation. @@ -271,22 +291,14 @@ def reconcile_tool_pairs(messages: list[dict]) -> list[dict]: Well-formed output is unaffected: every id pairs, so nothing is stripped. """ - completed_tool_call_ids = { - message['tool_call_id'] for message in messages if message.get('role') == 'tool' and message.get('tool_call_id') - } - requested_tool_call_ids = { - tool_call['id'] - for message in messages - for tool_call in message.get('tool_calls') or () - if message.get('role') == 'assistant' and tool_call.get('id') - } + paired_ids_by_message = get_paired_tool_call_ids(messages) reconciled_messages = [] - for message in messages: + for message, paired_ids in zip(messages, paired_ids_by_message): role = message.get('role') - # Orphan tool result — no assistant ever claimed this call_id. - if role == 'tool' and message.get('tool_call_id') not in requested_tool_call_ids: + # Orphan tool result: not claimed by the assistant heading its tool block. + if role == 'tool' and message.get('tool_call_id') not in paired_ids: continue # Non-assistant or no tool_calls — pass through unchanged. @@ -294,10 +306,8 @@ def reconcile_tool_pairs(messages: list[dict]) -> list[dict]: reconciled_messages.append(message) continue - # Keep only tool_calls whose id received a tool-role response. - valid_tool_calls = [ - tool_call for tool_call in message['tool_calls'] if tool_call.get('id') in completed_tool_call_ids - ] + # Keep only tool_calls whose id received an adjacent tool-role response. + valid_tool_calls = [tool_call for tool_call in message['tool_calls'] if tool_call.get('id') in paired_ids] if valid_tool_calls: reconciled_messages.append({**message, 'tool_calls': valid_tool_calls}) diff --git a/backend/open_webui/utils/models.py b/backend/open_webui/utils/models.py index 3926bfa9be7e..2ac1ba7c514a 100644 --- a/backend/open_webui/utils/models.py +++ b/backend/open_webui/utils/models.py @@ -19,7 +19,7 @@ from open_webui.models.users import UserModel from open_webui.routers import ollama, openai from open_webui.socket.utils import RedisDict -from open_webui.utils.access_control import has_access, has_base_model_access +from open_webui.utils.access_control import has_arena_model_access, has_base_model_access from open_webui.utils.json_codec import JSONCodec from open_webui.utils.plugin import ( get_functions_cache, @@ -182,11 +182,23 @@ async def get_all_models(request, refresh: bool = False, user: UserModel = None) if model: if custom_model.is_active: + arena_meta = model['info']['meta'] if model.get('arena') else None model['name'] = custom_model.name model['info'] = custom_model.model_dump() + if arena_meta: + # Evaluation config owns arena access grants and model_ids + model['info']['meta'].update( + { + key: arena_meta[key] + for key in ('access_grants', 'model_ids', 'filter_mode') + if key in arena_meta + } + ) schema = get_chat_variables_schema(custom_model.params.model_dump().get('system')) if schema: model['info'].setdefault('meta', {})['chat_variables_schema'] = schema + elif isinstance(model['info'].get('meta'), dict): + model['info']['meta'].pop('chat_variables_schema', None) action_ids = [] filter_ids = [] @@ -239,6 +251,8 @@ async def get_all_models(request, refresh: bool = False, user: UserModel = None) schema = get_chat_variables_schema(custom_model.params.model_dump().get('system')) if schema: info.setdefault('meta', {})['chat_variables_schema'] = schema + elif isinstance(info.get('meta'), dict): + info['meta'].pop('chat_variables_schema', None) if 'params' in info: # Remove params to avoid exposing sensitive info del info['params'] @@ -460,14 +474,7 @@ def get_action_priority(action_id): async def check_model_access(user, model, model_info=None, db=None): if model.get('arena'): - meta = model.get('info', {}).get('meta', {}) - access_grants = meta.get('access_grants', []) - if not await has_access( - user.id, - permission='read', - access_grants=access_grants, - db=db, - ): + if not await has_arena_model_access(user, model, db=db): log.warning( 'Model access denied: user_id=%r model_id=%r reason=arena_read_denied', user.id, @@ -545,13 +552,11 @@ async def get_filtered_models(models, user, db=None): filtered_models = [] for model in models: if model.get('arena'): - meta = model.get('info', {}).get('meta', {}) - access_grants = meta.get('access_grants', []) - if await has_access( - user.id, - permission='read', - access_grants=access_grants, + if await has_arena_model_access( + user, + model, user_group_ids=user_group_ids, + db=db, ): filtered_models.append(model) continue diff --git a/backend/open_webui/utils/oauth.py b/backend/open_webui/utils/oauth.py index 10dedb1e6c1a..061737a35b19 100644 --- a/backend/open_webui/utils/oauth.py +++ b/backend/open_webui/utils/oauth.py @@ -72,6 +72,7 @@ ENABLE_OAUTH_ID_TOKEN_COOKIE, OAUTH_CLIENT_INFO_ENCRYPTION_KEY, OAUTH_MAX_SESSIONS_PER_USER, + REDIS_KEY_PREFIX, WEBUI_AUTH_COOKIE_SAME_SITE, WEBUI_AUTH_COOKIE_SECURE, ) @@ -188,7 +189,7 @@ def _default_value(value): return getattr(value, 'value', value) -def _get_roles_claim(claims: dict, claim: str) -> list | str | int | None: +def _get_claim(claims: dict, claim: str) -> list | str | int | None: """Read nested or flat claims, preserving explicit empty values and zero.""" value = claims for key in claim.split('.'): @@ -351,6 +352,19 @@ def is_in_blocked_groups(group_name: str, groups: list) -> bool: return False +def _parse_blocked_groups(value) -> list[str]: + """Accept JSON arrays, persisted lists, and comma-separated admin input.""" + if isinstance(value, str): + try: + parsed = JSONCodec.loads(value) + except JSONCodec.JSONDecodeError: + parsed = None + value = parsed if isinstance(parsed, list) else [group.strip() for group in value.split(',')] + if not isinstance(value, list): + return [] + return [group for group in value if isinstance(group, str) and group] + + def get_parsed_and_base_url(server_url) -> tuple[urllib.parse.ParseResult, str]: parsed = urllib.parse.urlparse(server_url) base_url = f'{parsed.scheme}://{parsed.netloc}' @@ -598,6 +612,8 @@ async def get_oauth_client_info_with_dynamic_client_registration( oauth_client_info = OAuthClientInformationFull.model_validate( { **registration_response_json, + # RFC 7591: the server may omit scope; keep the requested one. + 'scope': registration_response_json.get('scope') or oauth_client_metadata.scope, 'issuer': oauth_server_metadata_url, 'server_metadata': oauth_server_metadata, 'resource': resource, @@ -785,10 +801,7 @@ def build_oauth_request_params(client_info: OAuthClientInformationFull | None) - return params -async def recover_static_oauth_client_metadata(connection: dict, oauth_client_info: dict) -> dict: - if connection.get('auth_type') != 'oauth_2.1_static': - return oauth_client_info - +async def recover_oauth_client_metadata(connection: dict, oauth_client_info: dict) -> dict: if oauth_client_info.get('scope') and oauth_client_info.get('resource'): return oauth_client_info @@ -799,13 +812,13 @@ async def recover_static_oauth_client_metadata(connection: dict, oauth_client_in try: resource_metadata = await get_protected_resource_metadata(server_url) except Exception as e: - log.debug('Unable to recover static OAuth metadata for %s: %s', server_url, e) + log.debug('Unable to recover OAuth metadata for %s: %s', server_url, e) return oauth_client_info recovered = {**oauth_client_info} if not recovered.get('scope') and resource_metadata.scopes_supported: recovered['scope'] = ' '.join(resource_metadata.scopes_supported) - log.info('Recovered static OAuth scopes for %s from protected resource metadata', server_url) + log.info('Recovered OAuth scopes for %s from protected resource metadata', server_url) if not recovered.get('resource') and resource_metadata.resource: recovered['resource'] = resource_metadata.resource @@ -902,7 +915,7 @@ async def ensure_client_from_config(self, client_id): try: oauth_client_info = resolve_oauth_client_info(connection) - oauth_client_info = await recover_static_oauth_client_metadata(connection, oauth_client_info) + oauth_client_info = await recover_oauth_client_metadata(connection, oauth_client_info) oauth_client_info = apply_connection_oauth_options(connection, oauth_client_info) return self.add_client(expected_client_id, OAuthClientInformationFull(**oauth_client_info))['client'] except InvalidToken: @@ -1320,6 +1333,7 @@ def __init__(self, app): self.app = app self._clients = {} + self._refresh_locks: dict[str, asyncio.Lock] = {} for name, provider_config in OAUTH_PROVIDERS.items(): if 'register' not in provider_config: @@ -1415,22 +1429,35 @@ async def _refresh_token(self, session) -> dict: Returns: dict: Refreshed token data, or None if refresh failed """ - try: - # Perform the actual refresh - refreshed_token = await self._perform_token_refresh(session) + redis = self.app.state.redis + if redis: + # Shared across workers and replicas + refresh_lock = redis.lock(f'{REDIS_KEY_PREFIX}:oauth:refresh_lock:{session.id}', timeout=60) + else: + refresh_lock = self._refresh_locks.setdefault(session.id, asyncio.Lock()) - if refreshed_token: - # Update the session with new token data - session = await OAuthSessions.update_session_by_id(session.id, refreshed_token) - log.info('Successfully refreshed token for session %s', session.id) - return session.token - else: - log.error(f'Failed to refresh token for session {session.id}') - return None + async with refresh_lock: + # Another request may have refreshed while we waited; its refresh token is now spent + current_session = await OAuthSessions.get_session_by_id(session.id) + if current_session and current_session.token != session.token: + return current_session.token - except Exception as e: - log.error(f'Error refreshing token for session {session.id}: {e}') - return None + try: + # Perform the actual refresh + refreshed_token = await self._perform_token_refresh(session) + + if refreshed_token: + # Update the session with new token data + session = await OAuthSessions.update_session_by_id(session.id, refreshed_token) + log.info('Successfully refreshed token for session %s', session.id) + return session.token + else: + log.error(f'Failed to refresh token for session {session.id}') + return None + + except Exception as e: + log.error(f'Error refreshing token for session {session.id}: {e}') + return None async def _perform_token_refresh(self, session) -> dict: """ @@ -1515,7 +1542,7 @@ async def _perform_token_refresh(self, session) -> dict: log.error(f'Exception during token refresh for provider {provider}: {e}') return None - async def get_user_role(self, user, user_data, *, access_token: str | None = None): + async def get_user_role(self, user, user_data, *, token_claims: dict | None = None): auth_config = await get_oauth_runtime_config() user_count = await Users.get_num_users() if user and user_count == 1: @@ -1540,14 +1567,9 @@ async def get_user_role(self, user, user_data, *, access_token: str | None = Non role = user.role if user else auth_config.DEFAULT_USER_ROLE if oauth_claim: - claim_data = _get_roles_claim(user_data, oauth_claim) - if claim_data is None and access_token is not None: - # The exchange endpoint has already validated this token with the provider's userinfo endpoint. - try: - token_claims = jwt.decode(access_token, options={'verify_signature': False}) - claim_data = _get_roles_claim(token_claims, oauth_claim) - except jwt.PyJWTError as e: - log.debug('Token exchange: cannot decode token claims: %s', e) + claim_data = _get_claim(user_data, oauth_claim) + if claim_data is None and token_claims is not None: + claim_data = _get_claim(token_claims, oauth_claim) if isinstance(claim_data, list): oauth_roles = claim_data @@ -1560,7 +1582,7 @@ async def get_user_role(self, user, user_data, *, access_token: str | None = Non elif isinstance(claim_data, int): oauth_roles = [str(claim_data)] - if access_token is not None and not oauth_roles and oauth_allowed_roles and '*' not in oauth_allowed_roles: + if token_claims is not None and not oauth_roles and oauth_allowed_roles and '*' not in oauth_allowed_roles: log.warning('Token exchange denied: no readable roles claim in userinfo or the token') raise HTTPException(status.HTTP_403_FORBIDDEN, detail=ERROR_MESSAGES.ACCESS_PROHIBITED) @@ -1610,10 +1632,10 @@ async def update_user_role_from_oauth( user_data, provider, *, - access_token: str | None = None, + token_claims: dict | None = None, db=None, ): - determined_role = await self.get_user_role(user, user_data, access_token=access_token) + determined_role = await self.get_user_role(user, user_data, token_claims=token_claims) if user.role == determined_role: return user @@ -1631,24 +1653,20 @@ async def update_user_role_from_oauth( return user - async def update_user_groups(self, request, user, user_data, default_permissions, db=None): + async def update_user_groups( + self, request, user, user_data, default_permissions, db=None, *, token_claims: dict | None = None + ): auth_config = await get_oauth_runtime_config() log.debug('Running OAUTH Group management') oauth_claim = auth_config.OAUTH_GROUPS_CLAIM - try: - blocked_groups = JSONCodec.loads(auth_config.OAUTH_BLOCKED_GROUPS) - except Exception as e: - log.exception(f'Error loading OAUTH_BLOCKED_GROUPS: {e}') - blocked_groups = [] + blocked_groups = _parse_blocked_groups(auth_config.OAUTH_BLOCKED_GROUPS) user_oauth_groups = [] - # Nested claim search for groups claim if oauth_claim: - claim_data = user_data - nested_claims = oauth_claim.split('.') - for nested_claim in nested_claims: - claim_data = claim_data.get(nested_claim, {}) + claim_data = _get_claim(user_data, oauth_claim) + if claim_data is None and token_claims is not None: + claim_data = _get_claim(token_claims, oauth_claim) if isinstance(claim_data, list): user_oauth_groups = claim_data @@ -1675,7 +1693,7 @@ async def update_user_groups(self, request, user, user_data, default_permissions log.debug('Using creator ID %s for potential group creation.', creator_id) for group_name in user_oauth_groups: - if group_name not in all_group_names: + if group_name not in all_group_names and not is_in_blocked_groups(group_name, blocked_groups): log.info("Group '%s' not found via OAuth claim. Creating group...", group_name) try: new_group_form = GroupForm( diff --git a/backend/open_webui/utils/payload.py b/backend/open_webui/utils/payload.py index 7ee2f9d3b350..94a9bc47697f 100644 --- a/backend/open_webui/utils/payload.py +++ b/backend/open_webui/utils/payload.py @@ -379,10 +379,6 @@ def convert_payload_openai_to_ollama(openai_payload: dict) -> dict: if 'tools' in openai_payload: ollama_payload['tools'] = openai_payload['tools'] - if 'max_tokens' in openai_payload: - ollama_payload['num_predict'] = openai_payload['max_tokens'] - del openai_payload['max_tokens'] - # If there are advanced parameters in the payload, format them in Ollama's options field if openai_payload.get('options'): # Copied before key deletions below so the caller's options stay intact @@ -430,6 +426,11 @@ def parse_json(value: str) -> dict: ollama_options['stop'] = openai_payload.get('stop') ollama_payload['options'] = ollama_options + if 'max_tokens' in openai_payload: + ollama_options = ollama_payload.get('options', {}) + ollama_options['num_predict'] = openai_payload['max_tokens'] + ollama_payload['options'] = ollama_options + if 'metadata' in openai_payload: ollama_payload['metadata'] = openai_payload['metadata'] diff --git a/backend/open_webui/utils/recurrence.py b/backend/open_webui/utils/recurrence.py new file mode 100644 index 000000000000..c4817eca07b7 --- /dev/null +++ b/backend/open_webui/utils/recurrence.py @@ -0,0 +1,180 @@ +"""Recurrence calculations isolated from application/DB imports for worker processes.""" + +import asyncio +import logging +from datetime import datetime, timedelta +from functools import partial +from typing import Optional +from zoneinfo import ZoneInfo + +import anyio +from anyio import fail_after, to_process, to_thread +from dateutil.rrule import HOURLY, MINUTELY, SECONDLY, rruleset, rrulestr +from open_webui.constants import ERROR_MESSAGES + +log = logging.getLogger(__name__) +RRULE_TIMEOUT_SECONDS = 2 + + +class RecurrenceEvaluationTimeout(ValueError): + """The evaluation budget expired; the schedule may still have occurrences.""" + + +def _resolve_tz(tz: str = None) -> Optional[ZoneInfo]: + """Safely resolve a timezone string to ZoneInfo. + + Returns None (→ server-local fallback) when *tz* is empty, None, + or an unrecognised IANA key. Logs a warning on bad keys so + misconfiguration is visible in the server logs. + """ + if not tz: + return None + try: + return ZoneInfo(tz) + except (KeyError, Exception): + log.warning('Unknown timezone %r — falling back to server time', tz) + return None + + +def _parse_rule(s: str, now: Optional[datetime] = None): + """Parse RRULE with clock-aligned DTSTART for sub-daily frequencies. + + SECONDLY/MINUTELY/HOURLY rules use a fixed epoch DTSTART (2000-01-01 00:00) + so intervals snap to clock boundaries (e.g. every 5min = :00, :05, :10). + """ + upper = s.upper() + if 'EXRULE' in upper: + raise ValueError('EXRULE is not supported in recurrence rules') + + parsed = rrulestr(s, ignoretz=True) + rules = parsed._rrule if isinstance(parsed, rruleset) else [parsed] + if len(rules) > 1: + raise ValueError('only one RRULE is supported per recurrence rule') + + rule = rules[0] + start = rule._dtstart.replace(tzinfo=None) + anchor = now or datetime.now() + parts = s.split() + stripped = '\n'.join(part for part in parts if not part.upper().startswith('DTSTART')) or s + has_dtstart = any(part.upper().startswith('DTSTART') for part in parts) + step = { + SECONDLY: timedelta(seconds=rule._interval), + MINUTELY: timedelta(minutes=rule._interval), + HOURLY: timedelta(hours=rule._interval), + }.get(rule._freq) + + if step is None: + if not rule._dtstart.tzinfo: + return parsed + return rrulestr(stripped, dtstart=start, ignoretz=True) + + if rule._interval < 1: + raise ValueError('RRULE INTERVAL must be a positive integer') + dtstart = None + if has_dtstart: + emitted = ((anchor - start) // step) if anchor > start else 0 + emitted *= len(rule._byminute or (0,)) * len(rule._bysecond or (0,)) + if emitted <= 100_000: + if rule._dtstart.tzinfo: + dtstart = start + else: + return parsed + if not has_dtstart or dtstart is None: + epoch = datetime(2000, 1, 1) + dtstart = epoch + ((anchor - epoch) // step) * step + + return rrulestr(stripped, dtstart=dtstart, ignoretz=True) + + +def _next_occurrences(s: str, now: datetime, n: int) -> list[datetime]: + rule = _parse_rule(s, now) + occurrences = [] + for _ in range(n): + now = rule.after(now) + if now is None: + break + occurrences.append(now) + return occurrences + + +async def _get_next_occurrences(s: str, now: datetime, n: int) -> list[datetime]: + # A result-count or date limit cannot bound work before the first match. + try: + with fail_after(RRULE_TIMEOUT_SECONDS): + return await to_process.run_sync(_next_occurrences, s, now, n, cancellable=True) + except TimeoutError as e: + raise RecurrenceEvaluationTimeout('Schedule took too long to evaluate; simplify its recurrence rule.') from e + except NotImplementedError: + # Windows' SelectorEventLoop (required by psycopg) cannot spawn subprocesses. + run_on_proactor_loop = partial( + anyio.run, _get_next_occurrences, s, now, n, backend_options={'loop_factory': asyncio.ProactorEventLoop} + ) + return await to_thread.run_sync(run_on_proactor_loop) + + +async def validate_rrule(s: str, tz: str = None) -> None: + """Raise ValueError if the RRULE is malformed or exhausted. + + When *tz* is provided the "now" reference uses the user's local + clock so that near-future schedules are not incorrectly rejected + on servers whose system clock is ahead (e.g. UTC vs US timezones). + """ + upper = s.upper() + if 'COUNT=' in upper and 'DTSTART' not in upper: + raise ValueError(ERROR_MESSAGES.AUTOMATION_COUNT_REQUIRES_DTSTART) + zi = _resolve_tz(tz) + now = datetime.now(zi).replace(tzinfo=None) if zi else datetime.now() + try: + occurrences = await _get_next_occurrences(s, now, 1) + except RecurrenceEvaluationTimeout: + raise + except Exception as e: + raise ValueError(ERROR_MESSAGES.AUTOMATION_INVALID_RRULE(e)) + if not occurrences: + raise ValueError(ERROR_MESSAGES.AUTOMATION_NO_FUTURE_RUNS) + + +async def next_run_ns(s: str, tz: str = None) -> Optional[int]: + """Next occurrence as epoch nanoseconds, respecting user timezone.""" + zi = _resolve_tz(tz) + now = datetime.now(zi) if zi else datetime.now() + now_naive = now.replace(tzinfo=None) + occurrences = await _get_next_occurrences(s, now_naive, 1) + if not occurrences: + return None + dt = occurrences[0] + if zi: + dt = dt.replace(tzinfo=zi) + return int(dt.timestamp() * 1_000_000_000) + + +async def next_n_runs_ns(s: str, n: int = 5, tz: str = None) -> list[int]: + """Compute next N occurrences for UI preview. + + Uses the user's timezone for the starting "now" so that the + preview matches the user's local clock (same as next_run_ns). + """ + zi = _resolve_tz(tz) + result = [] + now = datetime.now(zi).replace(tzinfo=None) if zi else datetime.now() + for dt in await _get_next_occurrences(s, now, n): + if zi: + dt_tz = dt.replace(tzinfo=zi) + result.append(int(dt_tz.timestamp() * 1_000_000_000)) + else: + result.append(int(dt.timestamp() * 1_000_000_000)) + return result + + +async def rrule_interval_seconds(s: str) -> Optional[int]: + """Approximate interval between recurrences in seconds. + + Returns None for one-shot (COUNT=1) schedules or rules + with fewer than two future occurrences. + """ + s = '\n'.join(part for part in s.split() if not part.upper().startswith('DTSTART')) or s + now = datetime.now() + occurrences = await _get_next_occurrences(s, now, 2) + if len(occurrences) < 2: + return None + return int((occurrences[1] - occurrences[0]).total_seconds()) diff --git a/backend/open_webui/utils/redis.py b/backend/open_webui/utils/redis.py index 90d81abb2689..0eb1469862ff 100644 --- a/backend/open_webui/utils/redis.py +++ b/backend/open_webui/utils/redis.py @@ -35,7 +35,7 @@ _redis_sync.exceptions.ReadOnlyError, _redis_sync.exceptions.TimeoutError, ) -_FACTORY_METHODS = frozenset({'pipeline', 'pubsub', 'monitor', 'client', 'transaction'}) +_FACTORY_METHODS = frozenset({'pipeline', 'pubsub', 'monitor', 'client', 'transaction', 'lock'}) _CONNECTION_POOL: dict[tuple, Any] = {} diff --git a/backend/open_webui/utils/skills.py b/backend/open_webui/utils/skills.py index 5adc07198673..1cbe439dc177 100644 --- a/backend/open_webui/utils/skills.py +++ b/backend/open_webui/utils/skills.py @@ -32,7 +32,7 @@ def extract_skill_ids_from_messages(messages: list[dict]) -> set[str]: SKILL_MENTION_STRIP_RE = re.compile(rf'<(?:\$({SKILL_ID_RE})(?:\|([^>]*))?|/({SKILL_ID_RE})\|([^>]*))>') -SKILLS_CREATE_RE = re.compile(r"^/skills:create(?:\s+(.*))?$", re.IGNORECASE | re.DOTALL) +SKILLS_CREATE_RE = re.compile(r'^/skills:create(?:\s+(.*))?$', re.IGNORECASE | re.DOTALL) OPEN_WEBUI_SKILL_AUTHORING_STANDARDS = """\ Follow the Open WebUI skill-authoring standards: @@ -126,8 +126,7 @@ def has_prior_real_chat_content(messages: list[dict]) -> bool: len(messages), ) return any( - message.get('role') == 'user' and _message_has_real_content(message) - for message in messages[:last_user_idx] + message.get('role') == 'user' and _message_has_real_content(message) for message in messages[:last_user_idx] ) diff --git a/backend/open_webui/utils/subagents.py b/backend/open_webui/utils/subagents.py index 809dd601aecf..13b9699216ce 100644 --- a/backend/open_webui/utils/subagents.py +++ b/backend/open_webui/utils/subagents.py @@ -14,7 +14,7 @@ from open_webui.models.config import Config from open_webui.models.users import UserModel, Users from open_webui.tasks import create_task, has_active_tasks -from open_webui.utils.auth import create_token +from open_webui.utils.auth import VERIFIED_USER_ROLES, create_token from open_webui.utils.json_codec import JSONCodec from open_webui.utils.misc import get_message_list from sqlalchemy import select @@ -84,7 +84,7 @@ async def process_pending_internal_messages( return user = await Users.get_user_by_id(user_id) - if not user: + if not user or user.role not in VERIFIED_USER_ROLES: return async with get_async_db() as db: diff --git a/backend/open_webui/utils/terminals.py b/backend/open_webui/utils/terminals.py index e7d34550d320..4dbd7e6b3f71 100644 --- a/backend/open_webui/utils/terminals.py +++ b/backend/open_webui/utils/terminals.py @@ -2,6 +2,7 @@ import asyncio import logging +import ntpath import posixpath from urllib.parse import quote @@ -119,12 +120,15 @@ def terminal_chat_uploads(connection: dict) -> str: return value if value in TERMINAL_CHAT_UPLOAD_MODES else 'default' -async def get_terminal_request_info(request, user, metadata: dict, extra_params: dict | None = None): +async def get_terminal_json(request, user, metadata: dict, path: str, extra_params: dict | None = None): + """Read from an admin terminal on the backend or a personal terminal in its browser.""" + import aiohttp + + from open_webui.env import AIOHTTP_CLIENT_SESSION_TOOL_SERVER_SSL, AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER_DATA from open_webui.models.config import Config from open_webui.models.groups import Groups from open_webui.models.users import UserModel from open_webui.utils.access_control import has_connection_access - from open_webui.utils.headers import bearer_auth_header from open_webui.utils.tools import build_tool_server_headers metadata = metadata or {} @@ -163,66 +167,39 @@ async def get_terminal_request_info(request, user, metadata: dict, extra_params: headers['X-Session-Id'] = metadata['chat_id'] if context_id: headers[TERMINAL_CONTEXT_HEADER] = context_id - return get_terminal_server_url(connection), headers, cookies - - selector = str(terminal_id).rstrip('/') - direct_terminal = next( - ( - server - for server in metadata.get('tool_servers') or [] - if str(server.get('url') or '').rstrip('/') == selector - ), - None, - ) - if not direct_terminal: - return None - - headers = {'Accept': 'application/json'} - key = str(direct_terminal.get('key') or '').strip() - if key: - headers.update(bearer_auth_header(key)) - if metadata.get('chat_id'): - headers['X-Session-Id'] = metadata['chat_id'] - return selector, headers, {} + timeout = aiohttp.ClientTimeout(total=AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER_DATA) + async with aiohttp.ClientSession(timeout=timeout, trust_env=True) as session: + async with session.get( + f'{get_terminal_server_url(connection)}{path}', + headers=headers, + cookies=cookies, + ssl=AIOHTTP_CLIENT_SESSION_TOOL_SERVER_SSL, + allow_redirects=False, + ) as response: + return await response.json() if response.status == 200 else None + + event_caller = (extra_params or {}).get('__event_call__') + if event_caller: + result = await event_caller( + { + 'type': 'request:terminal', + 'data': {'terminal_id': terminal_id, 'path': path, 'session_id': metadata.get('session_id')}, + } + ) + return result.get('data') if isinstance(result, dict) else None + return None async def get_terminal_agents_md(request, user, metadata: dict, extra_params: dict | None = None) -> str | None: """Load the selected terminal user's home AGENTS.md afresh for this turn.""" - import aiohttp - from open_webui.env import AIOHTTP_CLIENT_SESSION_TOOL_SERVER_SSL - try: async with asyncio.timeout(5): - terminal_request = await get_terminal_request_info(request, user, metadata, extra_params) - if not terminal_request: + data = await get_terminal_json(request, user, metadata, '/files/cwd', extra_params) + home = data.get('home') if isinstance(data, dict) else None + if not isinstance(home, str) or not (posixpath.isabs(home) or ntpath.isabs(home)): return None - base_url, headers, cookies = terminal_request - async with aiohttp.ClientSession( - headers=headers, cookies=cookies, timeout=aiohttp.ClientTimeout(total=5), trust_env=True - ) as session: - async with session.get( - f'{base_url.rstrip("/")}/files/cwd', - ssl=AIOHTTP_CLIENT_SESSION_TOOL_SERVER_SSL, - allow_redirects=False, - ) as response: - if response.status != 200: - log.debug('Skipping terminal AGENTS.md: home lookup returned HTTP %s', response.status) - return None - data = await response.json() - home = data.get('home') if isinstance(data, dict) else None - if not isinstance(home, str) or not posixpath.isabs(home): - return None - path = posixpath.join(home, 'AGENTS.md') - async with session.get( - f'{base_url.rstrip("/")}/files/read', - params={'path': path}, - ssl=AIOHTTP_CLIENT_SESSION_TOOL_SERVER_SSL, - allow_redirects=False, - ) as response: - if response.status == 404: - return None - response.raise_for_status() - data = await response.json() + path = quote(posixpath.join(home, 'AGENTS.md'), safe='') + data = await get_terminal_json(request, user, metadata, f'/files/read?path={path}', extra_params) content = data.get('content') if isinstance(data, dict) else None if not isinstance(content, str) or not content.strip(): @@ -247,25 +224,9 @@ def add_terminal_agents_md(messages: list[dict], agents_md: str) -> list[dict]: async def get_terminal_skill( request, user, metadata: dict, skill_name: str, extra_params: dict | None = None ) -> dict | None: - from urllib.parse import quote - - import aiohttp - from open_webui.env import AIOHTTP_CLIENT_SESSION_TOOL_SERVER_SSL, AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER_DATA - - terminal_request = await get_terminal_request_info(request, user, metadata, extra_params) - if not terminal_request: - return None - base_url, headers, cookies = terminal_request - - timeout = aiohttp.ClientTimeout(total=AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER_DATA) - async with aiohttp.ClientSession(timeout=timeout, trust_env=True) as session: - async with session.get( - f'{base_url.rstrip("/")}/skills/{quote(skill_name, safe="")}', - headers=headers, - cookies=cookies, - ssl=AIOHTTP_CLIENT_SESSION_TOOL_SERVER_SSL, - ) as response: - skill = await response.json() if response.status == 200 else None + skill = await get_terminal_json( + request, user, metadata, f'/skills/read?name={quote(skill_name, safe="")}', extra_params + ) if not isinstance(skill, dict): return None diff --git a/backend/open_webui/utils/timers.py b/backend/open_webui/utils/timers.py index 094a2130c87b..15a6b898db59 100644 --- a/backend/open_webui/utils/timers.py +++ b/backend/open_webui/utils/timers.py @@ -11,12 +11,13 @@ from typing import Literal from uuid import uuid4 -from fastapi import Request +from fastapi import HTTPException, Request from open_webui.internal.db import get_async_db from open_webui.models.chat_messages import ChatMessages from open_webui.models.chats import Chat, ChatForm, Chats from open_webui.models.users import UserModel, Users from open_webui.tasks import has_active_tasks +from open_webui.utils.auth import VERIFIED_USER_ROLES from open_webui.utils.json_codec import JSONCodec from open_webui.utils.misc import get_message_list from sqlalchemy import select @@ -258,6 +259,11 @@ async def execute_due_timer(app, timer_id: str, claim_id: str | None = None) -> await _set_timer_state(timer_id, 'error', timer_error='timer user no longer exists') return + # Re-gate the rehydrated owner: a demoted owner must not run. + if user.role not in VERIFIED_USER_ROLES: + await _set_timer_state(timer_id, 'error', timer_error='owner no longer permitted to run timers') + return + run = meta.get('run') or {} model_id = run.get('model_id') or meta.get('timer_model_id') if not model_id: @@ -410,7 +416,20 @@ async def execute_due_timer(app, timer_id: str, claim_id: str | None = None) -> await app.state.CHAT_COMPLETION_HANDLER(request, form_data, user=user) except Exception as exc: log.exception(f'Timer {timer_id} completion failed') - await _set_timer_state(timer_id, 'error', timer_error=str(exc)[:500]) + error_detail = exc.detail if isinstance(exc, HTTPException) else str(exc) + await _set_timer_state(timer_id, 'error', timer_error=error_detail[:500]) + await Chats.upsert_message_to_chat_by_id_and_message_id( + parent_chat_id, assistant_message_id, {'error': {'content': error_detail}, 'done': True} + ) + await sio.emit( + 'events', + { + 'chat_id': parent_chat_id, + 'message_id': assistant_message_id, + 'data': {'type': 'chat:message:error', 'data': {'error': {'content': error_detail}, 'done': True}}, + }, + room=f'user:{timer.user_id}', + ) async def _set_timer_state(timer_id: str, status: str, **fields) -> None: diff --git a/backend/open_webui/utils/tool_approval.py b/backend/open_webui/utils/tool_approval.py index c59b2d6316be..7d5376bb01ee 100644 --- a/backend/open_webui/utils/tool_approval.py +++ b/backend/open_webui/utils/tool_approval.py @@ -5,7 +5,10 @@ from sqlalchemy.ext.asyncio import AsyncSession from open_webui.constants import ERROR_MESSAGES +from open_webui.env import ENABLE_ADMIN_CHAT_ACCESS from open_webui.models.chats import Chats +from open_webui.models.config import Config +from open_webui.models.users import Users from open_webui.socket.main import get_event_emitter from open_webui.utils.json_codec import JSONCodec @@ -25,7 +28,7 @@ async def resolve_tool_call_output( db: AsyncSession | None = None, ) -> dict: chat = await Chats.get_chat_by_id(chat_id, db=db) - if not chat or (chat.user_id != user.id and user.role != 'admin'): + if not chat or (chat.user_id != user.id and not (user.role == 'admin' and ENABLE_ADMIN_CHAT_ACCESS)): raise HTTPException( status_code=status.HTTP_401_UNAUTHORIZED, detail=ERROR_MESSAGES.ACCESS_PROHIBITED, @@ -160,8 +163,17 @@ async def build_tool_approval_resume_payload(chat_id: str, message_id: str, chat raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail='Tool call message model is missing.') messages = [] - if params.get('system'): - messages.append({'role': 'system', 'content': params.get('system')}) + system_prompt = params.get('system') + if not system_prompt: + # Mirror the chat UI's system prompt fallback + user = await Users.get_user_by_id(chat.user_id) + ui_settings = (user.settings.ui if user and user.settings else None) or {} + system_prompt = ui_settings.get('system') + if system_prompt is None: + default_interface_settings = await Config.get('ui.default_interface_settings') or {} + system_prompt = default_interface_settings.get('system') + if system_prompt: + messages.append({'role': 'system', 'content': system_prompt}) return { 'stream': params.get('stream_response', True), diff --git a/backend/requirements-slim.txt b/backend/requirements-slim.txt index e482bf330dce..c688c89d23f3 100644 --- a/backend/requirements-slim.txt +++ b/backend/requirements-slim.txt @@ -21,7 +21,7 @@ joserfc==1.7.4 requests==2.34.2 regex==2026.5.9 google-re2==1.1.20251105 # bounded compilation and linear-time knowledge searches -aiohttp==3.13.5 # do not update to 3.13.3 - broken +aiohttp==3.14.3 # do not update to 3.13.3 - broken aiodns==3.6.1 # keep pinned: 4.x pulls pycares 5 (c-ares 1.34.6) which breaks DNS on some hosts (#28013, #28215); opt-in via AIOHTTP_CLIENT_ASYNC_DNS_RESOLVER aiocache==0.12.3 aiofiles==25.1.0 @@ -38,7 +38,7 @@ alembic==1.18.4 pycrdt==0.13.1 redis==8.0.1 -hiredis==3.4.0 +hiredis==3.4.2 python-dateutil==2.9.0.post0 pytz==2026.2 @@ -67,7 +67,7 @@ lxml==6.1.1 validators==0.35.0 psutil==7.2.2 -pillow==12.2.0 +pillow==12.3.0 rank-bm25==0.2.2 black==26.5.1 diff --git a/backend/requirements.txt b/backend/requirements.txt index 11282b280754..adbc0e560ee8 100644 --- a/backend/requirements.txt +++ b/backend/requirements.txt @@ -16,7 +16,7 @@ joserfc==1.7.4 requests==2.34.2 regex==2026.5.9 google-re2==1.1.20251105 # bounded compilation and linear-time knowledge searches -aiohttp==3.13.5 # do not update to 3.13.3 - broken +aiohttp==3.14.3 # do not update to 3.13.3 - broken aiodns==3.6.1 # keep pinned: 4.x pulls pycares 5 (c-ares 1.34.6) which breaks DNS on some hosts (#28013, #28215); opt-in via AIOHTTP_CLIENT_ASYNC_DNS_RESOLVER aiocache==0.12.3 aiofiles==25.1.0 @@ -34,7 +34,7 @@ alembic==1.18.4 pycrdt==0.13.1 redis==8.0.1 -hiredis==3.4.0 +hiredis==3.4.2 pytz==2026.2 @@ -85,7 +85,7 @@ psutil==7.2.2 sentencepiece==0.2.1 soundfile==0.13.1 -pillow==12.2.0 +pillow==12.3.0 opencv-python-headless==4.13.0.92 rapidocr==3.9.2 rank-bm25==0.2.2 diff --git a/package-lock.json b/package-lock.json index 45a7cf56deab..d17274c7c23d 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "open-webui", - "version": "0.11.3", + "version": "0.11.4", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "open-webui", - "version": "0.11.3", + "version": "0.11.4", "dependencies": { "@azure/msal-browser": "^4.5.0", "@codemirror/lang-javascript": "^6.2.2", diff --git a/package.json b/package.json index ed938ec131be..16f139835fe3 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "open-webui", - "version": "0.11.3", + "version": "0.11.4", "private": true, "scripts": { "dev": "npm run pyodide:fetch && vite dev --host", diff --git a/pyproject.toml b/pyproject.toml index 0f67b1b1200d..e410b2b788fd 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -22,7 +22,7 @@ dependencies = [ "joserfc==1.7.4", "requests==2.34.2", - "aiohttp==3.13.5", # do not update to 3.13.3 - broken + "aiohttp==3.14.3", # do not update to 3.13.3 - broken "aiodns==3.6.1", # keep pinned: 4.x pulls pycares 5 (c-ares 1.34.6) which breaks DNS on some hosts (#28013, #28215); opt-in via AIOHTTP_CLIENT_ASYNC_DNS_RESOLVER "aiocache==0.12.3", "aiofiles==25.1.0", @@ -40,7 +40,7 @@ dependencies = [ "pycrdt==0.13.1", "redis==8.0.1", - "hiredis==3.4.0", + "hiredis==3.4.2", # "valkey-glide-sync==2.3.1", # optional: install manually if VECTOR_DB=valkey "pytz==2026.2", @@ -93,7 +93,7 @@ dependencies = [ "soundfile==0.13.1", "azure-ai-documentintelligence==1.0.2", - "pillow==12.2.0", + "pillow==12.3.0", "opencv-python-headless==4.13.0.92", "rapidocr==3.9.2", "rank-bm25==0.2.2", diff --git a/scripts/prepare-pyodide.js b/scripts/prepare-pyodide.js index 8ec57426cb66..cb043f73126d 100644 --- a/scripts/prepare-pyodide.js +++ b/scripts/prepare-pyodide.js @@ -13,7 +13,8 @@ const packages = [ 'tiktoken', 'pytz', 'black', - 'openai' + 'openai', + 'lxml' ]; // Pure-Python packages whose wheels must be downloaded from PyPI and saved into @@ -28,13 +29,18 @@ const pypiPackages = [ 'pytokens', 'openpyxl', 'et-xmlfile', - 'seaborn' + 'seaborn', + 'python-pptx', + 'python-docx', + 'xlsxwriter' ]; const pypiDepends = { black: ['click', 'mypy-extensions', 'packaging', 'pathspec', 'platformdirs', 'pytokens'], openpyxl: ['et-xmlfile'], - seaborn: ['matplotlib', 'numpy', 'pandas'] + seaborn: ['matplotlib', 'numpy', 'pandas'], + 'python-pptx': ['lxml', 'pillow', 'xlsxwriter', 'typing-extensions'], + 'python-docx': ['lxml', 'typing-extensions'] }; import { loadPyodide } from 'pyodide'; diff --git a/src/lib/apis/configs/index.ts b/src/lib/apis/configs/index.ts index ec5d4a12ab55..236664f1159c 100644 --- a/src/lib/apis/configs/index.ts +++ b/src/lib/apis/configs/index.ts @@ -744,7 +744,7 @@ export const setSubagentsConfig = async (token: string, config: object) => { export const setDefaultPromptSuggestions = async ( token: string, - promptSuggestions: any[], + promptSuggestions: any[] | null, promptSuggestionsI18n: Record = {} ) => { let error = null; diff --git a/src/lib/apis/images/index.ts b/src/lib/apis/images/index.ts index ad2f79d7313f..3717093798ae 100644 --- a/src/lib/apis/images/index.ts +++ b/src/lib/apis/images/index.ts @@ -258,14 +258,12 @@ export const imageEdits = async ( ...(token && { authorization: `Bearer ${token}` }) }, body: JSON.stringify({ - form_data: { - image: images, - prompt, - ...(model && { model }), - ...(size && { size }), - ...(n && { n }), - ...(background && { background }) - } + image: images, + prompt, + ...(model && { model }), + ...(size && { size }), + ...(n && { n }), + ...(background && { background }) }) }) .then(async (res) => { diff --git a/src/lib/components/AddConnectionModal.svelte b/src/lib/components/AddConnectionModal.svelte index a7edcf391389..0ada5f55c36b 100644 --- a/src/lib/components/AddConnectionModal.svelte +++ b/src/lib/components/AddConnectionModal.svelte @@ -77,9 +77,29 @@ // remove trailing slash from url url = url.replace(/\/$/, ''); + let _headers = null; + + if (headers) { + try { + _headers = JSON.parse(headers); + if (typeof _headers !== 'object' || Array.isArray(_headers)) { + _headers = null; + throw new Error('Headers must be a valid JSON object'); + } + headers = JSON.stringify(_headers, null, 2); + } catch (error) { + toast.error($i18n.t('Headers must be a valid JSON object')); + return; + } + } + const res = await verifyOllamaConnection(localStorage.token, { url, - key + key, + config: { + auth_type, + ...(_headers ? { headers: _headers } : {}) + } }).catch((error) => { toast.error(`${error}`); }); @@ -241,6 +261,12 @@ showAdvanced = false; tags = []; modelIds = []; + headers = ''; + enable = true; + connectionType = 'external'; + provider = ''; + apiVersion = ''; + apiType = ''; }; const init = () => { diff --git a/src/lib/components/admin/Analytics.svelte b/src/lib/components/admin/Analytics.svelte index bd33b3c3729f..da69de19e111 100644 --- a/src/lib/components/admin/Analytics.svelte +++ b/src/lib/components/admin/Analytics.svelte @@ -18,7 +18,7 @@ {#if loaded} -
+
{/if} diff --git a/src/lib/components/admin/Settings/Authentication.svelte b/src/lib/components/admin/Settings/Authentication.svelte index 6c1224d636f5..189cb55e812a 100644 --- a/src/lib/components/admin/Settings/Authentication.svelte +++ b/src/lib/components/admin/Settings/Authentication.svelte @@ -92,6 +92,9 @@ toast.error(`${error}`); return null; }); + if (res) { + adminConfig = res; + } return !!res; }; diff --git a/src/lib/components/admin/Settings/Documents.svelte b/src/lib/components/admin/Settings/Documents.svelte index 8186d44d28b4..aa0c109769e1 100644 --- a/src/lib/components/admin/Settings/Documents.svelte +++ b/src/lib/components/admin/Settings/Documents.svelte @@ -169,10 +169,7 @@ toast.error($i18n.t('External Document Loader URL required.')); return; } - if ( - RAGConfig.CONTENT_EXTRACTION_ENGINE === 'external' && - RAGConfig.EXTERNAL_DOCUMENT_LOADER_HEADERS - ) { + if (RAGConfig.EXTERNAL_DOCUMENT_LOADER_HEADERS) { try { const headers = JSON.parse(RAGConfig.EXTERNAL_DOCUMENT_LOADER_HEADERS); if (headers === null || typeof headers !== 'object' || Array.isArray(headers)) { @@ -859,6 +856,7 @@ diff --git a/src/lib/components/admin/Settings/Models.svelte b/src/lib/components/admin/Settings/Models.svelte index f934977a410c..cb9fb1db4a64 100644 --- a/src/lib/components/admin/Settings/Models.svelte +++ b/src/lib/components/admin/Settings/Models.svelte @@ -153,6 +153,7 @@ const modelOrder = new Map(modelOrderList.map((id, idx) => [id, idx])); filteredModels = models + .filter((m) => !selectedTag || modelTags(m).includes(selectedTag)) .filter((m) => searchValue === '' || m.name.toLowerCase().includes(searchValue.toLowerCase())) .filter((m) => { if (viewOption === 'base') return !isPresetModel(m); @@ -180,9 +181,6 @@ } let searchValue = ''; - let canReorderModels = false; - - $: canReorderModels = searchValue === '' && viewOption === '' && selectedTag === ''; const enableAllHandler = async () => { const modelsToEnable = filteredModels.filter((m) => !(m.is_active ?? true)); @@ -292,26 +290,24 @@ const listedModelIds = new Set(allModels.map((model) => model.id)); allModels.push(...savedModels.filter((model) => !listedModelIds.has(model.id))); - models = allModels - .map((m: ModelListItem) => { - const savedModel = savedModels.find((model: ModelListItem) => model.id === m.id); - - if (savedModel) { - return { - ...m, - ...savedModel - }; - } else { - return { - ...m, - id: m.id, - name: m.name, - - is_active: true - }; - } - }) - .filter((model) => !selectedTag || modelTags(model).includes(selectedTag)); + models = allModels.map((m: ModelListItem) => { + const savedModel = savedModels.find((model: ModelListItem) => model.id === m.id); + + if (savedModel) { + return { + ...m, + ...savedModel + }; + } else { + return { + ...m, + id: m.id, + name: m.name, + + is_active: true + }; + } + }); modelOrderList = [ ...modelOrderList.filter((id) => models.some((model) => model.id === id)), @@ -445,15 +441,14 @@ const target = parent.children[oldIndex < newIndex ? oldIndex : oldIndex + 1]; parent.insertBefore(item, target); - const updatedModels = [...filteredModels]; - const [movedModel] = updatedModels.splice(oldIndex, 1); - updatedModels.splice(newIndex, 0, movedModel); + // Anchor on the visible neighbor so filtered-out models keep their place + const movedModelId = filteredModels[oldIndex].id; + const anchorModelId = filteredModels[newIndex].id; + const reorderedIds = modelOrderList.filter((id) => id !== movedModelId); + const anchorIndex = reorderedIds.indexOf(anchorModelId); + reorderedIds.splice(oldIndex < newIndex ? anchorIndex + 1 : anchorIndex, 0, movedModelId); - const orderedIds = updatedModels.map((model) => model.id); - const orderedSet = new Set(orderedIds); - - models = [...updatedModels, ...models.filter((model) => !orderedSet.has(model.id))]; - modelOrderList = models.map((model) => model.id); + modelOrderList = reorderedIds; modelOrderDirty = true; }; @@ -463,7 +458,7 @@ sortable = null; } - if (modelListElement && filteredModels.length > 0 && canReorderModels) { + if (modelListElement && filteredModels.length > 0) { sortable = new Sortable(modelListElement, { animation: 150, handle: '.model-item-handle', @@ -638,7 +633,7 @@ model = await getFullModel(model); sessionStorage.model = JSON.stringify({ ...model, - base_model_id: model.id, + ...(isPresetModel(model) ? {} : { base_model_id: model.id }), id: `${model.id}-clone`, name: `${model.name} (Clone)` }); @@ -825,14 +820,11 @@ items={tags.map((tag) => { return { value: tag, label: tag }; })} - onChange={async () => { - await init(); - }} /> {/if}
- + + {/if} + {$i18n.t('Copy Link')} - {#if model?.is_active ?? true} + {#if (model?.is_active ?? true) && model?.owned_by !== 'arena'} + {/if} + -
- {#if showWebSearchButton || showImageGenerationButton || showCodeInterpreterButton || showToolsButton || showSkillsButton || (toggleFilters && toggleFilters.length > 0)} - oauthRedirectHandler(tool, chatInputDraft)} - {onWebSearchToggle} - closeOnOutsideClick={integrationsMenuCloseOnOutsideClick} - onShowValves={(e) => { - const { type, id } = e; - selectedValvesType = type; - selectedValvesItemId = id; - showValvesModal = true; - integrationsMenuCloseOnOutsideClick = false; - }} - onClose={async () => { - await tick(); +
+ {#if selectedModelIds.length === 1 && $models.find((m) => m.id === selectedModelIds[0])?.has_user_valves} +
+ + + +
+ {/if} - const chatInput = document.getElementById('chat-input'); - chatInput?.focus(); - }} +
+ {#if (selectedToolIds ?? []).length > 0} + - - {/if} + - {#if selectedModelIds.length === 1 && $models.find((m) => m.id === selectedModelIds[0])?.has_user_valves} -
- - - -
+ + {(selectedToolIds ?? []).length} + + +
{/if} -
- {#if (selectedToolIds ?? []).length > 0} - 0} + + - - {/if} + + {(selectedSkillIds ?? []).length} + + + + {/if} - {#if (selectedSkillIds ?? []).length > 0} + {#each selectedFilterIds as filterId (filterId)} + {@const filter = toggleFilters.find((f) => f.id === filterId)} + {#if filter} - - {/if} - - {#each selectedFilterIds as filterId (filterId)} - {@const filter = toggleFilters.find((f) => f.id === filterId)} - {#if filter} - - - - {/if} - {/each} - - {#if webSearchEnabled && showWebSearchButton} - - {/if} + {/each} - {#if imageGenerationEnabled && showImageGenerationButton} - - - - {/if} + {#if webSearchEnabled && showWebSearchButton} + + + + {/if} - {#if codeInterpreterEnabled && showCodeInterpreterButton} - - + + {/if} - - - - {/if} + {#if codeInterpreterEnabled && showCodeInterpreterButton} + + - - {/each} - - - {#if showTerminalSelector} - - {/if} -
+ + + + {/if} + + {#each pendingOAuthTools as pendingTool (pendingTool.id)} + + + + {/each} + + + {#if showTerminalSelector} + + {/if}
diff --git a/src/lib/components/chat/MessageInput/CallOverlay.svelte b/src/lib/components/chat/MessageInput/CallOverlay.svelte index 6227c2035982..209f63f508cf 100644 --- a/src/lib/components/chat/MessageInput/CallOverlay.svelte +++ b/src/lib/components/chat/MessageInput/CallOverlay.svelte @@ -44,6 +44,7 @@ let mediaRecorder; let audioStream = null; let audioChunks = []; + let destroyed = false; let videoInputDevices = []; let selectedVideoInputDeviceId = null; @@ -184,7 +185,8 @@ }; const stopRecordingCallback = async (_continue = true) => { - if ($showCallOverlay) { + // $showCallOverlay stays true when the chat page unmounts + if ($showCallOverlay && !destroyed) { console.log('%c%s', 'color: red; font-size: 20px;', '🚨 stopRecordingCallback 🚨'); // deep copy the audioChunks array @@ -231,7 +233,7 @@ }; const startRecording = async () => { - if ($showCallOverlay) { + if ($showCallOverlay && !destroyed) { if (!audioStream) { audioStream = await navigator.mediaDevices.getUserMedia({ audio: { @@ -379,6 +381,7 @@ }; let finishedMessages = {}; + let failedMessages = {}; let currentMessageId = null; let currentUtterance: SpeechSynthesisUtterance | null = null; @@ -453,21 +456,17 @@ }; audioElement.src = audio.src; - audioElement.muted = true; + // stopAllAudio mutes it; unmuting after play() outside a gesture makes WebKit pause it + audioElement.muted = false; audioElement.playbackRate = $settings.audio?.tts?.playbackRate ?? 1; audioElement.onended = finish; audioElement.onerror = () => finish(); audioElement.onpause = finish; - audioElement - .play() - .then(() => { - audioElement.muted = false; - }) - .catch((error) => { - console.error(error); - finish(error); - }); + audioElement.play().catch((error) => { + console.error(error); + finish(error); + }); }); } else { return Promise.resolve(); @@ -502,7 +501,9 @@ const emojiCache = new Map(); const fetchAudio = async (content) => { - if (!audioCache.has(content)) { + const id = currentMessageId; + + if (!audioCache.has(content) && !failedMessages[id]) { try { // Set the emoji for the content if needed if ($settings?.showEmojiInCall ?? false) { @@ -535,6 +536,10 @@ const res = await synthesizeOpenAISpeech(localStorage.token, getVoiceId(), content).catch( (error) => { console.error(error); + if (!failedMessages[id]) { + failedMessages[id] = true; + toast.error(`${error}`); + } return null; } ); @@ -550,6 +555,10 @@ } catch (error) { console.error('Error synthesizing speech:', error); } + + if (!audioCache.has(content)) { + failedMessages[id] = true; + } } return audioCache.get(content); @@ -591,7 +600,7 @@ } else { await speakSpeechSynthesisHandler(content); } - } else { + } else if (!failedMessages[id]) { // If not available in the cache, push it back to the queue and delay messages[id].unshift(content); // Re-queue the content at the start console.log(`Audio for "${content}" not yet available in the cache, re-queued...`); @@ -765,6 +774,7 @@ }); onDestroy(async () => { + destroyed = true; await stopAllAudio(); await stopRecordingCallback(false); await stopCamera(); diff --git a/src/lib/components/chat/MessageInput/CommandSuggestionList.svelte b/src/lib/components/chat/MessageInput/CommandSuggestionList.svelte index 8988ab9e29c4..e1e0ff016fe4 100644 --- a/src/lib/components/chat/MessageInput/CommandSuggestionList.svelte +++ b/src/lib/components/chat/MessageInput/CommandSuggestionList.svelte @@ -55,6 +55,7 @@ const onKeyDown = (event: KeyboardEvent) => { if (!['ArrowUp', 'ArrowDown', 'Enter', 'Tab', 'Escape'].includes(event.key)) return false; + if ((filteredItems ?? []).length === 0) return false; if (event.key === 'ArrowUp') { suggestionElement?.selectUp(); @@ -89,7 +90,10 @@ } -
0 ? '' : 'hidden'} id="suggestions-container"> +
0 ? '' : 'hidden'} + id={(filteredItems ?? []).length > 0 ? 'suggestions-container' : undefined} +>
{#if char === '/'} diff --git a/src/lib/components/chat/Messages.svelte b/src/lib/components/chat/Messages.svelte index 30ac1a10a350..4a08eecc02f8 100644 --- a/src/lib/components/chat/Messages.svelte +++ b/src/lib/components/chat/Messages.svelte @@ -370,6 +370,8 @@ parentId: parentId, childrenIds: [], files: undefined, + annotation: undefined, + feedbackId: undefined, content: output !== undefined ? '' : content, ...(output !== undefined ? { output } : {}), timestamp: Math.floor(Date.now() / 1000) // Unix epoch diff --git a/src/lib/components/chat/Messages/CodeBlock.svelte b/src/lib/components/chat/Messages/CodeBlock.svelte index 4ae1a202b995..b52700494c12 100644 --- a/src/lib/components/chat/Messages/CodeBlock.svelte +++ b/src/lib/components/chat/Messages/CodeBlock.svelte @@ -235,7 +235,9 @@ /\bimport\s+tiktoken\b|\bfrom\s+tiktoken\b/.test(code) ? 'tiktoken' : null, /\bimport\s+pytz\b|\bfrom\s+pytz\b/.test(code) ? 'pytz' : null, /\bimport\s+openpyxl\b|\bfrom\s+openpyxl\b/.test(code) ? 'openpyxl' : null, - /\.(read|to)_excel\(|\.Excel(Writer|File)\(/.test(code) ? 'openpyxl' : null + /\.(read|to)_excel\(|\.Excel(Writer|File)\(/.test(code) ? 'openpyxl' : null, + /\bimport\s+pptx\b|\bfrom\s+pptx\b/.test(code) ? 'python-pptx' : null, + /\bimport\s+docx\b|\bfrom\s+docx\b/.test(code) ? 'python-docx' : null ].filter(Boolean); console.log(packages); diff --git a/src/lib/components/chat/Messages/CodeExecutionModal.svelte b/src/lib/components/chat/Messages/CodeExecutionModal.svelte index 4b4ee0b7a7f3..fd33b7dd23d7 100644 --- a/src/lib/components/chat/Messages/CodeExecutionModal.svelte +++ b/src/lib/components/chat/Messages/CodeExecutionModal.svelte @@ -1,5 +1,6 @@
@@ -199,7 +203,7 @@ >
- {#if hasActiveToolCalls} + {#if hasActiveDetails}
@@ -223,7 +227,7 @@
- {prefixText} {#if summaryText} diff --git a/src/lib/components/chat/Messages/ResponseMessage.svelte b/src/lib/components/chat/Messages/ResponseMessage.svelte index 48af8d0304fc..fa7f7ab93347 100644 --- a/src/lib/components/chat/Messages/ResponseMessage.svelte +++ b/src/lib/components/chat/Messages/ResponseMessage.svelte @@ -287,9 +287,14 @@ } else { $audioQueue.setId(`${message.id}`); $audioQueue.setPlaybackRate($settings.audio?.tts?.playbackRate ?? 1); - $audioQueue.onStopped = () => { + $audioQueue.onStopped = ({ event }) => { speaking = false; speakingIdx = undefined; + + if (event === 'error') { + speakAbort?.abort(); + toast.error($i18n.t('Audio playback failed')); + } }; loadingSpeech = true; diff --git a/src/lib/components/chat/ModelSelector/ModelItem.svelte b/src/lib/components/chat/ModelSelector/ModelItem.svelte index b15041b08c9a..dad2ab33e591 100644 --- a/src/lib/components/chat/ModelSelector/ModelItem.svelte +++ b/src/lib/components/chat/ModelSelector/ModelItem.svelte @@ -33,6 +33,7 @@ export let selectionOnly = false; export let onClick: () => void = () => {}; + export let onEdit: () => void = () => {}; $: localizedDescription = resolveLocalizedModelDescription(item.model, $i18n.language); @@ -304,6 +305,7 @@ model={item.model} {pinModelHandler} {deleteModelHandler} + {onEdit} copyLinkHandler={() => { copyLinkHandler(item.model); }} diff --git a/src/lib/components/chat/ModelSelector/ModelItemMenu.svelte b/src/lib/components/chat/ModelSelector/ModelItemMenu.svelte index 57fd390f6a1f..acf546d5f654 100644 --- a/src/lib/components/chat/ModelSelector/ModelItemMenu.svelte +++ b/src/lib/components/chat/ModelSelector/ModelItemMenu.svelte @@ -20,6 +20,7 @@ export let pinModelHandler: (modelId: string) => void = () => {}; export let copyLinkHandler: Function = () => {}; export let deleteModelHandler: Function = () => {}; + export let onEdit: () => void = () => {}; export let onClose: Function = () => {}; @@ -30,6 +31,7 @@ bind:show align="end" sideOffset={-2} + contentClass="model-selector-child-menu" onOpenChange={(state) => { if (state === false) { onClose(); @@ -61,6 +63,7 @@ showSettings.set({ tab: 'admin:models', state: { id: model?.id ?? null } }); } show = false; + onEdit(); }} > diff --git a/src/lib/components/chat/ModelSelector/Selector.svelte b/src/lib/components/chat/ModelSelector/Selector.svelte index 405f4863e1ef..81b4f28b2a35 100644 --- a/src/lib/components/chat/ModelSelector/Selector.svelte +++ b/src/lib/components/chat/ModelSelector/Selector.svelte @@ -159,8 +159,8 @@ } }; - const handlePointerDown = (e: PointerEvent) => { - if (!show) return; + const handleWindowClick = (e: MouseEvent) => { + if (!show || e.detail === 0) return; const target = e.target as Node; if ( (triggerElement && triggerElement.contains(target)) || @@ -169,6 +169,8 @@ ) { return; } + e.preventDefault(); + e.stopPropagation(); show = false; document.getElementById(`model-selector-${id}-button`)?.blur(); }; @@ -808,6 +810,7 @@ let deleteModelTarget: any = null; const deleteModelHandler = async (model: any) => { + show = false; deleteModelTarget = model; showDeleteConfirm = true; }; @@ -901,7 +904,7 @@ }} /> - +
{/if} + + {#if !$temporaryChatEnabled && ($user?.role === 'admin' || ($user?.permissions?.chat?.delete ?? true))} + + {/if}
{:else} @@ -977,7 +993,13 @@ {#if (params?.tfs_z ?? null) !== null} - {@render rangeParam('tfs_z', $i18n.t('settings.personal.general.parameters.tfsZ.label'), 0, 2, 0.05)} + {@render rangeParam( + 'tfs_z', + $i18n.t('settings.personal.general.parameters.tfsZ.label'), + 0, + 2, + 0.05 + )} {/if}
diff --git a/src/lib/components/chat/Settings/ArchivedChats.svelte b/src/lib/components/chat/Settings/ArchivedChats.svelte index 350085dbd3b4..6a3373e55158 100644 --- a/src/lib/components/chat/Settings/ArchivedChats.svelte +++ b/src/lib/components/chat/Settings/ArchivedChats.svelte @@ -221,7 +221,7 @@ {/if}
- +
diff --git a/src/lib/components/chat/Suggestions.svelte b/src/lib/components/chat/Suggestions.svelte index 2d3d52b8ca84..f61695b2dc2d 100644 --- a/src/lib/components/chat/Suggestions.svelte +++ b/src/lib/components/chat/Suggestions.svelte @@ -27,14 +27,13 @@ // Update the filteredPrompts if inputValue changes // Only increase version if something wirklich geändert hat - $: getFilteredPrompts(inputValue); + $: if (fuse) getFilteredPrompts(inputValue); - // Helper function to check if arrays are the same - // (based on unique IDs oder content) + // Compare objects so translated text refreshes even when IDs stay the same. function arraysEqual(a, b) { if (a.length !== b.length) return false; for (let i = 0; i < a.length; i++) { - if ((a[i].id ?? a[i].content) !== (b[i].id ?? b[i].content)) { + if (a[i] !== b[i]) { return false; } } @@ -60,7 +59,6 @@ $: if (suggestionPrompts) { sortedPrompts = [...(suggestionPrompts ?? [])].sort(() => Math.random() - 0.5); - getFilteredPrompts(inputValue); } diff --git a/src/lib/components/common/Dropdown.svelte b/src/lib/components/common/Dropdown.svelte index 9234ec7afc1a..781785020a9d 100644 --- a/src/lib/components/common/Dropdown.svelte +++ b/src/lib/components/common/Dropdown.svelte @@ -14,6 +14,9 @@ /** Close when clicking outside */ export let closeOnOutsideClick = true; + /** Close action menus after selection. Submenu triggers prevent the default click. */ + export let closeOnSelect = false; + /** Called when open/close state changes */ export let onOpenChange: (state: boolean) => void = () => {}; @@ -281,11 +284,15 @@ afterOpen(); } - function handleWindowPointerDown(event: PointerEvent) { - if (!show || !closeOnOutsideClick) return; + function handleWindowClick(event: MouseEvent) { + if (!show || !closeOnOutsideClick || event.detail === 0) return; if (!(event.target instanceof Node)) return; if (triggerEl?.contains(event.target)) return; if (contentEl?.contains(event.target)) return; + // Submenu content is portaled outside contentEl. + if (event.target instanceof Element && event.target.closest('[role="menu"]')) return; + event.preventDefault(); + event.stopPropagation(); closeDropdown(false); } @@ -302,10 +309,7 @@ import { onMount, onDestroy } from 'svelte'; - let onPointerDown: ((e: PointerEvent) => void) | undefined; onMount(() => { - onPointerDown = (e) => handleWindowPointerDown(e); - document.addEventListener('pointerdown', onPointerDown, true); if (visualViewportAware) { window.visualViewport?.addEventListener('resize', scheduleSettledPositionUpdates); window.visualViewport?.addEventListener('scroll', schedulePositionUpdate); @@ -314,9 +318,6 @@ onDestroy(() => { if (positionFrame != null) cancelAnimationFrame(positionFrame); for (const timer of settleTimers) window.clearTimeout(timer); - if (onPointerDown) { - document.removeEventListener('pointerdown', onPointerDown, true); - } if (visualViewportAware) { window.visualViewport?.removeEventListener('resize', scheduleSettledPositionUpdates); window.visualViewport?.removeEventListener('scroll', schedulePositionUpdate); @@ -325,6 +326,7 @@ e.stopPropagation()} + on:click={(e) => { + e.stopPropagation(); + if (closeOnSelect && !e.defaultPrevented) closeDropdown(); + }} on:pointerdown={(e) => e.stopPropagation()} > diff --git a/src/lib/components/common/FileItem.svelte b/src/lib/components/common/FileItem.svelte index cf1695018c2f..1172f05c428a 100644 --- a/src/lib/components/common/FileItem.svelte +++ b/src/lib/components/common/FileItem.svelte @@ -2,7 +2,7 @@ import { createEventDispatcher, getContext } from 'svelte'; import { WEBUI_API_BASE_URL } from '$lib/constants'; - import { formatFileSize } from '$lib/utils'; + import { formatFileSize, safeLinkUrl } from '$lib/utils'; import { settings, showFileNavPath } from '$lib/stores'; import FileItemModal from './FileItemModal.svelte'; @@ -71,7 +71,7 @@ } else { window.open(`${WEBUI_API_BASE_URL}/files/${url}/content`, '_blank').focus(); } - } else { + } else if (safeLinkUrl(url)) { window.open(`${url}`, '_blank').focus(); } } diff --git a/src/lib/components/common/FileItemModal.svelte b/src/lib/components/common/FileItemModal.svelte index f17b70d329d3..a19e5246d568 100644 --- a/src/lib/components/common/FileItemModal.svelte +++ b/src/lib/components/common/FileItemModal.svelte @@ -4,7 +4,7 @@ import { getContext, onMount, tick } from 'svelte'; - import { formatFileSize, getLineCount } from '$lib/utils'; + import { formatFileSize, getLineCount, safeLinkUrl } from '$lib/utils'; import { WEBUI_API_BASE_URL } from '$lib/constants'; import { settings } from '$lib/stores'; import { getKnowledgeById } from '$lib/apis/knowledge'; @@ -264,7 +264,7 @@ href="#" class="hover:underline line-clamp-1" on:click|preventDefault={() => { - if (item.type === 'file' || item.url) { + if (item.type === 'file' || safeLinkUrl(item.url)) { let fileId = item?.id ?? item?.tempId; window.open( item.type === 'file' diff --git a/src/lib/components/common/InterfaceSettings.svelte b/src/lib/components/common/InterfaceSettings.svelte index c40cd3b495ec..d64c56291d76 100644 --- a/src/lib/components/common/InterfaceSettings.svelte +++ b/src/lib/components/common/InterfaceSettings.svelte @@ -42,7 +42,6 @@ let userLocation = false; // Interface - let defaultModelId = ''; let showUsername = false; let highContrastMode = false; @@ -247,7 +246,6 @@ } saveSettings({ - models: [defaultModelId], imageCompressionSize: imageCompressionSize, fontFamily }); @@ -387,11 +385,6 @@ imageCompressionSize = currentSettings?.imageCompressionSize ?? { width: '', height: '' }; imageCompressionInChannels = currentSettings?.imageCompressionInChannels ?? true; - defaultModelId = currentSettings?.models?.at(0) ?? ''; - if ($config?.default_models) { - defaultModelId = $config.default_models.split(',')[0]; - } - backgroundImageUrl = currentSettings?.backgroundImageUrl ?? null; webSearch = currentSettings?.webSearch ?? null; @@ -459,7 +452,9 @@ />
-

{$i18n.t('settings.personal.interface.sections.ui.title')}

+

+ {$i18n.t('settings.personal.interface.sections.ui.title')} +

@@ -754,7 +749,9 @@
{/if} -
{$i18n.t('settings.personal.interface.sections.chat.title')}
+
+ {$i18n.t('settings.personal.interface.sections.chat.title')} +
@@ -1526,7 +1523,9 @@

-
{$i18n.t('settings.personal.interface.sections.input.title')}
+
+ {$i18n.t('settings.personal.interface.sections.input.title')} +
@@ -1792,7 +1791,9 @@

-
{$i18n.t('settings.personal.interface.sections.voice.title')}
+
+ {$i18n.t('settings.personal.interface.sections.voice.title')} +
@@ -1840,7 +1841,9 @@

-
{$i18n.t('settings.personal.interface.sections.file.title')}
+
+ {$i18n.t('settings.personal.interface.sections.file.title')} +
diff --git a/src/lib/components/common/RichTextInput.svelte b/src/lib/components/common/RichTextInput.svelte index 5e4f5b1d90c2..b4b54d7b439c 100644 --- a/src/lib/components/common/RichTextInput.svelte +++ b/src/lib/components/common/RichTextInput.svelte @@ -1,8 +1,8 @@ { @@ -62,7 +63,6 @@ diff --git a/src/lib/components/workspace/Knowledge/KnowledgeBase/AddContentMenu.svelte b/src/lib/components/workspace/Knowledge/KnowledgeBase/AddContentMenu.svelte index dc539ff04e9d..ce02bdb60e00 100644 --- a/src/lib/components/workspace/Knowledge/KnowledgeBase/AddContentMenu.svelte +++ b/src/lib/components/workspace/Knowledge/KnowledgeBase/AddContentMenu.svelte @@ -26,6 +26,7 @@ { if (state === false) { @@ -62,7 +63,6 @@ class="select-none flex h-[1.6875rem] w-full cursor-pointer items-center gap-2 rounded-xl bg-transparent px-2 text-xs hover:text-gray-900 dark:hover:text-gray-100" on:click={() => { onUpload({ type: 'new_directory' }); - show = false; }} > @@ -134,7 +134,6 @@ class="select-none flex h-[1.6875rem] w-full cursor-pointer items-center gap-2 rounded-xl bg-transparent px-2 text-xs hover:text-gray-900 dark:hover:text-gray-100" on:click={() => { onReset(); - show = false; }} > diff --git a/src/lib/components/workspace/Knowledge/KnowledgeBase/DirectoryRow.svelte b/src/lib/components/workspace/Knowledge/KnowledgeBase/DirectoryRow.svelte index f496826409b8..96c86ab2e2c9 100644 --- a/src/lib/components/workspace/Knowledge/KnowledgeBase/DirectoryRow.svelte +++ b/src/lib/components/workspace/Knowledge/KnowledgeBase/DirectoryRow.svelte @@ -34,12 +34,10 @@ let editName = ''; let editInput: HTMLInputElement; let dragOver = false; - let showDropdown = false; const startRename = () => { editName = directory.name; editing = true; - showDropdown = false; setTimeout(() => editInput?.select(), 0); }; @@ -163,7 +161,7 @@ {#if writeAccess}
- +