feat: add --toolset/-t option to vlmrun chat CLI#164
Merged
Conversation
Add support for enabling a list of toolsets in the chat CLI command. Users can now specify tool categories to enable via repeatable --toolset/-t flags. Available toolsets: core, image, image-gen, world_gen, viz, document, video, web. Example usage: vlmrun chat "Describe this" -i photo.jpg -t web -t image vlmrun chat "Generate a chart" -t viz -t image-gen Co-Authored-By: Sudeep Pillai <sudeep.pillai@gmail.com>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Contributor
|
Update the references for |
Co-Authored-By: Sudeep Pillai <sudeep.pillai@gmail.com>
Contributor
|
Bump up the patch version. |
Co-Authored-By: Sudeep Pillai <sudeep.pillai@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a repeatable
--toolset/-tCLI option tovlmrun chatthat lets users specify which tool categories the Orion agent should have access to during a chat session. The toolsets are validated against theAgentToolsetliteral type and passed viaextra_body["toolsets"]to the OpenAI-compatible chat completions endpoint.Also fixes the
AgentToolsetliteral intypes.pyto use"world-gen"(hyphen) instead of"world_gen"(underscore), matching the API docs. The same rename is applied to the docstring inagent.py.Bumps package version to
0.5.8.Example usage:
Review & Testing Checklist for Human
world-genrename is backward-compatible:AgentToolsetwas changed from"world_gen"to"world-gen". Confirm the server accepts"world-gen"and that no other SDK consumers (e.g. users passingtoolsets=["world_gen"]toclient.agent.execute()) will break.vlmrun chat "what can you do?" -t web -t corewith a valid API key and confirm the toolsets appear in the request and are respected in the response behavior.AgentToolset.__args__stability: TheAVAILABLE_TOOLSETSlist is derived fromLiteral.__args__, which works on Python 3.9–3.12 but is not a documented public API. Confirm this is acceptable or consider usingget_args()fromtyping.Notes
--helptext could drift fromAgentToolset.__args__if the type is updated — a minor maintenance concern.Link to Devin session: https://app.devin.ai/sessions/74e0d8fb30974dc3938dd63f5eb22ae8
Requested by: @spillai