Repository navigation
feat(agent-service): add docs search and read tools - #9424
Draft
eugenegujing wants to merge 1 commit into
Draft
eugenegujing wants to merge 1 commit into
eugenegujing wants to merge 1 commit into
Conversation
Contributor
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9424 +/- ##
============================================
+ Coverage 91.78% 91.88% +0.09%
Complexity 5023 5023
============================================
Files 1283 1285 +2
Lines 56875 57563 +688
Branches 7442 7442
============================================
+ Hits 52205 52893 +688
Misses 2800 2800
Partials 1870 1870
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Add a keyword index over docs/ built when the first agent is created, the searchDocs and readDoc tools over it, a Documentation Read section in the agent's context that carries their output for the current task, a prompt rule that uses them for questions about Texera, a search evaluation script, a TEXERA_DOCS_DIR setting exported by bin/local-dev, and a COPY docs line in the agent-service Dockerfile.
eugenegujing
force-pushed
the
feat/agent-docs-tools
branch
from
November 14, 2026 06:20
7d8d053 to
985bc72
Compare
This was referenced Nov 15, 2026
eugenegujing
marked this pull request as draft
November 16, 2026 20:39
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
The agent could not answer a question about using Texera from the documentation. It had no way to read
docs/, so it answered from the model's memory and could not give a link. This PR adds two read-only tools over the repository's owndocs/folder and a place in the agent's context for what they return.The design in one picture
In the UI. The same question, "What are the output port modes of an operator?", on the same workflow. The answer is on the docs page
reference/operators/output-modes.mainSize and reading order
1,002 lines of code in 10 files, 898 of them in two new files, and 2,669 lines of tests and evaluation data in 8 files.
tools/docs-index.tsnode:fsandnode:pathtools/docs-tools.tsutil/context-utils.tsDocumentation Readsectiontexera-agent.ts,tools/index.tsprompts.tsagent-service.dockerfile,config/env.ts,.env.example,bin/local-dev/main.shThe tools
searchDocs(query, limit?)returns the best matching sections with their heading chain, page path, page link and a snippet. When nothing matches it returns a short outline of the documentation areas (about 1,400 characters today).readDoc(path, section?, offset?)returns the whole page when it fits in 4,000 characters (212 of the 219 pages do), otherwise its section list and introduction, or the named section. A directory path lists the pages under it, and an unknown path returns the closest page names and the outline. Longer output is clipped, and its last line names theoffsetto continue from..mdor a/docs/...prefix, or be the page link. A section may differ in case, punctuation or emoji, or be given as thePage > Headingchain thatsearchDocsprints.nullfor an optional argument counts as absent. A limit out of range is clamped and the output says so, and a bad offset comes back as an error the model can read.How the output reaches the model
The agent rebuilds its context on every step, and its history lists a past tool call by name only. That is enough for the operator tools, whose effect shows in the dataflow, but text returned by a docs tool had nowhere to go.
assembleContextnow adds a# Documentation Readsection between the ongoing task and the dataflow. It lists the docs calls of the current task, oldest first, each under its turn number, tool and input, with the output. A call that failed is listed with its error, and one that never ran, because the tool is not registered or the SDK refused its arguments, with a note saying so. Its entries stay within 20,000 characters, and the intro line and the dropped-outputs line add about 150 on top. Over that, the newest calls are kept whole and one line says how many older ones were left out.serializeTaskis untouched because #9244 edits it.One question, step by step
sequenceDiagram actor U as User participant A as Agent service participant D as Docs index participant M as LLM U->>A: "How do I upload a dataset?" A->>M: Context (step 1) M-->>A: Call searchDocs A->>D: searchDocs("upload dataset") D-->>A: Matching sections rect rgba(46, 160, 67, 0.15) Note over A: Documentation Read now holds the search output A->>M: Context (step 2) end M-->>A: Call readDoc A->>D: readDoc("tutorials/create-dataset-upload-data") D-->>A: Page text rect rgba(46, 160, 67, 0.15) Note over A: Documentation Read now holds both outputs A->>M: Context (step 3) end M-->>A: Answer with the page link A-->>U: AnswerThe index and the prompt
TEXERA_DOCS_DIRonce, when the first agent is created, skippingcontribution-guidelines/. A missing, empty or unreadable folder is logged once, each with its own line, and no docs tool is registered until the service restarts. The prompt text is the same either way.<img>tags, with heading words at three times the weight, camelCase splitting, stopwords and a light suffix stemmer.<img>tags are removed outside code, and relative link targets become absolute, so every link in the text can be cited as it stands.COPY docs ./docsin the Dockerfile, whereTEXERA_DOCS_DIRdefaults to./docs.bin/local-dev/main.shexports the repo'sdocs, and.env.examplesets../docs.No embeddings, no model download, no new dependency. The index covers about 430 KB of markdown and loads in about 30 ms.
Measured, without a model
agent-service/scripts/docs-search-eval.tsruns 25 questions against the realdocs/. The expected page is in the top 3 for 21 and in the top 5 for 23. The two misses are documentation gaps.examples/_index.mdand a sample CSV intutorials/create-dataset-upload-data.md).Known limits
https://texera.apache.org/docs/latest/<page>/. Today each of those is a small stub that forwards to the current release's page and drops the#anchor, so a 200 there does not prove the page exists in that release.(succeeded)when its output is an[ERROR], as for every tool today. After feat(agent-service): show past tool results in the agent's history #9244 a docs error will show both there and in the new section.COPY docsalso puts a 1.8 MB screenshot and the excluded contributor pages into the image.tutorials/create-dataset-upload-data.mdstill tells the user to open an environment tab and click Add Dataset, which the editor no longer has, and the agent repeated that step.Any related issues, documentation, discussions?
Closes #9421. Proposed in #9377 (change 2), following up on #9246. Related: #9244 (past tool results in the history, first line only, which complements the new section), #8773 (docs gaps and broken links found here), #9031, #4932.
Note for reviewers: this branch and the run-errors branch from #9377 both add to
prompts.ts,texera-agent.tsandcontext-utils.ts(and their specs) in neighbouring places, so the two do not apply on top of each other as they are. Whichever lands second will be rebased.How was this PR tested?
cd agent-service && bun test: 526 pass, 0 fail across 21 files (326 before this PR), plusbun run typecheckandbunx prettier --check.docs-index.spec.tsdocs-tools.spec.tscontext-utils.spec.tstexera-agent.spec.tsprompts.spec.tsMutation checks on this logic: 163 mutants, all killed. Also a local
docker buildof the agent-service image to confirm the index loads inside it as thetexerauser, and a new check inbin/local-dev/tests/test_local_dev_sh.shfor the export, run on its own here.Live test on a local stack running this branch, with
claude-haiku-4.5searchDocs,readDoc, answersearchDocswith the same query on every step and never answeredsearchDocswith mixed keywords,readDoc, answerreadDocwith the sectionoffsetas the trailer told itoffsetsearchDocs,readDoc,addOperatorNo workflow was changed by a question. A question took 3 steps and about 33k input tokens, of which about 10.5k per step is the existing system prompt.
Was this PR authored or co-authored using generative AI tooling?
Yes. Generated-by: Claude Code (claude-fable-5-2). Co-authored with Claude; the author reviewed the change before submission.