Skip to content

Fix #3086: Parser actions: add maxDocumentSize, checked before the parse #3087

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
oscerd merged 2 commits into from
Oct 12, 2026
Merged

Fix #3086: Parser actions: add maxDocumentSize, checked before the parse #3087

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3b12069
Fix #3086: Parser actions: add maxDocumentSize, checked before the parse
jiriOndrusek-agent Oct 5, 2026
0098a14
Fix #3086: Parser actions: measure an uncached stream without drainin…
jiriOndrusek-agent Oct 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 41 additions & 1 deletion kamelets/docling-convert-action.kamelet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@ spec:
the body is pinned to bytes and the `CamelDocling*` control headers are swept, because
none of them may be decided by a consumer-delivered payload. By default the `docling`
component executes a local docling CLI; set `doclingServeUrl` to use a Docling Serve
instance instead - the natural fit for self-contained Pipes.
instance instead - the natural fit for self-contained Pipes. With `maxDocumentSize`
set, a payload larger than that many bytes fails the exchange before the conversion.
type: object
properties:
documentIdHeader:
Expand All @@ -50,6 +51,13 @@ spec:
CamelLangChain4jIngestDocumentId exchange property before the conversion.
type: string
default: CamelLangChain4jIngestDocumentId
maxDocumentSize:
title: Max Document Size
description: Maximum size of the payload in bytes, checked before the conversion; a
larger document fails the exchange without being converted. 0, the default, means no
limit.
type: integer
default: 0
doclingServeUrl:
title: Docling Serve URL
description: Address of a Docling Serve instance to convert through, such as
Expand All @@ -74,6 +82,38 @@ spec:
expression:
simple:
expression: "${header[{{documentIdHeader}}]}"
# the size cap, present only when maxDocumentSize is set: the conversion materializes
# the document, so an oversized one is rejected first. The precondition is evaluated
# once, when the Kamelet is instantiated - only then does ${properties:maxDocumentSize:0}
# see the parameter, at runtime it resolves to 0 - so the check must stay a
# precondition, and the comparison below uses {{maxDocumentSize}}
- choice:
precondition: true
when:
- expression:
simple:
expression: "${properties:maxDocumentSize:0} > 0"
steps:
# length() reads a stream to its end and closes it, so a stream body that
# stream caching (on by default) left uncached is read into bytes first; files,
# byte arrays and cached streams are measured in place, without a copy
- choice:
when:
- expression:
simple:
expression: "${body} is 'java.io.InputStream' && ${body} !is 'org.apache.camel.StreamCache'"
steps:
- convertBodyTo:
type: byte[]
- choice:
when:
- expression:
simple:
expression: "${length()} > {{maxDocumentSize}}"
steps:
- throwException:
exceptionType: java.lang.IllegalArgumentException
message: "Document '${exchangeProperty.CamelLangChain4jIngestDocumentId}' exceeds maxDocumentSize (${length()} > {{maxDocumentSize}} bytes)"
- convertBodyTo:
type: byte[]
- removeHeaders:
Expand Down
41 changes: 40 additions & 1 deletion kamelets/tika-extract-text-action.kamelet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ spec:
after the parse all non-Camel headers are removed: past this action, parsed document
metadata is indistinguishable from caller headers, so it must not travel further. Each
format needs its Tika parser module on the classpath; camel-tika ships the HTML and
text modules, PDF needs org.apache.tika:tika-parser-pdf-module.
text modules, PDF needs org.apache.tika:tika-parser-pdf-module. With `maxDocumentSize`
set, a payload larger than that many bytes fails the exchange before the parse.
type: object
properties:
documentIdHeader:
Expand All @@ -51,6 +52,12 @@ spec:
CamelLangChain4jIngestDocumentId exchange property before the parse.
type: string
default: CamelLangChain4jIngestDocumentId
maxDocumentSize:
title: Max Document Size
description: Maximum size of the payload in bytes, checked before the parse; a larger
document fails the exchange without being parsed. 0, the default, means no limit.
type: integer
default: 0
dependencies:
- "camel:tika"
- "camel:kamelet"
Expand All @@ -65,6 +72,38 @@ spec:
expression:
simple:
expression: "${header[{{documentIdHeader}}]}"
# the size cap, present only when maxDocumentSize is set: the parse materializes the
# document, so an oversized one is rejected first. The precondition is evaluated once,
# when the Kamelet is instantiated - only then does ${properties:maxDocumentSize:0} see
# the parameter, at runtime it resolves to 0 - so the check must stay a precondition,
# and the comparison below uses {{maxDocumentSize}}
- choice:
precondition: true
when:
- expression:
simple:
expression: "${properties:maxDocumentSize:0} > 0"
steps:
# length() reads a stream to its end and closes it, so a stream body that
# stream caching (on by default) left uncached is read into bytes first; files,
# byte arrays and cached streams are measured in place, without a copy
- choice:
when:
- expression:
simple:
expression: "${body} is 'java.io.InputStream' && ${body} !is 'org.apache.camel.StreamCache'"
steps:
- convertBodyTo:
type: byte[]
- choice:
when:
- expression:
simple:
expression: "${length()} > {{maxDocumentSize}}"
steps:
- throwException:
exceptionType: java.lang.IllegalArgumentException
message: "Document '${exchangeProperty.CamelLangChain4jIngestDocumentId}' exceeds maxDocumentSize (${length()} > {{maxDocumentSize}} bytes)"
# tika answers plain text (body only, blocks newline-separated) at the pinned encoding
- to:
uri: tika
Expand Down
42 changes: 41 additions & 1 deletion library/camel-kamelets/src/main/resources/kamelets/docling-convert-action.kamelet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@ spec:
the body is pinned to bytes and the `CamelDocling*` control headers are swept, because
none of them may be decided by a consumer-delivered payload. By default the `docling`
component executes a local docling CLI; set `doclingServeUrl` to use a Docling Serve
instance instead - the natural fit for self-contained Pipes.
instance instead - the natural fit for self-contained Pipes. With `maxDocumentSize`
set, a payload larger than that many bytes fails the exchange before the conversion.
type: object
properties:
documentIdHeader:
Expand All @@ -50,6 +51,13 @@ spec:
CamelLangChain4jIngestDocumentId exchange property before the conversion.
type: string
default: CamelLangChain4jIngestDocumentId
maxDocumentSize:
title: Max Document Size
description: Maximum size of the payload in bytes, checked before the conversion; a
larger document fails the exchange without being converted. 0, the default, means no
limit.
type: integer
default: 0
doclingServeUrl:
title: Docling Serve URL
description: Address of a Docling Serve instance to convert through, such as
Expand All @@ -74,6 +82,38 @@ spec:
expression:
simple:
expression: "${header[{{documentIdHeader}}]}"
# the size cap, present only when maxDocumentSize is set: the conversion materializes
# the document, so an oversized one is rejected first. The precondition is evaluated
# once, when the Kamelet is instantiated - only then does ${properties:maxDocumentSize:0}
# see the parameter, at runtime it resolves to 0 - so the check must stay a
# precondition, and the comparison below uses {{maxDocumentSize}}
- choice:
precondition: true
when:
- expression:
simple:
expression: "${properties:maxDocumentSize:0} > 0"
steps:
# length() reads a stream to its end and closes it, so a stream body that
# stream caching (on by default) left uncached is read into bytes first; files,
# byte arrays and cached streams are measured in place, without a copy
- choice:
when:
- expression:
simple:
expression: "${body} is 'java.io.InputStream' && ${body} !is 'org.apache.camel.StreamCache'"
steps:
- convertBodyTo:
type: byte[]
- choice:
when:
- expression:
simple:
expression: "${length()} > {{maxDocumentSize}}"
steps:
- throwException:
exceptionType: java.lang.IllegalArgumentException
message: "Document '${exchangeProperty.CamelLangChain4jIngestDocumentId}' exceeds maxDocumentSize (${length()} > {{maxDocumentSize}} bytes)"
- convertBodyTo:
type: byte[]
- removeHeaders:
Expand Down
41 changes: 40 additions & 1 deletion library/camel-kamelets/src/main/resources/kamelets/tika-extract-text-action.kamelet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ spec:
after the parse all non-Camel headers are removed: past this action, parsed document
metadata is indistinguishable from caller headers, so it must not travel further. Each
format needs its Tika parser module on the classpath; camel-tika ships the HTML and
text modules, PDF needs org.apache.tika:tika-parser-pdf-module.
text modules, PDF needs org.apache.tika:tika-parser-pdf-module. With `maxDocumentSize`
set, a payload larger than that many bytes fails the exchange before the parse.
type: object
properties:
documentIdHeader:
Expand All @@ -51,6 +52,12 @@ spec:
CamelLangChain4jIngestDocumentId exchange property before the parse.
type: string
default: CamelLangChain4jIngestDocumentId
maxDocumentSize:
title: Max Document Size
description: Maximum size of the payload in bytes, checked before the parse; a larger
document fails the exchange without being parsed. 0, the default, means no limit.
type: integer
default: 0
dependencies:
- "camel:tika"
- "camel:kamelet"
Expand All @@ -65,6 +72,38 @@ spec:
expression:
simple:
expression: "${header[{{documentIdHeader}}]}"
# the size cap, present only when maxDocumentSize is set: the parse materializes the
# document, so an oversized one is rejected first. The precondition is evaluated once,
# when the Kamelet is instantiated - only then does ${properties:maxDocumentSize:0} see
# the parameter, at runtime it resolves to 0 - so the check must stay a precondition,
# and the comparison below uses {{maxDocumentSize}}
- choice:
precondition: true
when:
- expression:
simple:
expression: "${properties:maxDocumentSize:0} > 0"
steps:
# length() reads a stream to its end and closes it, so a stream body that
# stream caching (on by default) left uncached is read into bytes first; files,
# byte arrays and cached streams are measured in place, without a copy
- choice:
when:
- expression:
simple:
expression: "${body} is 'java.io.InputStream' && ${body} !is 'org.apache.camel.StreamCache'"
steps:
- convertBodyTo:
type: byte[]
- choice:
when:
- expression:
simple:
expression: "${length()} > {{maxDocumentSize}}"
steps:
- throwException:
exceptionType: java.lang.IllegalArgumentException
message: "Document '${exchangeProperty.CamelLangChain4jIngestDocumentId}' exceeds maxDocumentSize (${length()} > {{maxDocumentSize}} bytes)"
# tika answers plain text (body only, blocks newline-separated) at the pinned encoding
- to:
uri: tika
Expand Down
101 changes: 101 additions & 0 deletions ...kamelets-itest/src/test/resources/langchain4j-ingest/parser-max-size-route.citrus.it.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# ---------------------------------------------------------------------------
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ---------------------------------------------------------------------------

name: parser-max-size-route-test
actions:
# drain any request a previous test may have left on the shared server
- purge:
endpoints:
- name: "httpServer"
timeout: 100
- createVariables:
variables:
- name: "docling.serve.url"
value: "http://localhost:${http.server.port}"
- name: "report.url"
value: "http://localhost:${http.server.port}/result/parser-size"
# a fresh directory per test run, see the file-source test
- name: "ingest.dir"
value: "ingest-size-citrus:randomNumber(5)"

- camel:
cli:
run:
waitForRunningState: false
args:
- "--max-seconds=60"
integration:
file: "langchain4j-ingest/parser-max-size-route.yaml"
systemProperties:
properties:
- name: "docling.serve.url"
value: "${docling.serve.url}"
- name: "report.url"
value: "${report.url}"
- name: "ingest.dir"
value: "${ingest.dir}"

# under the cap: parsed as usual
- http:
server: "httpServer"
receiveRequest:
POST:
path: "/result/parser-size"
type: "plaintext"
body:
data: "@contains('parsed: Short text.')@"
- http:
server: "httpServer"
sendResponse:
response:
status: 200
reasonPhrase: "OK"
version: "HTTP/1.1"

# over the cap: rejected before the conversion; a conversion request instead would fail here
- http:
server: "httpServer"
receiveRequest:
POST:
path: "/result/parser-size"
type: "plaintext"
body:
data: "@matches('rejected: Document .long.pdf. exceeds maxDocumentSize .100 > 40 bytes.')@"
- http:
server: "httpServer"
sendResponse:
response:
status: 200
reasonPhrase: "OK"
version: "HTTP/1.1"

# a file over the cap: rejected before the parse
- http:
server: "httpServer"
receiveRequest:
POST:
path: "/result/parser-size"
type: "plaintext"
body:
data: "@matches('rejected: Document .big.txt. exceeds maxDocumentSize .100 > 40 bytes.')@"
- http:
server: "httpServer"
sendResponse:
response:
status: 200
reasonPhrase: "OK"
version: "HTTP/1.1"
Loading
Loading