Skip to content

fix(sessions): keep inlineData mimeType when encoding or decoding content - #1607

Open
hermanrous wants to merge 1 commit into
google:mainfrom
hermanrous:fix/sessions-inline-data-mimetype
Open

hermanrous wants to merge 1 commit into
google:mainfrom
hermanrous:fix/sessions-inline-data-mimetype

Conversation

@hermanrous

Copy link
Copy Markdown

What

SessionUtils.encodeContent and SessionUtils.decodeContent rebuilt every inline blob from scratch:

part.toBuilder().inlineData(Blob.builder().data(encodedData).build()).build()

Blob.builder().data(...) produces a Blob carrying only data, so mimeType and displayName are silently discarded. com.google.genai.types.Blob documents mimeType as "Required. The IANA standard MIME type of the source data.", and this is the live persistence path for Vertex AI sessions:

VertexAiSessionService.appendEvent → SessionJsonConverter.convertEventToJson → SessionUtils.encodeContent

and back on reload via fromApiEvent → SessionUtils.decodeContent.

So every multimodal event (image / audio / PDF inlineData) stored to Vertex AI loses its MIME type permanently, and decodeContent(encodeContent(content)) does not return the input either.

adk-python preserves inline_data.mime_type on the same path (model_dump(mode='json'), and _drop_vertex_unsupported_part_fields only pops part_metadata), and CONTRIBUTING.md names adk-python as the source of truth for validation.

How

Reuse the existing blob's builder instead of constructing a fresh Blob:

part.toBuilder().inlineData(inlineDataBlob.toBuilder().data(encodedData).build()).build()

The fields that are not being rewritten (mimeType, displayName) are preserved; displayName comes back for free, which is the same class of loss.

Backward compatible: the stored JSON simply gains a mimeType field that adk-python already writes, and sessions written before this change still read fine (mimeType stays absent, exactly as today).

Tests

New SessionUtilsTest covers encode, decode, the round trip, and the JSON that is actually persisted:

test before after
encodeContent_inlineData_preservesMimeType FAIL pass
decodeContent_inlineData_preservesMimeType FAIL pass
decodeContent_encodeContent_roundTripsInlineData FAIL pass
convertEventToJson_inlineData_persistsMimeType FAIL pass

On unmodified main all four fail:

[ERROR] Tests run: 4, Failures: 4, Errors: 0, Skipped: 0
[INFO] BUILD FAILURE

With the change applied:

[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[info] Processed 360 files (0 reformatted).
[INFO] BUILD SUCCESS

Verified with mvn -B -pl core -am -Dtest=SessionUtilsTest test on Java 21.

Note

I used an AI assistant to draft this change. I reviewed it, ran the build and the tests locally, and verified each claim in this description myself.

…tent

SessionUtils.encodeContent and SessionUtils.decodeContent rebuilt each inline
blob with Blob.builder().data(...), which produces a Blob carrying only `data`
and silently drops `mimeType` and `displayName`. `mimeType` is documented as
required on com.google.genai.types.Blob, and this is the persistence path used
by VertexAiSessionService (appendEvent -> SessionJsonConverter.convertEventToJson
-> encodeContent, and back through fromApiEvent -> decodeContent), so every
multimodal event stored to Vertex AI lost its MIME type permanently. The round
trip decodeContent(encodeContent(content)) did not return the input either.

Reuse the existing blob's builder so the remaining fields are preserved, and add
tests covering encode, decode, the round trip and the JSON that is persisted.
@google-cla

google-cla Bot commented Oct 4, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant