Skip to content

Fix Job.payload/result MySQL column size (text 64KB -> longtext) - #38

Merged
buzz-code merged 1 commit into
mainfrom
claude/job-longtext-columns-fix
Aug 19, 2026
Merged

buzz-code merged 1 commit into
mainfrom
claude/job-longtext-columns-fix

Conversation

@buzz-code

Copy link
Copy Markdown
Owner

What

JsonColumn (backed by TypeORM's simple-json) always maps to MySQL text (64KB) — the mysql driver hardcodes this regardless of any type override. Job.result can hold a base64-encoded zip from the user-data-export handler when S3 isn't configured, which routinely exceeds 64KB and fails to save with Data too long for column 'result'.

Found by Copilot review on buzz-code/react-admin-nestjs#141.

Fix

  • New LongJsonColumn helper: LongTextColumn + a JSON transformer (same to/from shape as JsonColumn), so MySQL gets longtext / SQLite gets text (unlimited there anyway).
  • Job.payload and Job.result now use it.

Testing

  • New unit tests for LongJsonColumn (options wiring, round-trip, null handling, custom-transformer override) — 4/4 passing.
  • Full jobs suite (25 tests) + tsc --noEmit (0 errors) + e2e app boot — all green.
  • Verified directly against real MySQL (MariaDB): a longtext column accepts a 200KB value; the same value against a text column reproduces the exact Data too long for column error this fixes.

🤖 Generated with Claude Code

https://claude.ai/code/session_011dEYULADLgGNfZ4nv6gz8s


Generated by Claude Code

JsonColumn (simple-json) always maps to MySQL text (64KB) regardless
of any type override -- TypeORM's mysql driver hardcodes this. The
user-data-export handler can return a base64 zip in Job.result when
S3 isn't configured, which routinely exceeds 64KB and fails to save
with "Data too long for column 'result'".

Add LongJsonColumn (LongTextColumn + a JSON transformer) and use it
for Job.payload/Job.result. Verified against real MySQL: a longtext
column accepts a 200KB value that a text column rejects.

Found by Copilot review on buzz-code/react-admin-nestjs#141.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011dEYULADLgGNfZ4nv6gz8s
Copilot AI lite review requested due to automatic review settings August 19, 2026 16:05
@buzz-code
buzz-code merged commit 0c3ea44 into main Aug 19, 2026
1 check passed
buzz-code pushed a commit to buzz-code/react-admin-nestjs that referenced this pull request Aug 19, 2026
jobs.result can hold a base64-encoded zip from the user-data-export
handler when S3 isn't configured, which routinely exceeds MySQL
text's 64KB limit. Widen to longtext (matches the entity-side fix in
buzz-code/nra-server#38, picked up by the server/shared bump).

Verified against real MySQL: a longtext column now accepts a 200KB
value that the previous text column would have rejected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011dEYULADLgGNfZ4nv6gz8s

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new TypeORM column decorator helper to store JSON in a MySQL longtext column (instead of text via simple-json), and migrates Job.payload / Job.result to use it so large JSON payloads (e.g., base64-encoded zips) can be persisted without hitting the 64KB limit.

Changes:

  • Add LongJsonColumn implemented as LongTextColumn + explicit JSON transformer (bypassing simple-json’s MySQL text mapping).
  • Update Job.payload and Job.result to use @LongJsonColumn({ nullable: true }).
  • Add unit tests validating LongJsonColumn’s options wiring and transformer behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
utils/entity/column-types.util.ts Adds LongJsonColumn helper built on LongTextColumn with a JSON transformer to avoid MySQL text size limits.
utils/entity/tests/column-types.util.spec.ts Adds unit tests covering LongJsonColumn wiring/round-trip/null handling/custom transformer override.
entities/Job.entity.ts Switches payload and result columns from JsonColumn to LongJsonColumn to support larger stored results.
Suppressed comments (1)

utils/entity/column-types.util.ts:164

  • The default transformer from callback is typed as (value: string), but the implementation explicitly handles null/undefined and the tests call it with null. Using any (or unknown) here better matches TypeORM's ValueTransformer contract and avoids potential strictFunctionTypes issues.
      from: (value: string) => (value == null || value === '' ? null : JSON.parse(value)),

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

* instead.
*/
export function LongJsonColumn(options?: ColumnOptions) {
const { transformer, ...rest } = options ?? {};
Comment on lines +21 to +27
it('backs the column with a longtext-mapped type (via LongTextColumn)', () => {
LongJsonColumn({ nullable: true });
expect(columnCalls).toHaveLength(1);
const [, options] = columnCalls[0];
expect(options.nullable).toBe(true);
expect(options.transformer).toBeDefined();
});
Comment thread entities/Job.entity.ts
Comment on lines +35 to 39
@LongJsonColumn({ nullable: true })
payload: Record<string, any>;

@JsonColumn({ nullable: true })
@LongJsonColumn({ nullable: true })
result: Record<string, any>;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants