fix(provenance): make one length floor the whole substitution rule (#6551)

* fix(provenance): make one length floor the whole substitution rule

A literal shorter than eight characters is no longer substituted anywhere. It
was already the floor for matches inside a larger token; below it a second tier
still substituted whenever the hit sat on a word boundary — standing alone,
delimited, or as the whole value — on the theory that those positions made the
hit unambiguous.

Position is not the variable that matters. A hit on `7` is uninformative wherever
it sits, because the value space is ten. That tier rewrote `_raw_idx = 7` into
`_raw_idx = {{WEEKLY_OWNWORK_TTL}}` for the one shard whose index collided with a
TTL variable, and turned 2,000 boolean `had_error` cells into `[REDACTED_SECRET]`
because a `*_ENABLED` variable held `false`. Each was patched with a per-value
exception list; the floor subsumes both, so the lists are deleted.

With no literal below the floor reaching a matcher, the tier's machinery is
unreachable and goes with it: the match-policy type, its classifier, the word
boundary test, and the detect/render mode that existed only to select between
them. One constant now governs the question.

The cost is explicit and accepted: a secret shorter than eight characters is no
longer redacted from logs or model-visible content. Substitution cannot hide a
value that short — an observer who can read the surrounding text can enumerate
it. Two tests that pinned the old tier are rewritten to pin this, rather than
deleted, so the trade stays visible.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(provenance): restore two tests that the length floor had hollowed out

Greptile caught the byte-limit test: lengthening the catalog fixture without the
`recordResolved` value beside it left them mismatched, so the registry latched and
`projectResolvedSecretModelJsonContent` returned `{ safe: false }` at its
completeness guard — before any alias projection. The test passed while covering
nothing it is named for. It now records the same literal it catalogs, sizes the
limit between the raw bytes and the projected bytes, and asserts both directions
so a limit applied to the wrong side fails it.

Auditing every changed test for the same shape — a `recordResolved` value that
does not match its own catalog entry — found one more, pre-dating this branch: the
legacy-memory test drew its teeth from substituting the one-character secret `x`
inside `Box`, which the floor no longer substitutes, so it too had become vacuous.
Its fixture is now a full-length secret that appears in the message, which is what
makes "not projected" meaningful.

The three remaining mismatches are deliberate: those tests are about a resolution
that fails to verify.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(provenance): give the label-protection test a live matcher again

Cursor caught the third instance of the same shape: the fixture `TOK` is three
characters, so the floor drops it and the matcher is empty. The assertion was
`'Bearer {{TOKEN}}'` in and out, which an empty matcher satisfies exactly as well
as working label protection — the test could no longer tell them apart.

Use a fixture over the floor whose label still contains its own plaintext, and
project the bare plaintext first. That control fails if the matcher is inert, so
the atomic-label assertion beside it can only pass for the right reason.

Auditing every test on this branch for the shape — all fixtures below the floor —
returned twenty-four, but the rest are sound: most assert that nothing is
substituted, which the floor makes more certain rather than less, and the
remainder run through the resolver's causal path, which never consults a matcher.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(provenance): make the byte-limit test reject at the gate it names

Three limits can reject this value and my previous fix still tripped the wrong
one. `projectResolvedSecretModelJsonContent` checks the raw encoding, then walks
the content against a running budget, then re-encodes the projected object — and
only the last is what the test is for. At 16 the walk charged the key `a` and
then measured the 17-byte alias against the remaining 15, so it failed before the
re-encoding ran; at 25 everything passed. Neither assertion touched the check.

Twenty is the band that isolates it: the walk admits the alias against its
remaining 19, so a rejection there can only come from re-encoding the 25-byte
result. Asserting the content projection succeeds at the same limit pins that,
and deleting the re-encoding check now fails the test rather than leaving it
green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: icecrasher321

apps/sim/executor/execution/block-executor.test.ts

@@ -530,15 +530,15 @@ describe('BlockExecutor', () => {
     const resolver = new VariableResolver(workflow, {}, state)
     const onBlockComplete = vi.fn(async () => {})
     const registry = new ResolvedSecretTraceRegistry([
-      { name: 'SHORT_SECRET', plaintext: 'Test', encryptedValue: 'encrypted-test' },
+      { name: 'SHORT_SECRET', plaintext: 'TestValue', encryptedValue: 'encrypted-test' },
     ])
     const handler: BlockHandler = {
       canHandle: () => true,
       execute: async (blockContext, block) => {
         if (block.id === secretBlock.id) {
-          blockContext.resolvedSecretTraceRegistry?.recordResolved('SHORT_SECRET', 'Test')
+          blockContext.resolvedSecretTraceRegistry?.recordResolved('SHORT_SECRET', 'TestValue')
         }
-        return { result: 'Test' }
+        return { result: 'TestValue' }
       },
     }
     const executor = new BlockExecutor([handler], resolver, { onBlockComplete }, state)

apps/sim/executor/handlers/agent/memory.test.ts

@@ -347,14 +347,14 @@ describe('Memory', () => {
       expect(result.content).toBe('foreign-secret')
     })
 
-    it.each(['123'])(
+    it.each(['12345678'])(
       'projects short secret %s only in model text and arguments',
       async (secret) => {
         const registry = new ResolvedSecretTraceRegistry([
           { name: 'TOKEN', plaintext: secret, encryptedValue: 'ciphertext' },
         ])
         registry.recordResolved('TOKEN', secret)
-        const converted = secret === '123' ? 123 : true
+        const converted = secret === '12345678' ? 12345678 : true
         const message: Message = {
           role: 'assistant',
           content: `Result: ${secret}`,
@@ -465,11 +465,11 @@ describe('Memory', () => {
 
     it('does not project unrelated active secrets into legacy memory', async () => {
       const registry = new ResolvedSecretTraceRegistry([
-        { name: 'TOKEN', plaintext: 'x', encryptedValue: 'ciphertext' },
+        { name: 'TOKEN', plaintext: 'unrelated-secret', encryptedValue: 'ciphertext' },
       ])
-      registry.recordResolved('TOKEN', 'x')
+      expect(registry.recordResolved('TOKEN', 'unrelated-secret')).toBe(true)
       vi.spyOn(memoryService as any, 'fetchMemory').mockResolvedValueOnce({
-        messages: [{ role: 'assistant', content: 'Box' }],
+        messages: [{ role: 'assistant', content: 'Box unrelated-secret' }],
         provenance: { status: 'exact', entries: [] },
       })
 
@@ -478,7 +478,7 @@ describe('Memory', () => {
         inputs
       )
 
-      expect(messages).toEqual([{ role: 'assistant', content: 'Box' }])
+      expect(messages).toEqual([{ role: 'assistant', content: 'Box unrelated-secret' }])
     })
 
     it('does not activate provenance from a message dropped by the selected window', async () => {

apps/sim/executor/handlers/pi/search/tool.test.ts

@@ -36,7 +36,7 @@ function executionContext(
 const ctx = executionContext()
 
 function buildTool(provider: 'exa' | 'serper' | 'parallel' | 'firecrawl' = 'exa', context = ctx) {
-  return buildPiSearchToolSpec(context, { provider, apiKey: 'key-123' }, 'local')
+  return buildPiSearchToolSpec(context, { provider, apiKey: 'key-1234567' }, 'local')
 }
 
 async function run(
@@ -69,7 +69,7 @@ describe('buildPiSearchToolSpec', () => {
 
     const [toolId, params, options] = mockExecuteTool.mock.calls[0]
     expect(toolId).toBe('exa_search')
-    expect(params.apiKey).toBe('key-123')
+    expect(params.apiKey).toBe('key-1234567')
     expect(params.timeout).toBe(10_000)
     expect(options.executionContext).toBe(ctx)
     expect(options.resolvedSecretTraceRegistry).toBeInstanceOf(ResolvedSecretTraceRegistry)
@@ -83,7 +83,7 @@ describe('buildPiSearchToolSpec', () => {
 
     const [toolId, params] = mockExecuteTool.mock.calls[0]
     expect(toolId).toBe('serper_search')
-    expect(params).toEqual({ query: 'pi', num: 2, apiKey: 'key-123', timeout: 10_000 })
+    expect(params).toEqual({ query: 'pi', num: 2, apiKey: 'key-1234567', timeout: 10_000 })
   })
 
   it('normalizes a successful provider response into the envelope', async () => {
@@ -193,17 +193,17 @@ describe('buildPiSearchToolSpec', () => {
 
   it('projects only the exact resolver-recorded search key and leaves the raw result unchanged', async () => {
     const registry = new ResolvedSecretTraceRegistry([
-      { name: 'SEARCH_KEY', plaintext: 'key-123', encryptedValue: 'search-ciphertext' },
+      { name: 'SEARCH_KEY', plaintext: 'key-1234567', encryptedValue: 'search-ciphertext' },
       { name: 'UNRELATED', plaintext: 'Test', encryptedValue: 'unrelated-ciphertext' },
     ])
-    registry.recordResolvedAtInputPath('SEARCH_KEY', 'key-123', ['searchApiKey'])
-    registry.recordResolvedInputProjection(['searchApiKey'], 'key-123', '{{SEARCH_KEY}}')
+    registry.recordResolvedAtInputPath('SEARCH_KEY', 'key-1234567', ['searchApiKey'])
+    registry.recordResolvedInputProjection(['searchApiKey'], 'key-1234567', '{{SEARCH_KEY}}')
     registry.recordResolvedAtInputPath('UNRELATED', 'Test', ['task'])
     registry.recordResolvedInputProjection(['task'], 'Test', '{{UNRELATED}}')
     const output = {
       results: [
         {
-          title: 'key-123',
+          title: 'key-1234567',
           url: 'https://example.com/docs',
           text: 'Test',
         },
@@ -213,7 +213,7 @@ describe('buildPiSearchToolSpec', () => {
 
     const result = await buildPiSearchToolSpec(
       executionContext(registry),
-      { provider: 'exa', apiKey: 'key-123' },
+      { provider: 'exa', apiKey: 'key-1234567' },
       'local',
       '{{SEARCH_KEY}}'
     ).execute({ query: 'pi' })
@@ -231,7 +231,7 @@ describe('buildPiSearchToolSpec', () => {
     expect(output).toEqual({
       results: [
         {
-          title: 'key-123',
+          title: 'key-1234567',
           url: 'https://example.com/docs',
           text: 'Test',
         },

apps/sim/executor/utils/resolved-secret-content-projection.test.ts

@@ -161,37 +161,37 @@ describe('projectResolvedSecretModelContent', () => {
 
   it('keeps longest-match semantics when a known opaque placeholder is nested in a secret', () => {
     const registry = new ResolvedSecretTraceRegistry([
-      { name: 'Test', plaintext: 'Test', encryptedValue: 'test-ciphertext' },
+      { name: 'TestName', plaintext: 'TestName', encryptedValue: 'test-ciphertext' },
       {
         name: 'COMPOSITE',
-        plaintext: 'x{{Test}}y',
+        plaintext: 'x{{TestName}}y',
         encryptedValue: 'composite-ciphertext',
       },
     ])
-    registry.recordResolved('Test', 'Test')
-    registry.recordResolved('COMPOSITE', 'x{{Test}}y')
+    registry.recordResolved('TestName', 'TestName')
+    registry.recordResolved('COMPOSITE', 'x{{TestName}}y')
 
-    expect(projectResolvedSecretModelContent('x{{Test}}y', registry)).toEqual({
+    expect(projectResolvedSecretModelContent('x{{TestName}}y', registry)).toEqual({
       safe: true,
       value: '{{COMPOSITE}}',
     })
   })
 
   it('projects exact typed numeric secrets, leaving booleans and null identifying nothing', () => {
     const registry = new ResolvedSecretTraceRegistry([
-      { name: 'NUMBER', plaintext: '123', encryptedValue: 'number-ciphertext' },
+      { name: 'NUMBER', plaintext: '12345678', encryptedValue: 'number-ciphertext' },
       { name: 'BOOLEAN', plaintext: 'true', encryptedValue: 'boolean-ciphertext' },
       { name: 'NULL', plaintext: 'null', encryptedValue: 'null-ciphertext' },
     ])
-    registry.recordResolved('NUMBER', '123')
+    registry.recordResolved('NUMBER', '12345678')
     registry.recordResolved('BOOLEAN', 'true')
     registry.recordResolved('NULL', 'null')
 
     expect(
       projectResolvedSecretModelContent(
         {
-          strings: ['123', 'true', 'null'],
-          number: 123,
+          strings: ['12345678', 'true', 'null'],
+          number: 12345678,
           boolean: true,
           nothing: null,
           unrelatedNumber: 1234,
@@ -212,12 +212,12 @@ describe('projectResolvedSecretModelContent', () => {
     })
   })
 
-  it.each(['123'])('keeps projected JSON argument strings valid (%s)', (secret) => {
+  it.each(['12345678'])('keeps projected JSON argument strings valid (%s)', (secret) => {
     const registry = new ResolvedSecretTraceRegistry([
       { name: 'TOKEN', plaintext: secret, encryptedValue: 'ciphertext' },
     ])
     registry.recordResolved('TOKEN', secret)
-    const typedValue = secret === '123' ? 123 : true
+    const typedValue = secret === '12345678' ? 12345678 : true
 
     const projection = projectResolvedSecretModelJsonStrings(
       [JSON.stringify({ secret, converted: typedValue, nested: [typedValue] })],
@@ -255,50 +255,59 @@ describe('projectResolvedSecretModelContent', () => {
 
   it('is stable when a secret literal overlaps its own provenance alias', () => {
     const registry = new ResolvedSecretTraceRegistry([
-      { name: 'TOKEN', plaintext: 'TOKEN', encryptedValue: 'ciphertext' },
+      { name: 'TOKEN', plaintext: 'TOKENTOKEN', encryptedValue: 'ciphertext' },
     ])
-    registry.recordResolved('TOKEN', 'TOKEN')
+    registry.recordResolved('TOKEN', 'TOKENTOKEN')
 
-    const first = projectResolvedSecretModelContent('Bearer TOKEN', registry)
+    const first = projectResolvedSecretModelContent('Bearer TOKENTOKEN', registry)
     expect(first).toEqual({ safe: true, value: 'Bearer {{TOKEN}}' })
     if (!first.safe) return
     expect(projectResolvedSecretModelContent(first.value, registry)).toEqual(first)
   })
 
   it('preserves the canonical provenance label when its name equals the secret plaintext', () => {
     const registry = new ResolvedSecretTraceRegistry([
-      { name: 'Test', plaintext: 'Test', encryptedValue: 'ciphertext' },
+      { name: 'TestName', plaintext: 'TestName', encryptedValue: 'ciphertext' },
     ])
-    registry.recordResolved('Test', 'Test')
+    registry.recordResolved('TestName', 'TestName')
 
     expect(
       projectResolvedSecretModelContent(
         {
-          result: 'Test',
-          source: 'return {{Test}}',
-          error: "NameError: name 'Test' is not defined",
+          result: 'TestName',
+          source: 'return {{TestName}}',
+          error: "NameError: name 'TestName' is not defined",
         },
         registry
       )
     ).toEqual({
       safe: true,
       value: {
-        result: '{{Test}}',
-        source: 'return {{Test}}',
-        error: "NameError: name '{{Test}}' is not defined",
+        result: '{{TestName}}',
+        source: 'return {{TestName}}',
+        error: "NameError: name '{{TestName}}' is not defined",
       },
     })
   })
 
   it('atomically projects the selected provenance label when its name contains the value', () => {
     const registry = new ResolvedSecretTraceRegistry([
-      { name: 'TOKEN', plaintext: 'TOK', encryptedValue: 'ciphertext' },
+      { name: 'TOKENVALUE_NAME', plaintext: 'TOKENVALUE', encryptedValue: 'ciphertext' },
     ])
-    registry.recordResolved('TOKEN', 'TOK')
-
-    expect(projectResolvedSecretModelContent('Bearer {{TOKEN}}', registry)).toEqual({
+    expect(registry.recordResolved('TOKENVALUE_NAME', 'TOKENVALUE')).toBe(true)
+
+    /**
+     * The label `{{TOKENVALUE_NAME}}` contains the plaintext that produced it, so it is only left
+     * alone because the label is treated atomically. Projecting the bare plaintext first proves the
+     * matcher is live — without it an empty matcher would satisfy the second assertion too.
+     */
+    expect(projectResolvedSecretModelContent('Bearer TOKENVALUE', registry)).toEqual({
+      safe: true,
+      value: 'Bearer {{TOKENVALUE_NAME}}',
+    })
+    expect(projectResolvedSecretModelContent('Bearer {{TOKENVALUE_NAME}}', registry)).toEqual({
       safe: true,
-      value: 'Bearer {{TOKEN}}',
+      value: 'Bearer {{TOKENVALUE_NAME}}',
     })
   })
 
@@ -308,7 +317,7 @@ describe('projectResolvedSecretModelContent', () => {
       complete: true,
       matches: [
         {
-          plaintext: 'x'.repeat(64 * 1024),
+          plaintext: 'xxxxxxxx'.repeat(64 * 1024),
           replacement: '[REDACTED_SECRET]',
         },
       ],
@@ -321,16 +330,18 @@ describe('projectResolvedSecretModelContent', () => {
 
   it('keeps provenance-shaped content deterministic without trusting it as a protocol handle', () => {
     const registry = new ResolvedSecretTraceRegistry([
-      { name: 'Test', plaintext: 'Test', encryptedValue: 'ciphertext' },
+      { name: 'TestName', plaintext: 'TestName', encryptedValue: 'ciphertext' },
     ])
-    registry.recordResolved('Test', 'Test')
+    registry.recordResolved('TestName', 'TestName')
 
-    expect(projectResolvedSecretModelContent('{{Test}}', registry)).toEqual({
+    expect(projectResolvedSecretModelContent('{{TestName}}', registry)).toEqual({
       safe: true,
-      value: '{{Test}}',
+      value: '{{TestName}}',
     })
-    expect(isResolvedSecretModelContentUnchanged('{{Test}}', registry)).toBe(false)
-    expect(isResolvedSecretModelContentUnchanged(['resource', '{{Test}}'], registry)).toBe(false)
+    expect(isResolvedSecretModelContentUnchanged('{{TestName}}', registry)).toBe(false)
+    expect(isResolvedSecretModelContentUnchanged(['resource', '{{TestName}}'], registry)).toBe(
+      false
+    )
     expect(isResolvedSecretModelContentUnchanged(['resource', 'safe'], registry)).toBe(true)
   })
 })
@@ -409,11 +420,29 @@ describe('projectResolvedSecretModelJsonContent', () => {
 
   it('enforces the byte limit after secret aliases are projected', () => {
     const registry = new ResolvedSecretTraceRegistry([
-      { name: 'X', plaintext: 'x', encryptedValue: 'ciphertext' },
+      { name: 'X_LONGER_NAME', plaintext: 'xxxxxxxx', encryptedValue: 'ciphertext' },
     ])
-    registry.recordResolved('X', 'x')
-
-    expect(projectResolvedSecretModelJsonContent({ a: 'x' }, registry, 9)).toEqual({ safe: false })
+    expect(registry.recordResolved('X_LONGER_NAME', 'xxxxxxxx')).toBe(true)
+
+    /**
+     * Three separate limits can reject this value, and only the last one is what this test is for:
+     * the raw encoding (16 bytes), the content walk's running budget, and the JSON re-encoding of
+     * the projected object (25 bytes). A limit of 20 is the only band that isolates the third —
+     * the walk charges the key `a` and then admits the 17-byte alias against the remaining 19, so
+     * anything that rejects at 20 can only be the wire check. Pinning both halves keeps it that
+     * way: drop the re-encoding check and the second assertion starts passing.
+     */
+    expect(projectResolvedSecretModelContent({ a: 'xxxxxxxx' }, registry, 20)).toEqual({
+      safe: true,
+      value: { a: '{{X_LONGER_NAME}}' },
+    })
+    expect(projectResolvedSecretModelJsonContent({ a: 'xxxxxxxx' }, registry, 20)).toEqual({
+      safe: false,
+    })
+    expect(projectResolvedSecretModelJsonContent({ a: 'xxxxxxxx' }, registry, 25)).toEqual({
+      safe: true,
+      value: { a: '{{X_LONGER_NAME}}' },
+    })
   })
 })
 
@@ -445,7 +474,7 @@ describe('projectResolvedSecretDiagnosticError', () => {
 
   it('sanitizes an inactive compiler alias without activating or scanning its secret', () => {
     const registry = new ResolvedSecretTraceRegistry([
-      { name: 'X', plaintext: 'x', encryptedValue: 'ciphertext' },
+      { name: 'X', plaintext: 'xxxxxxxx', encryptedValue: 'ciphertext' },
     ])
     const error = new Error('Box __var_X')
 

apps/sim/executor/utils/resolved-secret-content-projection.ts

@@ -38,7 +38,6 @@ function createResolvedSecretModelMatcher(
 ): ResolvedSecretMatcher | undefined {
   const matcher = createResolvedSecretMatcher(matches, {
     preserveNamedProvenanceLabels: true,
-    mode: 'render',
   })
   if (!matcher) return undefined
 
@@ -75,7 +74,7 @@ function createResolvedSecretModelMatcher(
       })),
       ...opaquePlaceholderMatches,
     ],
-    { preserveNamedProvenanceLabels: true, mode: 'render' }
+    { preserveNamedProvenanceLabels: true }
   )
 }