Repository navigation
Expand file tree
/
Copy pathpattern.html
More file actions
442 lines (362 loc) · 21.7 KB
/
Copy pathpattern.html
File metadata and controls
442 lines (362 loc) · 21.7 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Before the LLM did the mapping</title>
<meta name="description" content="A 2017 restaurant food-costing implementation, done by hand, read for what it says about systems where a model does the semantic mapping and data entry. The claims that generalize are stated without the restaurant nouns.">
<style>
:root {
--paper: #F7F8F5;
--paper-raised: #FFFFFF;
--tint: #ECF1EC;
--rule: #D8DCD3;
--rule-strong: #AEB6AB;
--ink: #1B211D;
--ink-soft: #545C53;
--ink-faint: #7E867C;
--accent: #2E5D4B;
--accent-dim: #7FA394;
--font-text: Charter, "Bitstream Charter", "Sitka Text", "Iowan Old Style", Cambria, Georgia, serif;
--font-ui: "Segoe UI", system-ui, -apple-system, "Helvetica Neue", sans-serif;
--font-mono: "Cascadia Mono", Consolas, "SF Mono", ui-monospace, "Roboto Mono", monospace;
--measure: 68ch;
--step: 1.6rem;
}
@media (prefers-color-scheme: dark) {
:root:not([data-theme="light"]) {
--paper: #12160F;
--paper-raised: #191E17;
--tint: #1D2419;
--rule: #2C3329;
--rule-strong: #454E41;
--ink: #E5E9DF;
--ink-soft: #A5ADA0;
--ink-faint: #7C8478;
--accent: #79C4A0;
--accent-dim: #3F6B57;
}
}
:root[data-theme="dark"] {
--paper: #12160F;
--paper-raised: #191E17;
--tint: #1D2419;
--rule: #2C3329;
--rule-strong: #454E41;
--ink: #E5E9DF;
--ink-soft: #A5ADA0;
--ink-faint: #7C8478;
--accent: #79C4A0;
--accent-dim: #3F6B57;
}
body {
background: var(--paper);
color: var(--ink);
font-family: var(--font-text);
font-size: 1.0625rem;
line-height: 1.65;
-webkit-font-smoothing: antialiased;
}
.sheet {
max-width: 78ch;
margin: 0 auto;
padding: clamp(2rem, 6vw, 5rem) clamp(1.15rem, 5vw, 3rem) 6rem;
display: flex;
flex-direction: column;
gap: var(--step);
}
.masthead {
display: flex;
flex-direction: column;
gap: 1.1rem;
padding-bottom: 2rem;
border-bottom: 2px solid var(--rule-strong);
}
.eyebrow {
font-family: var(--font-ui);
font-size: 0.7rem;
font-weight: 600;
letter-spacing: 0.14em;
text-transform: uppercase;
color: var(--accent);
margin: 0;
}
h1 {
font-size: clamp(1.85rem, 5vw, 2.9rem);
line-height: 1.12;
font-weight: 600;
letter-spacing: -0.015em;
text-wrap: balance;
margin: 0;
}
.standfirst {
font-size: 1.18rem;
line-height: 1.5;
color: var(--ink-soft);
max-width: var(--measure);
text-wrap: pretty;
margin: 0;
}
.byline {
border-left: 3px solid var(--accent);
padding: 0.15rem 0 0.15rem 1.15rem;
font-size: 0.98rem;
line-height: 1.6;
color: var(--ink-soft);
max-width: var(--measure);
margin: 0;
}
section {
display: flex;
flex-direction: column;
gap: 1.15rem;
padding-top: 1.4rem;
}
section > * { max-width: var(--measure); }
section > .wide { max-width: none; }
h2 {
font-size: clamp(1.35rem, 3.4vw, 1.85rem);
line-height: 1.2;
font-weight: 600;
letter-spacing: -0.01em;
text-wrap: balance;
margin: 1.2rem 0 0;
padding-top: 1.3rem;
border-top: 1px solid var(--rule);
}
h3 {
font-family: var(--font-ui);
font-size: 0.78rem;
font-weight: 700;
letter-spacing: 0.1em;
text-transform: uppercase;
color: var(--ink-soft);
margin: 1rem 0 0;
}
p { margin: 0; text-wrap: pretty; }
strong { font-weight: 600; color: var(--ink); }
em { font-style: italic; }
ul {
margin: 0;
padding-left: 1.15rem;
display: flex;
flex-direction: column;
gap: 0.5rem;
}
li::marker { color: var(--accent-dim); }
.num, td.n {
font-family: var(--font-mono);
font-variant-numeric: tabular-nums;
}
.scroller {
overflow-x: auto;
border-block: 1px solid var(--rule-strong);
margin: 0.6rem 0;
}
table {
border-collapse: collapse;
width: 100%;
min-width: 34rem;
font-family: var(--font-ui);
font-size: 0.9rem;
}
caption {
caption-side: top;
text-align: left;
font-family: var(--font-ui);
font-size: 0.72rem;
font-weight: 600;
letter-spacing: 0.1em;
text-transform: uppercase;
color: var(--ink-faint);
padding-bottom: 0.6rem;
}
thead th {
text-align: left;
font-size: 0.7rem;
font-weight: 700;
letter-spacing: 0.09em;
text-transform: uppercase;
color: var(--ink-faint);
padding: 0.6rem 0.9rem;
border-bottom: 1px solid var(--rule-strong);
white-space: nowrap;
}
tbody td {
padding: 0.62rem 0.9rem;
border-bottom: 1px solid var(--rule);
vertical-align: baseline;
}
tbody tr:nth-child(even) { background: var(--tint); }
th.n, td.n { text-align: right; white-space: nowrap; }
tbody tr.total td {
border-bottom: none;
border-top: 2px solid var(--rule-strong);
font-weight: 700;
color: var(--ink);
background: none;
}
td.verdict { color: var(--ink-soft); }
.bar { display: block; height: 5px; min-width: 2px; background: var(--accent); border-radius: 1px; }
.bar-cell { width: 5.5rem; padding-right: 1.1rem; }
.bar-track { display: block; height: 5px; background: var(--rule); border-radius: 1px; }
.visually-hidden {
position: absolute; width: 1px; height: 1px; margin: -1px; padding: 0;
overflow: hidden; clip-path: inset(50%); white-space: nowrap;
}
.callout {
background: var(--paper-raised);
border: 1px solid var(--rule);
border-left: 3px solid var(--accent);
padding: 1.15rem 1.35rem;
display: flex;
flex-direction: column;
gap: 0.75rem;
}
.headline-figure {
font-family: var(--font-mono);
font-variant-numeric: tabular-nums;
font-size: clamp(1.1rem, 2.6vw, 1.35rem);
line-height: 1.45;
color: var(--ink);
margin: 0;
}
.rule-box {
border: 1px solid var(--rule-strong);
padding: 0.9rem 1.15rem;
font-family: var(--font-ui);
font-size: 0.95rem;
line-height: 1.55;
color: var(--ink);
}
.rule-box .tag {
display: block;
font-size: 0.68rem;
font-weight: 700;
letter-spacing: 0.1em;
text-transform: uppercase;
color: var(--accent);
margin-bottom: 0.35rem;
}
.endnote {
border-top: 1px solid var(--rule);
margin-top: 2.4rem;
padding-top: 1.4rem;
display: flex;
flex-direction: column;
gap: 0.85rem;
font-size: 0.92rem;
line-height: 1.6;
color: var(--ink-soft);
max-width: var(--measure);
}
.endnote h3 { margin-top: 0; }
a { color: var(--accent); text-underline-offset: 0.15em; }
a:focus-visible {
outline: 2px solid var(--accent);
outline-offset: 2px;
}
</style>
<article class="sheet">
<header class="masthead">
<p class="eyebrow">Field report · For a builder</p>
<h1>Before the LLM did the mapping</h1>
<p class="standfirst">You are building a system where a model does the semantic mapping and the data entry, and then does something with the result. I did the mapping and entry by hand in 2017, for restaurant food cost across 15 locations. This is what it cost, which lines a model removes, and what I learned about the material being mapped. What your system does after the data is in is outside what I can report on.</p>
<p class="byline">This is a re-cut of a longer field report on the actual-vs-theoretical food costing implementation I ran at Mendocino Farms from 2017 to 2020, across 15 locations and two commissaries. The restaurant specifics stay in, because they are the evidence. The full report is at <a href="https://chodizzle.github.io/avt-implementation/">chodizzle.github.io/avt-implementation</a>. Figures are reconstructed from memory. See the note on method at the end.</p>
</header>
<section>
<h2>What it cost</h2>
<p>The work: vendor invoice lines were mapped onto inventory items, inventory items onto recipe ingredients, recipe ingredients onto POS menu objects. Four namespaces, about 5,000 vendor identifiers, 200–300 inventory items, 400–500 recipes and sub-recipes, 2,000–3,000 POS objects. Recipes were entered into a recipe tree. Count sheets were built for every location. The software computed variance from all of it, and operators and corporate acted on the variance. About 1,100 hours over six to seven months, one full-time person, then about eight hours a week to keep it true.</p>
<p>Here are the line items, what they cost, and what I would expect a model to remove from each.</p>
<div class="scroller wide">
<table>
<caption>Same project, with a competent AI layer</caption>
<thead>
<tr>
<th scope="col">Line item</th>
<th scope="col" class="n">Then</th>
<th scope="col" class="n">Now</th>
<th scope="col" class="bar-cell"><span class="visually-hidden">Share remaining</span></th>
<th scope="col">Verdict</th>
</tr>
</thead>
<tbody>
<tr><td>Recipe validation</td><td class="n">~500</td><td class="n">~350</td><td class="bar-cell"><span class="bar-track"><span class="bar" style="width:70%"></span></span></td><td class="verdict">Trials do not move; targeting and record-keeping do</td></tr>
<tr><td>Inventory item creation</td><td class="n">40</td><td class="n">~10</td><td class="bar-cell"><span class="bar-track"><span class="bar" style="width:25%"></span></span></td><td class="verdict">Collapses</td></tr>
<tr><td>Recipe entry</td><td class="n">~100</td><td class="n">~15</td><td class="bar-cell"><span class="bar-track"><span class="bar" style="width:15%"></span></span></td><td class="verdict">Collapses</td></tr>
<tr><td>Vendor → inventory mapping</td><td class="n">~100</td><td class="n">~35</td><td class="bar-cell"><span class="bar-track"><span class="bar" style="width:35%"></span></span></td><td class="verdict">Mostly, with the exceptions below</td></tr>
<tr><td>POS → recipe mapping</td><td class="n">~60</td><td class="n">~10</td><td class="bar-cell"><span class="bar-track"><span class="bar" style="width:17%"></span></span></td><td class="verdict">Collapses</td></tr>
<tr><td>Count sheet build</td><td class="n">~100</td><td class="n">~40</td><td class="bar-cell"><span class="bar-track"><span class="bar" style="width:40%"></span></span></td><td class="verdict">Item list and units collapse; shelf order does not</td></tr>
<tr><td>Commissary transfers</td><td class="n">~80</td><td class="n">~72</td><td class="bar-cell"><span class="bar-track"><span class="bar" style="width:90%"></span></span></td><td class="verdict">Mechanics collapse; the costing decision does not</td></tr>
<tr><td>Invoice capture setup</td><td class="n">~60</td><td class="n">~15</td><td class="bar-cell"><span class="bar-track"><span class="bar" style="width:25%"></span></span></td><td class="verdict">Mostly</td></tr>
<tr><td>Live validation</td><td class="n">~80</td><td class="n">~25</td><td class="bar-cell"><span class="bar-track"><span class="bar" style="width:31%"></span></span></td><td class="verdict">Anomaly catching collapses; the phone call does not</td></tr>
<tr class="total"><td>Total</td><td class="n">~1,120</td><td class="n">~570</td><td class="bar-cell"><span class="bar-track"><span class="bar" style="width:51%"></span></span></td><td></td></tr>
</tbody>
</table>
</div>
<div class="callout">
<p class="headline-figure">Mapping, all of it combined, was <strong>15–20%</strong> of the project. Validation was <strong>45%</strong>, and it is the only line that does not collapse.</p>
</div>
<p>That is the number to hold onto, because mapping is the line everyone assumes is the expensive one. It was not. It was also not where the dangerous failures were. The dangerous failures were in the material being mapped.</p>
</section>
<section>
<h2>Where the cost actually was</h2>
<p>If the job had been mapping and entry, it would have taken about two months. The mandate was to <em>validate</em>: to double-check recipes accumulated over ten or fifteen years. The recipe book existed and it was well kept. Versions needed reconciling. That was not the expensive part.</p>
<p>The expensive part was that <strong>a document written for the person doing the work is not a document that supports arithmetic.</strong> Stores worked from instructions like <em>a two-ounce spoodle of crispy quinoa</em>. That is a good instruction and the kitchens executed it consistently. However, a spoodle is a volume tool, the density of crispy quinoa depends on how it fried, and the commissary yields and packs it by weight. The true grams per spoodle was a number that existed nowhere, because nobody had ever needed it.</p>
<p>The 500 hours were not typing. They were standing in kitchens during prep, running yields with cooks, and converting a corpus written for execution into one that could be costed. My stopping rule was three consecutive consistent yields. On a nine-step item (marinate, braise, cool, cut, fry, bag in juices, ship, reheat, serve) that took about ten trials, and the chain crossed from the commissary into a store I was not standing in.</p>
<p>The corpus was clean. It was consistently followed. It was in the wrong units, and nothing in the text said so.</p>
</section>
<section>
<h2>Two exhibits</h2>
<p>Both are from the mapping work, which is the work the model is best at. Neither is ambiguity. Ambiguity is the easy case: the model is uncertain, it flags, a human resolves it. These produce <strong>clean text, high confidence, and a wrong number</strong>, and a wrong number does not raise an error. Calibrated confidence does not help, because confidence is computed from the text and the text is fine.</p>
<h3>The consolidation trap</h3>
<p>Southern California stores reliably got avocados in 40–60 count cases. Northern California could only get 60–70. The vendor printed them as separate SKUs, so the data was not lying. However, the names were nearly identical, and the correct handling runs directly against the schema's own default.</p>
<p>The premise of vendor mapping is consolidation: many vendor identities collapse onto one real thing you count. My fan-out was five to seven vendor identifiers per inventory item, and that policy was right almost every time. Two near-identical avocado SKUs is exactly the shape it wants to merge. Merging them is the obvious, defensible, wrong answer: they are different physical units with different per-each yields going to different regions. The fix was in the schema, not the mapping: two items, forked recipes, region-gated POS maps. It took me a while to see it, and the failure was mine before it was anyone's.</p>
<div class="rule-box">
<span class="tag">General form</span>
Two entities the schema wants to merge. Same base identity, different spec, and they never appear together in the same context. The signal is in the co-occurrence structure, not in the text. Semantic similarity says merge; the join pattern says ask.
</div>
<h3>The execution-unit trap</h3>
<p>The spoodle again, now as a mapping failure. A model reading <em>two-ounce spoodle</em> records two ounces, with no reason to doubt it, and is silently off on every record that follows. Multiply that across a corpus written in scoops, ladles, portions, and pans. The avocado data was fine and the modeling decision was hard. Here the data is clean, complete, and wrong.</p>
<div class="rule-box">
<span class="tag">General form</span>
A unit in the source that names an instrument rather than a quantity is a placeholder, not a value. It cannot be resolved from the document because the conversion was never written down. Someone has to measure it.
</div>
<p>The countermeasure in both cases is a rule, not a better model, and rules of this kind are written by people who have already been wrong once. I know each of these because I got it wrong first, not because it can be derived from the corpus.</p>
</section>
<section>
<h2>What generalizes</h2>
<p>Stated without the restaurant nouns.</p>
<p><strong>Mapping is smaller than it looks.</strong> It was a fifth of the project at most, and it is the line that collapses hardest. Budget the project around validation, not around the mapping.</p>
<p><strong>The dangerous failure is not ambiguity.</strong> It is clean text, high confidence, wrong number. Design for the case where the model is right about the text and the text is wrong about the world.</p>
<p><strong>Ground truth has to be manufactured.</strong> Any "X% accurate" needs "against what." My reference values were built by running the process until it stabilized, and the number of runs is a property of the process's variance, not something in a document. A system can tell you which records probably need a trial. It cannot run the trial or tell you when you have run enough.</p>
<p><strong>Errors compound down the tree, and the tree crosses boundaries.</strong> A bad value at step two of nine poisons everything above it. The last steps of the chain happened in a building the first steps never saw. You can model the chain once you have the numbers. Getting the numbers means being there.</p>
<p><strong>Internal consistency is not correctness.</strong> The system reported variance for months on a cost basis that was structurally incomplete, because commissary items moved to stores at breakeven. Nothing flagged it because nothing was inconsistent. "Live" and "true" are different dates, and the system cannot tell you which one you are on.</p>
<p><strong>The capture format serves the capturer.</strong> We counted in whatever unit a tired person could judge at 6am, with the conversions hidden behind it. Those unit choices were the accuracy mechanism, not a schema. If a human is at the point of capture, the interface at that point is where accuracy is won or lost.</p>
<p><strong>The job moves from input to audit.</strong> Validation was 45% of the project. With a model doing the input it becomes roughly 60% and keeps climbing, because everything that gets cheaper sits on the other side of the ledger. That is an improvement: audit is higher-leverage than typing. An audit still needs an auditor, and the residue is front-loaded and per-customer.</p>
<p><strong>The model of the operation outlives the tool.</strong> The company later migrated to a different system, and the migration was not painful, because the expensive part had already been paid: validated values, a correct tree, working maps. The software was replaceable. The model of the operation was not.</p>
</section>
<section>
<h2>What does not generalize</h2>
<p>Discount the claims above according to which of these your domain shares.</p>
<ul>
<li><strong>Physical transformation with loss.</strong> Restaurants turn goods into other goods through lossy processes, and the losses are why validation was 500 hours. If your entities do not change on the way through, most of that line disappears and the composition changes completely.</li>
<li><strong>Human-authored source material written for execution.</strong> The recipe book was written for cooks over fifteen years. If your source data is machine-structured, the execution-unit trap may not apply. If it is human notes, forms, or free text written for someone doing a job, it applies harder.</li>
<li><strong>Multi-site with an organizational boundary in the chain.</strong> The commissary-to-store handoff is where the tree crossed buildings. Single-site, or single-org, and the compounding-across-boundaries claim is moot.</li>
<li><strong>Human capture on a cadence.</strong> Weekly counts by tired people are where unit ergonomics mattered. If your capture is instrumented, that claim is moot. The "against what ground truth" question is not.</li>
</ul>
<p>The consolidation trap and the clean-text-wrong-number failure shape are the two I would expect to survive any of those discounts. They are properties of mapping onto a schema, not of restaurants.</p>
</section>
<section>
<h2>Where this leaves it</h2>
<p>A model collapses the work. It does not eliminate it. That is obvious once stated, and it is worth stating, because the two words get used as if they meant the same thing.</p>
<p>What survives has a shape. It is three kinds of work.</p>
<ul>
<li><strong>Running the validations and experiments a decision needs.</strong> Three trials, five, ten, until the number holds. The model can tell you where to look. It cannot stand in the kitchen.</li>
<li><strong>Making the decision.</strong> Two avocado items or one. Breakeven or allocated. Merge or ask. These are calls, and someone has to own them.</li>
<li><strong>Auditing the result.</strong> Matching against history to find the anomaly, then the conversation that fixes it.</li>
</ul>
<p>All three can be helped. The "now" column is conservative for each of them: targeting in validation, record-keeping, and anomaly detection in audit will compress further than I have given them. None of the three goes to zero.</p>
</section>
<aside class="endnote">
<h3>Note on method</h3>
<p>The line items and hours are the same as in the full report. I no longer have access to the records. Hours, spans, and object counts are recollections of a project that ran from 2017 to 2018, reconstructed in 2026, and should be read as order-of-magnitude. The "now" column is an estimate from experience and has not been measured against a real automated onboarding. Absolute cost figures, vendor pricing, recipes, yields, and inventory sheets are omitted. They are not mine to publish.</p>
</aside>
</article>