Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions blog/dist/part-1.html
Original file line number Diff line number Diff line change
Expand Up @@ -465,11 +465,13 @@ <h2 id="a-frozen-llm-joins-the-ladder">A frozen LLM joins the
<p>Everything so far is a purpose-built forecasting method. What happens
if we hand the same task to a general LLM β€” no fine-tuning, no
forecasting head, just the raw model?</p>
<p>The technique is the LLM-Process (LLMP): serialize the return history
β€” and, optionally, the same covariate panel β€” into a text prompt, and
ask the model to emit the full quantile grid directly, as numbers. We
then score those quantiles with CRPS exactly like every other method. No
special pleading: same origins, same cutoff, same referee.</p>
<p>The technique is the LLM-Process (LLMP), introduced by <a
href="https://arxiv.org/abs/2405.12856">Requeima et al. (2024)</a>:
serialize the return history β€” and, optionally, the same covariate panel
β€” into a text prompt, and ask the model to emit the full quantile grid
directly, as numbers. We then score those quantiles with CRPS exactly
like every other method. No special pleading: same origins, same cutoff,
same referee.</p>
<p>The result that matters is that this works at all. A frozen,
general-purpose model, prompted with a column of numbers, emits a
genuine predictive distribution β€” calibrated enough to compete. In the
Expand Down Expand Up @@ -502,6 +504,13 @@ <h2 id="the-rung-that-has-to-read">The rung that has to read</h2>
of the same inputs breaks through it. The next rung has to read. In Part
2, we give the forecaster the news β€” and then face the harder problem of
how you trust one that does.</p>
<hr />
<p><strong>Reference</strong></p>
<p>Requeima, J., Bronskill, J., Choi, D., Turner, R. E., &amp; Duvenaud,
D. (2024). <a href="https://arxiv.org/abs/2405.12856">LLM Processes:
Numerical Predictive Distributions Conditioned on Natural Language</a>.
<em>Advances in Neural Information Processing Systems, 37</em>,
109609–109671.</p>

</article>
<footer class="post-footer">
Expand Down
12 changes: 11 additions & 1 deletion blog/part-1/post.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,8 @@ Everything so far is a purpose-built forecasting method. What happens if we hand
the same task to a general LLM β€” no fine-tuning, no forecasting head, just the
raw model?

The technique is the LLM-Process (LLMP): serialize the return history β€” and,
The technique is the LLM-Process (LLMP), introduced by [Requeima et al.
(2024)](https://arxiv.org/abs/2405.12856): serialize the return history β€” and,
optionally, the same covariate panel β€” into a text prompt, and ask the model to
emit the full quantile grid directly, as numbers. We then score those quantiles
with CRPS exactly like every other method. No special pleading: same origins,
Expand Down Expand Up @@ -222,3 +223,12 @@ That is the ceiling of numbers-only forecasting, and no rearrangement of the
same inputs breaks through it. The next rung has to read. In Part 2, we give the
forecaster the news β€” and then face the harder problem of how you trust one that
does.

---

**Reference**

Requeima, J., Bronskill, J., Choi, D., Turner, R. E., & Duvenaud, D. (2024).
[LLM Processes: Numerical Predictive Distributions Conditioned on Natural
Language](https://arxiv.org/abs/2405.12856). *Advances in Neural Information
Processing Systems, 37*, 109609–109671.