You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current "upscale" path is pure interpolation: libplacebo=…:upscaler=ewa_lanczos plus cas=0.5 (fetchforge/server.py:590-604). Measured on a real 720p→1080p job
(FFX-2 walkthrough, 38 min), cas=0.5 adds only +14–16% high-frequency energy over
the bare scaler — visually near-invisible, which is what prompted this.
libplacebo in this ffmpeg exposes custom_shader_path (mpv .hook format), so an
actual learned upscaler can be dropped in for free. Measured with FSRCNNX
(igv/FSRCNN-TensorFlow, LGPL-3.0) on three frames of that same file, identical NVENC
settings (p4/uhq/cq24/maxrate 3M):
chain
HF gain vs bare scaler
30 s clip
full 38-min file
ewa_lanczos only
baseline
—
—
+ cas=0.5 (current)
+14 to +16%
3.60× realtime
11 min
FSRCNNX-8
+40 to +48%
3.68× realtime
10 min
FSRCNNX-16
+38 to +49%
3.42× realtime
11 min
FSRCNNX-8 + cas=0.5
+55 to +72%
—
—
So ~3× the sharpening benefit at no measurable cost — the pipeline is bound by
decode/encode and the CUDA→RAM→Vulkan→RAM→NVENC round trip, not shader math. Use the _8-0-4-1 variant: statistically identical to _16-0-4-1, 71 KB vs 247 KB,
marginally faster.
Acceptance criteria
fetchforge/shaders/FSRCNNX_x2_8-0-4-1.glsl is committed, and pyproject.toml [tool.setuptools.package-data] includes it so pip install fetchforge ships it.
_decode_filter_args(...) accepts a new neural: bool = False keyword and, when neural=TrueandHAS_LIBPLACEBOandtarget_height > 0, appends :custom_shader_path=<abs path under PKG_DIR> to the existing libplacebo=… filter.
neural=True with HAS_LIBPLACEBO=False (or target_height=0) changes nothing —
falls back to the current chain, no crash, no bogus arg on scale_cuda/CPU branches.
build_video_ffmpeg_args(...) forwards neural through to _decode_filter_args.
/download and /convert-local both accept a neural: str = Form("false") field,
parsed exactly like the existing sharpen field (_neural = neural == "true"), and
pass it down.
Card 04 gets a Neural upscale toggle next to the existing Sharpen toggle; both startDownload/local-convert opts objects send form.append('neural', opts.neural).
The toggle is disabled/greyed with a hint when the selected target resolution is
less than 1.3× the source height (see danger zone Move import-time side effects (mkdir, logging) into run_server() #2) — the shader silently
no-ops there, and a silent no-op is worse than a disabled control.
Unit tests in tests/test_server.py (alongside the existing test_upscale_libplacebo
family) assert: custom_shader_path present when neural=True + libplacebo +
upscale; absent when neural=False; absent on the scale_cuda and CPU branches;
and that the referenced shader file actually exists on disk.
fetchforge/index.html:2432-2433 — where targetRes/sharpen are read from the DOM
pyproject.toml:32-33 — package-data
tests/test_server.py:313-345 — the libplacebo scaling test block
Shader source (download, do not hand-transcribe): https://github.com/igv/FSRCNN-TensorFlow/releases/download/1.1/FSRCNNX_x2_8-0-4-1.glsl
(sha: 71296 bytes, 426 lines). Keep the LGPL-3.0 header comment intact and add a line
crediting igv/FSRCNN-TensorFlow to the README or a fetchforge/shaders/LICENSE note.
How to verify
.venv/bin/python -m unittest discover -s tests -v — full suite green
python3 -m py_compile fetchforge/server.py
Argv shape, no GPU needed: .venv/bin/python -c "from pathlib import Path; from fetchforge import server; server.HAS_LIBPLACEBO=True; server.HAS_SCALE_CUDA=False; print(server._decode_filter_args(Path('in.mkv'),'yuv420p',None,target_height=1080,sharpen=True,neural=True))"
→ must contain custom_shader_path=and the existing upscaler=ewa_lanczos and cas=0.5
Preserve:neural=False must produce byte-identical argv to today for every branch —
the existing test_passthrough_* / test_downscale_* / test_upscale_* tests must pass
untouched.
Preserve: the sharpen/cas path stays. neural and sharpen are independent
toggles and must compose (FSRCNNX + cas=0.5 is a valid, measured-good combination).
Match: the existing sharpen plumbing exactly — same param position, same Form("false") + == "true" parse idiom, same toggle markup/CSS classes.
Resolve the shader path from PKG_DIR, never STATE_DIR/cwd — it's read-only
installed package data.
⚠ Danger zones (review focus)
The shader fails silently, and that is the whole risk here.FSRCNNX_x2_8-0-4-1.glsl
is //!HOOK LUMA, so it only fires when libplacebo's input has a luma plane. Feed it
RGB and it is a no-op with no warning and no error — I hit exactly this while
benchmarking (RGB PNG input produced byte-identical output to the plain scaler; the
real yuv420p video path worked). A unit test asserting the argv string is therefore necessary but not sufficient: it proves the flag is passed, not that it did anything.
⚠ Review: do not accept "tests pass" as evidence the feature works. Real verification
needs a GPU run — see the follow-up note below.
Scale-factor guard inside the shader: //!WHEN OUTPUT.w LUMA.w / 1.300 > OUTPUT.h LUMA.h / 1.300 > * — it only activates
above 1.3× in both axes. 720p→1080p (1.5×) fires; 1080p→1440p (1.33×) barely fires;
720p→900p (1.25×) silently does not. This is why the UI must gate the toggle on the
source/target ratio rather than letting the user enable a no-op.
Chroma is not shader-upscaled — FSRCNNX is luma-only, chroma still rides ewa_lanczos. That's standard and correct; don't "fix" it by adding a chroma shader.
Build integrity: a missing package-data entry means the shader is absent from a
real pip install while working fine from a source clone. The pip install -e .
check above does not catch that — verify with a built wheel or pip install .
into a scratch venv.
Out of scope
Any other shader family (Anime4K, ravu, FSRCNNX-16) or a shader-picker UI. Ship the
one measured-best default; more options can follow if wanted.
Changing the default CAS_STRENGTH (0.5). Separate question — the strength ladder
showed 0.7–0.8 is the sweet spot and cas=1.0 blows up into chroma speckle, but
that's its own issue.
Making neural the default. Ship it opt-in.
Downscale-time use of the shader (it's an upscaler; the //!WHEN guard blocks it anyway).
Notes
Executor tier: cheap. The plumbing is a mechanical clone of the existing sharpen
path; all the non-obvious reasoning is captured above.
Context
The current "upscale" path is pure interpolation:
libplacebo=…:upscaler=ewa_lanczospluscas=0.5(fetchforge/server.py:590-604). Measured on a real 720p→1080p job(FFX-2 walkthrough, 38 min),
cas=0.5adds only +14–16% high-frequency energy overthe bare scaler — visually near-invisible, which is what prompted this.
libplaceboin this ffmpeg exposescustom_shader_path(mpv.hookformat), so anactual learned upscaler can be dropped in for free. Measured with FSRCNNX
(igv/FSRCNN-TensorFlow, LGPL-3.0) on three frames of that same file, identical NVENC
settings (
p4/uhq/cq24/maxrate 3M):ewa_lanczosonly+ cas=0.5(current)+ cas=0.5So ~3× the sharpening benefit at no measurable cost — the pipeline is bound by
decode/encode and the CUDA→RAM→Vulkan→RAM→NVENC round trip, not shader math.
Use the
_8-0-4-1variant: statistically identical to_16-0-4-1, 71 KB vs 247 KB,marginally faster.
Acceptance criteria
fetchforge/shaders/FSRCNNX_x2_8-0-4-1.glslis committed, andpyproject.toml[tool.setuptools.package-data]includes it sopip install fetchforgeships it._decode_filter_args(...)accepts a newneural: bool = Falsekeyword and, whenneural=TrueandHAS_LIBPLACEBOandtarget_height > 0, appends:custom_shader_path=<abs path under PKG_DIR>to the existinglibplacebo=…filter.neural=TruewithHAS_LIBPLACEBO=False(ortarget_height=0) changes nothing —falls back to the current chain, no crash, no bogus arg on
scale_cuda/CPU branches.build_video_ffmpeg_args(...)forwardsneuralthrough to_decode_filter_args./downloadand/convert-localboth accept aneural: str = Form("false")field,parsed exactly like the existing
sharpenfield (_neural = neural == "true"), andpass it down.
startDownload/local-convert opts objects sendform.append('neural', opts.neural).less than 1.3× the source height (see danger zone Move import-time side effects (mkdir, logging) into run_server() #2) — the shader silently
no-ops there, and a silent no-op is worse than a disabled control.
tests/test_server.py(alongside the existingtest_upscale_libplacebofamily) assert:
custom_shader_pathpresent whenneural=True+ libplacebo +upscale; absent when
neural=False; absent on thescale_cudaand CPU branches;and that the referenced shader file actually exists on disk.
unchanged when
neural=True.Files / where this lives
fetchforge/server.py:577-642—_decode_filter_args, the libplacebo branch at 593-604fetchforge/server.py:900-920—build_video_ffmpeg_argsfetchforge/server.py:1514-1515,2345-2346— thetarget_res/sharpenform fieldson
/downloadand/convert-local; mirror them forneuralfetchforge/server.py:1585-1586—_sharpen = sharpen == "true"parse sitefetchforge/server.py:1959-1969,2223-2233— the four call sites that threadtarget_res=/sharpen=intocalc_encode_paramsandbuild_video_ffmpeg_argsfetchforge/index.html:1290-1297— the Sharpen toggle markup to mirrorfetchforge/index.html:2245-2246,2264-2265,2347-2348—form.appendsitesfetchforge/index.html:2432-2433— wheretargetRes/sharpenare read from the DOMpyproject.toml:32-33—package-datatests/test_server.py:313-345— the libplacebo scaling test blockShader source (download, do not hand-transcribe):
https://github.com/igv/FSRCNN-TensorFlow/releases/download/1.1/FSRCNNX_x2_8-0-4-1.glsl(sha: 71296 bytes, 426 lines). Keep the LGPL-3.0 header comment intact and add a line
crediting igv/FSRCNN-TensorFlow to the README or a
fetchforge/shaders/LICENSEnote.How to verify
.venv/bin/python -m unittest discover -s tests -v— full suite greenpython3 -m py_compile fetchforge/server.py.venv/bin/python -c "from pathlib import Path; from fetchforge import server; server.HAS_LIBPLACEBO=True; server.HAS_SCALE_CUDA=False; print(server._decode_filter_args(Path('in.mkv'),'yuv420p',None,target_height=1080,sharpen=True,neural=True))"→ must contain
custom_shader_path=and the existingupscaler=ewa_lanczosandcas=0.5pip install -e . && .venv/bin/python -c "from fetchforge import server; import pathlib; p=server.PKG_DIR/'shaders'/'FSRCNNX_x2_8-0-4-1.glsl'; print(p, p.exists(), p.stat().st_size)"Constraints
neural=Falsemust produce byte-identical argv to today for every branch —the existing
test_passthrough_*/test_downscale_*/test_upscale_*tests must passuntouched.
sharpen/caspath stays.neuralandsharpenare independenttoggles and must compose (
FSRCNNX + cas=0.5is a valid, measured-good combination).calc_encode_paramsbitrate/CQ math, the quality floor from Quality floor for sharpen/upscale: never below CQ24 / 3M / p4 / hq / yuv420p #29,or the
scale_cuda/ CPU fallback filter strings.sharpenplumbing exactly — same param position, sameForm("false")+== "true"parse idiom, same toggle markup/CSS classes.PKG_DIR, neverSTATE_DIR/cwd — it's read-onlyinstalled package data.
⚠ Danger zones (review focus)
FSRCNNX_x2_8-0-4-1.glslis
//!HOOK LUMA, so it only fires when libplacebo's input has a luma plane. Feed itRGB and it is a no-op with no warning and no error — I hit exactly this while
benchmarking (RGB PNG input produced byte-identical output to the plain scaler; the
real yuv420p video path worked). A unit test asserting the argv string is therefore
necessary but not sufficient: it proves the flag is passed, not that it did anything.
⚠ Review: do not accept "tests pass" as evidence the feature works. Real verification
needs a GPU run — see the follow-up note below.
//!WHEN OUTPUT.w LUMA.w / 1.300 > OUTPUT.h LUMA.h / 1.300 > *— it only activatesabove 1.3× in both axes. 720p→1080p (1.5×) fires; 1080p→1440p (1.33×) barely fires;
720p→900p (1.25×) silently does not. This is why the UI must gate the toggle on the
source/target ratio rather than letting the user enable a no-op.
ewa_lanczos. That's standard and correct; don't "fix" it by adding a chroma shader.package-dataentry means the shader is absent from areal
pip installwhile working fine from a source clone. Thepip install -e .check above does not catch that — verify with a built wheel or
pip install .into a scratch venv.
Out of scope
one measured-best default; more options can follow if wanted.
CAS_STRENGTH(0.5). Separate question — the strength laddershowed 0.7–0.8 is the sweet spot and
cas=1.0blows up into chroma speckle, butthat's its own issue.
neuralthe default. Ship it opt-in.//!WHENguard blocks it anyway).Notes
sharpenpath; all the non-obvious reasoning is captured above.
companion issue should follow the pattern of Verify HDR/10-bit passthrough through libplacebo on real HDR footage #33 (HDR passthrough through libplacebo)
and Verify scale_cuda + sharpen path at runtime on Windows NVENC build #34 (scale_cuda + sharpen on the Windows build): encode a short real clip with
neural=trueand confirm the output differs from theneural=falserender(compare high-frequency energy, don't eyeball it).