sync to tot - #4

Open

SiddhanthRaja-Nvidia wants to merge 1347 commits into

aukarande:pshard/profilingfrom

SiddhanthRaja-Nvidia:pshard/profiling

This pull request is big! We're only showing the most recent 250 commits

Commits on Jul 7, 2026

speculative : fix out-of-bounds read in ngram-map on prompt shrink (#23936)
o7si
authored
defa95c
ggml : make ggml_time_init idempotent (#24422)
aisk
authored
1a7c25b
sycl : rename the env vars from "disable" to "enable" (#25042)
arthw
authored
26145b3
sycl : use sycl func to fix AOT double type issue (#25081)
arthw
authored
3d4cbdf
sycl : enhance argsort to support all UT cases (#25125)
arthw
authored
9e5ef0d
[SYCL] fix unsupport ACC UT cases for noncontiguous (#25124)
arthw
authored
95e5254
sycl : set K_QUANTS_PER_ITERATION to 1 on DMMV path (#25063)
malsbat
authored
d209086
[SYCL] support OP cross_entropy_loss, cross_entropy_loss_back (#25236)
arthw
authored
55edb2d
[SYCL] support op col2im_1d (#25264)
arthw
authored
47e1de7
[SYCL] fix unsupported UT cases of CONT & CPY (#25231)
arthw
authored
108f186
llama: fix quantized kv-cache for dsv4 (#25202)
am17an
authored
024c46a
ggml-hip : add -fno-finite-math-only alongside -ffast-math (#25373)
asf0
authored
33ca0dc
common : add missing <fstream> include in common.h (#25220)
zhangrunda
authored
c1a411f
server: enforce prompt cache RAM limit (#25070)
tarruda
authored
6c487e2
server : add timings and progress to /responses API stream (#25348)
thom-dev-fr
authored
5eca4e3
server : fix draft model fit vs load inconsistency (#25056)
wadealexc
authored
f5525f7
CUDA: Fuse MMVQ post-scale for NVFP4 (#24481)

authored
3899b39
spec : fix naming, spacing (#25410)
ggerganov
authored
c198af4
Add Q2_0 quantization: type definition and CPU backend (#24448)
khosravipasha
authored
bec4772

Commits on Jul 8, 2026

Commits on Jul 9, 2026

hexagon: add VISION RoPE support (#25216)
aparmp-quic
authored
f2d1c2f
server : respect min-step when splitting prompt batches (#25420)
aldehir
authored
64c8b7d
cuda: align snake fusion matcher with the other backends (#25460)
ServeurpersoCom
authored
2021515
ggml-hip: enable -funsafe-math-optimizations (#24668)

RapidMark
and
Mark Caldwell
authored
ccb0c34
metal : add CONV_2D_DW (depthwise convolution) support (#21565)

Sou-ly
and
ggerganov
authored
92b187c
llama-bench : init params.offline (#25476)
angt
authored
259f2e2
Only index by compile times + always multiply/add (#25445)
ORippler
authored
683f0c7
Refactor: Consistently use smart pointers in `test-backend-ops` (#25440)
ORippler
authored
f84a519
meta: add hard emphasis on agents not writing descriptions/comments (#25480)
pwilkin
authored
c15c5c7
ggml : fix conv 2d dw (#25490)
ggerganov
authored
5c3a586
server : move chat-template thinking probe inside the init try/catch (#24093)
palios-taey
authored
82fce65
hexagon: tiling, tracing and optimizations for unary ops (#25474)

aparmp-quic
and
max-krasnyansky
authored
fb30ba9
cli: add --output option (#25484)
ngxson
authored
3de7dd4
ggml : process data in smaller chunks in CUDA ggml_top_k() and ggml_argsort() to reduce temporary buffers memory usage (#24776)

authored
0749449
opencl: cluster-parallel decode FA for Adreno (#25473)
wanghqc
authored
049326a

Commits on Jul 10, 2026

llama-batch: add unit test (#25471)
am17an
authored
961e4b2
ggml-et: Initial ET backend (#24179)

authored
082b326
llama : make all KQ masks f16 if FA is used, remove zero attention bias, remove raw_k repeats in DeepSeek V4 (#25370)

fairydreaming
and
sszymczy
authored
2ed3c1a
ui: export full message tree instead of active path only (#25501)
ServeurpersoCom
authored
d46786f
sync : ggml (#25517)
ggerganov
authored
8f114a9
ui: prevent tooltip from flickering open and closed on hover (#25503)
ServeurpersoCom
authored
c7af942
cli: fix crash on wrong server base url (#25497)
marcoStocchi
authored
1b9691b
server: improve tools, remove apply_diff (#25498)
ngxson
authored
c4ae9a8
ggml : use ggml_vqtbl1q_u8 for 32-bit compat (whisper/0)
ggerganov
committed
ac17f8a
sync : ggml
ggerganov
committed
0badc06
server: remove loading.html (#25500)
ngxson
authored
a935fbf
ui: use server modalities in non-router mode (#24874)
coder543
authored
9f623c6
feat: pre-select models in the webui using alias (#25492)

felix314159
and
example name
authored
07d9378
mtmd: deepseek-ocr v1 multi-tile (#24717)

ngxson
and
sfallah
authored
3e706dd
arg: prevent duplicate spec model downloads (#25527)
ngxson
authored
22b69b6
hexagon: improve ARGSORT performance for small tensors (#25512)
max-krasnyansky
authored
67776ea
llama : make tensor-split regex patterns static (#24710)
eduardopessin
authored
c749cb0
server: accept null sampling params (#25538)
ServeurpersoCom
authored
4f37f51

Commits on Jul 11, 2026

Commits on Jul 12, 2026

Commits on Jul 13, 2026

ui: Remove recommended MCP Servers + improve MCP Servers Settings UI/UX (#25535)
allozaur
authored
38fd5c9
sycl: add Q2_K to DMMV reorder path (#25064)
malsbat
authored
e474bba
sycl: add fused top-k MoE (#25217)

newjordan
and
claude
authored
efb3036
chat : fix reasoning leak with force-opened bare <think> templates (#24674)

newjordan
and
aldehir
authored
91c631b
gguf : add tensor shape accessor (#24405)
QuintinShaw
authored
ad8d821
tests: Harmonize header use (#25616)
ckastner
authored
f4253ef
readme : add link to maintainer PRs (#25621)
ggerganov
authored
4193ea6
spec: add Minimax2 eagle3 support

adrianisk
and
ggerganov
authored
259ae1d
vulkan: Use native e2m1 and e4m3 conversions for mxfp4/nvfp4 (#25338)
jeffbolznv
authored
e920c52
CUDA: refactor MMQ kernel configuration (#24127)
JohannesGaessler
authored
6eddde0
model: add Hy3 (hy_v3) support with MTP speculative decoding (#25395)

authored
2969d6d

Commits on Jul 14, 2026

Commits on Jul 15, 2026

Commits on Jul 16, 2026

Commits on Jul 17, 2026

Commits on Jul 18, 2026

Commits on Jul 20, 2026

Commits on Jul 21, 2026

Commits on Jul 22, 2026

Add support for Laguna XS.2 & M.1 (#25165)
joerowell
authored
1f66c3c
llama-arch: fix DeepSeek4 APE tensor op (#25945)
helanfxz
authored
67b9b0e
cuda: GET_ROWS quants (#25962)
ServeurpersoCom
authored
c5a4a0b
webgpu : add CONV_2D_DW (depthwise conv2d) kernel (#25847)
m1el
authored
7347430
ci : fix SYCL package shared library lookup (#25987)
KyleHagy
authored
b4d6c7d
ggml: enable PowerPC backend variants on AIX (#25983)
shalinib-ibm
authored
3ce7da2
Fix DeepSeek4 crafted template (#25414)
pwilkin
authored
f534da2
common: infer the speculative type from the draft repo sidecars (#25989)
ServeurpersoCom
authored
6d5a910
minor: fix reasoning preserve var for DS4 [no ci] (#25999)
pwilkin
authored
e8e6c7a
feat(ui): add symbolic math support to JS sandbox via nerdamer (#25948)

rankaiyx
and
ServeurpersoCom
authored
61328e6
mtmd: use RAII for setting and resetting non-causal attention (#25723)
nikwen
authored
e0833bf
hexagon: activation ops update (#25974)

tboinovski1
and
max-krasnyansky
authored
0278d83
CUDA: Improve NVFP4 W4A4 activation quantization (#25730)
ORippler
authored
1a064ab
ui: Add a "Default" option for the reasoning selector (#25846)
ServeurpersoCom
authored
cf51256
contrib: allow all AI-generated code in general (#26012)
ngxson
authored
4310aa4

Commits on Jul 23, 2026

Commits on Jul 24, 2026

hexagon: further improved pipeline of the core bits (L2, DMA, MM, FA) (#26049)
max-krasnyansky
authored
0a50d99
vendor: update subprocess.h (#26061)
ngxson
authored
0cea362
cohere2 moe template parser: enforce JSON schema for text responses if a response schema is provided (#26018)
boondocklabs
authored
8f5ab83
UI: Fix settings precedence, Factory < Admin (--ui-config-file) < Users (Settings panel) (#26002)
ServeurpersoCom
authored
54ce507
skill: create `add-new-model` and `code-review` (#26042)

ngxson
and
pwilkin
authored
77095ee
opencl: do not treat NULL-mask flash attention as causal (#25771)
kumaal
authored
56a8386
opencl: cache compiled cl_program binaries on disk (#26050)
wanghqc
authored
ed7adbf
HIP: remove rocWMMA FlashAttention (#26046)
JohannesGaessler
authored
fa72aec
llama: various bug fixes (#26051)
ngxson
authored
298219f
server: support "reasoning_effort": "none" in OAI API (#26045)
pnb
authored
27209a5
ui: fix MCP server display name conflicts in tools lists (#26011)
ServeurpersoCom
authored
95a923a
model: add GLM 5.2 Indexer support (#25407)

pcuenca
and
fairydreaming
authored
88bfee1
ui: remove render effects (#26083)
ServeurpersoCom
authored
96013c5
ui: reduce per-token render cost when streaming (#26053)

Zach Winter
and
zachwinter
authored
555881e

Commits on Jul 25, 2026

Commits on Jul 26, 2026

ggml-webgpu: Fix WASM compilation with OpenMP (#25943)
reeselevine
authored
7cdd557
ui: fix context gauge card regressions and land at the conversation end (#26099)
ServeurpersoCom
authored
ff067f7
opencl: fix fused RMS norm mul view offset (#26085)
happyyzy
authored
42fc243
model: Add MiniMax-M3 (MSA: MiniMax Sparse Attention) support (#24908)

authored
b1d4c65
mtmd: add GLM-5.2-Vision (#26126)

ehartford
and
Eric Hartford
authored
af28502
common: add `subproc.h` wrapper, disabled on android/ios (#26102)
ngxson
authored
d2a8182
ui: detect the conversation import format from file contents (#26121)
ServeurpersoCom
authored
55b7d6c
Keep Minimax's indexer tensors at F32 for speed and accuracy (#26144)
bartowski1182
authored
7657a6c
ui: fix system message edit box not expanding to fit content (#26006)
pieroevcc
authored
d4d057b
mtmd: fix android build (#26150)
ngxson
authored
0d47ea7
mtmd: Add Vision Support for Minimax-M3 (#25113)

authored
3d1c3a8

Commits on Jul 27, 2026

ui: Fix symbolic math tool JS sandbox prompt (#26131)
rankaiyx
authored
88b47a7
feat: GPU/CPU roofline profiler for per-op benchmarking

aukarande
authored and
SiddhanthRaja-Nvidia
committed
ac0cc47