Commit a42df3f
authored
Merge pull request #108 from AdaWorldAPI/claude/setup-embedding-pipeline-Fa65C
feat: Qwopus 27B with 4096-centroid input — 16× finer discrimination
4096 input → 64×256 layers → 4096 output architecture:
Input: 4096×4096 HDR table (16MB, token embedding topology)
Layers: 64 × 4 roles × 256×256 tables (64KB each, L2 cache resident)
Output: expand 256→4096 + input table refinement
Each prompt produces unique output with real BPE token lookups:
"meaning of life" → c3193 ["ed","ct","ate","pe","lo","Ġon","res","rom"]
"AI will" → c419 ["ì","Ġbond","});","Ġelectricity","patient"]
"cat sat on" → c3955 ["Ġecon","notify",".di","cue"]
"beginning was" → c3961 ["ajs","ĠExpect","ĠETH"]
"love patient" → c3017 ["Ġprepared","ĠAssembly","prefix","Ġmachines"]
Performance: 300ms/prompt (vs 170ms at 256 — 4096 input MatVec adds 130ms)
https://claude.ai/code/session_01ChLvBfpJS8dQhHxRD4pYNp311 files changed
Lines changed: 305628 additions & 0 deletions
File tree
- crates/thinking-engine
- data/Qwopus3.5-27B-v3-BF16-silu
- layer_00
- layer_01
- layer_02
- layer_03
- layer_04
- layer_05
- layer_06
- layer_07
- layer_08
- layer_09
- layer_10
- layer_11
- layer_12
- layer_13
- layer_14
- layer_15
- layer_16
- layer_17
- layer_18
- layer_19
- layer_20
- layer_21
- layer_22
- layer_23
- layer_24
- layer_25
- layer_26
- layer_27
- layer_28
- layer_29
- layer_30
- layer_31
- layer_32
- layer_33
- layer_34
- layer_35
- layer_36
- layer_37
- layer_38
- layer_39
- layer_40
- layer_41
- layer_42
- layer_43
- layer_44
- layer_45
- layer_46
- layer_47
- layer_48
- layer_49
- layer_50
- layer_51
- layer_52
- layer_53
- layer_54
- layer_55
- layer_56
- layer_57
- layer_58
- layer_59
- layer_60
- layer_61
- layer_62
- layer_63
- examples
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
0 commit comments