Attention before downsampling - #4

Open

apapiu wants to merge 6 commits into

mainfrom

attention_before_downsampling

Commits on Feb 4, 2024

use pytorch instead of rearrange and add compile step
apapiu
committed
908c57c
reshape fix
apapiu
committed
4a698f6

Commits on Feb 5, 2024

change the reshape to correctly put head first
apapiu
committed
f8f636e
add pre norm
apapiu
committed
b249d8a

Commits on Feb 6, 2024

new architecture for attention at highest rez
apapiu
committed
d3721b3

Commits on Feb 9, 2024

comments for later
apapiu
committed
5e5e389