Repository navigation
Expand file tree
/
Copy pathindex.html
More file actions
581 lines (543 loc) · 32 KB
/
Copy pathindex.html
File metadata and controls
581 lines (543 loc) · 32 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Junyi Hu</title>
<meta name="description" content="Junyi Hu — mathematics undergraduate at Fudan University and research assistant at New York University. How visual systems can understand interaction in the physical world: visual navigation and tracking, human motion, and robots that learn from human demonstrations. Applying for PhD study, Fall 2027.">
<meta property="og:title" content="Junyi Hu">
<meta property="og:description" content="How visual systems can understand interaction in the physical world: visual navigation and tracking, and human motion. Fudan University · New York University.">
<meta property="og:type" content="profile">
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><text y='.9em' font-size='90'>🤟</text></svg>">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@300..800&display=swap" rel="stylesheet">
<style>
:root{
--bg:#0B0D10; --surface:#14171C; --surface-2:#1B1F26; --line:#2A303A;
--text:#F2F4F7; --muted:#AEB7C4; --dim:#7F8A99;
--accent:#5B8CFF; --accent-2:#8E6BD6; --accent-3:#FF5252;
--radius:14px; --maxw:900px;
}
@media (prefers-color-scheme: light){
:root{ --bg:#FBFBFD; --surface:#FFFFFF; --surface-2:#F3F5F8; --line:#E1E5EC;
--text:#10131A; --muted:#4E5866; --dim:#6B7684; --accent:#2B5FE3; }
}
:root[data-theme="dark"]{ --bg:#0B0D10; --surface:#14171C; --surface-2:#1B1F26; --line:#2A303A;
--text:#F2F4F7; --muted:#AEB7C4; --dim:#7F8A99; --accent:#5B8CFF; }
:root[data-theme="light"]{ --bg:#FBFBFD; --surface:#FFFFFF; --surface-2:#F3F5F8; --line:#E1E5EC;
--text:#10131A; --muted:#4E5866; --dim:#6B7684; --accent:#2B5FE3; }
*{box-sizing:border-box}
html{scroll-behavior:smooth}
body{
margin:0; background:var(--bg); color:var(--text);
font-family:Inter,-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,sans-serif;
font-size:17px; line-height:1.65; -webkit-font-smoothing:antialiased;
}
.wrap{max-width:var(--maxw); margin:0 auto; padding:0 24px}
a{color:var(--accent)}
p{max-width:none;text-wrap:pretty}
h1,h2,h3{text-wrap:balance}
/* ---------- nav ---------- */
nav{position:sticky;top:0;z-index:50;background:color-mix(in srgb,var(--bg) 88%,transparent);
backdrop-filter:blur(12px);border-bottom:1px solid var(--line)}
nav .wrap{display:flex;align-items:center;justify-content:space-between;height:62px;gap:16px}
nav .brand{font-weight:650;font-size:15px;letter-spacing:-.01em}
nav .links{display:flex;gap:30px;font-size:15.5px;font-weight:580;letter-spacing:-.005em}
nav .links a{color:var(--text);text-decoration:none;padding:4px 0;border-bottom:2px solid transparent;transition:color .15s,border-color .15s}
nav .links a:hover{color:var(--accent);border-bottom-color:var(--accent)}
@media(max-width:640px){nav .links{gap:16px;font-size:14px}}
/* ---------- hero ---------- */
header{padding:72px 0 8px}
h1{font-size:clamp(34px,6vw,54px);line-height:1.1;letter-spacing:-.03em;
font-weight:700;margin:0 0 14px}
.tagline{font-size:clamp(17px,2.2vw,20px);line-height:1.5;color:var(--muted);
font-weight:400;margin:20px 0 22px;max-width:25.6em;letter-spacing:-.011em}
.tagline b{font-weight:650;color:var(--text)}
.affil{color:var(--muted);font-size:16px;margin:0 0 4px;max-width:none}
.affil b{color:var(--text);font-weight:600}
.rule{height:3px;border:0;border-radius:2px;margin:34px 0 0;max-width:180px;
background:linear-gradient(90deg,var(--accent),var(--accent-2),var(--accent-3));}
.btns{display:flex;gap:10px;flex-wrap:wrap;margin-top:28px}
.btn{display:inline-flex;align-items:center;gap:8px;background:var(--surface-2);
border:1px solid var(--line);color:var(--text);text-decoration:none;
padding:9px 18px;border-radius:999px;font-size:14.5px;font-weight:520;
transition:border-color .15s,transform .15s}
.btn:hover{border-color:var(--accent);transform:translateY(-1px)}
.btn svg{width:15px;height:15px;fill:currentColor;flex:none}
/* ---------- sections ---------- */
section{padding:56px 0;border-top:1px solid var(--line);margin-top:56px}
.kicker{color:var(--accent);font-size:12.5px;font-weight:640;letter-spacing:.09em;
text-transform:uppercase;margin:0 0 8px}
h2{font-size:clamp(27px,3.2vw,34px);letter-spacing:-.022em;font-weight:700;margin:0 0 18px;line-height:1.15}
h2::before{content:"";display:block;width:44px;height:3px;border-radius:2px;margin:0 0 16px;
background:linear-gradient(90deg,var(--accent),var(--accent-2))}
/* ---------- publications ---------- */
.pub{padding:30px 0;border-bottom:1px solid var(--line)}
.dirlabel + .pub, .dirlabel + * + .pub{padding-top:22px}
.pub:last-of-type{border-bottom:0;padding-bottom:6px}
.pub-top{display:grid;grid-template-columns:clamp(240px,34%,290px) 1fr;gap:26px;align-items:start;margin-bottom:15px}
@media(max-width:760px){.pub-top{grid-template-columns:1fr;gap:16px}}
.pub-head .meta{margin-bottom:0}
.pub .teaser{display:flex;align-items:center;justify-content:center;aspect-ratio:16/10;margin:0;
border-radius:10px;overflow:hidden;border:1px solid var(--line);background:#fff;padding:6px;
transition:border-color .15s,transform .15s}
.pub .teaser:hover{border-color:var(--accent);transform:translateY(-1px)}
.pub .teaser img{width:100%;height:100%;object-fit:contain;display:block}
.pub h3{margin:0 0 8px;font-size:19px;font-weight:660;letter-spacing:-.015em;line-height:1.35}
.pub h3 a{color:var(--text);text-decoration:none}
.pub h3 a:hover{color:var(--accent)}
.authors{margin:0 0 10px;font-size:14px;text-wrap:balance;color:var(--muted);max-width:none}
.authors .me{color:var(--text);font-weight:650}
.pub p.blurb{margin:0 0 14px;font-size:15.6px;line-height:1.72;color:var(--muted);max-width:none}
.meta{display:flex;gap:8px;flex-wrap:wrap;margin:0 0 12px}
.tag{display:inline-block;font-size:12.5px;font-weight:620;letter-spacing:.005em;
padding:3.5px 11px;border-radius:999px;
background:var(--surface-2);border:1px solid var(--line);color:var(--muted)}
.tag.venue{border-color:color-mix(in srgb,var(--accent-2) 55%,var(--line));
color:color-mix(in srgb,var(--accent-2) 78%,var(--text))}
.tag.preprint{border-color:color-mix(in srgb,var(--dim) 60%,var(--line));color:var(--dim)}
.tag.role{border-color:color-mix(in srgb,var(--accent) 55%,var(--line));
color:color-mix(in srgb,var(--accent) 82%,var(--text))}
.links-row{display:flex;gap:16px;flex-wrap:wrap;font-size:14px}
.links-row a{text-decoration:none;font-weight:540}
.links-row a:hover{text-decoration:underline}
.links-row .sep{color:var(--dim)}
/* ---------- long-form statement ---------- */
.prose{max-width:none}
.prose h3{font-size:16px;font-weight:650;letter-spacing:-.008em;margin:30px 0 10px;color:var(--text)}
.prose h3:first-of-type{margin-top:24px}
.prose p{margin:0 0 15px;color:var(--muted);font-size:16px;max-width:none}
.prose p b{color:var(--text);font-weight:620}
.prose a{text-decoration:none;font-weight:550}
.prose a:hover{text-decoration:underline}
.prose .claim{color:var(--text)}
/* ---------- hero portrait ---------- */
.hero{display:grid;grid-template-columns:1fr 200px;gap:40px;align-items:start}
@media(max-width:700px){.hero{grid-template-columns:1fr;gap:24px}}
.portrait{width:200px;max-width:52vw;height:auto;display:block;border-radius:12px;
border:1px solid var(--line);box-shadow:0 1px 3px rgba(0,0,0,.18)}
/* ---------- news ---------- */
.news{padding:40px 0 8px}
.news ul{list-style:none;margin:6px 0 0;padding:0;display:grid;gap:7px;max-width:none}
.news li{display:grid;grid-template-columns:104px 1fr;gap:14px;font-size:15px;color:var(--muted);line-height:1.55}
.news .when{color:var(--dim);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;padding-top:1px}
.news a{text-decoration:none;font-weight:560}
.news a:hover{text-decoration:underline}
@media(max-width:560px){.news li{grid-template-columns:1fr;gap:0}}
#news + section{margin-top:28px}
/* ---------- research statement ---------- */
#research p.statement{font-size:16.6px;line-height:1.72;color:var(--muted);max-width:none}
/* ---------- direction labels ---------- */
.dirlabel{margin:64px 0 4px;padding:2px 0 2px 16px;border-left:3px solid var(--accent)}
.dirlabel .idx{display:block;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;
font-size:12.5px;color:var(--accent);letter-spacing:.12em;margin:0 0 6px}
.dirlabel h3{margin:0;font-size:clamp(20px,2.3vw,24px);font-weight:680;letter-spacing:-.015em;line-height:1.25;color:var(--text)}
.dirlabel .bar{display:none}
/* ---------- contact ---------- */
.contact p{color:var(--muted)}
.contact .note{color:var(--dim)}
.mono{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:.94em}
footer{border-top:1px solid var(--line);padding:28px 0 44px;color:var(--dim);font-size:13.5px}
footer p{margin:0;max-width:none}
</style>
</head>
<body>
<nav>
<div class="wrap">
<span class="brand">Junyi Hu</span>
<div class="links">
<a href="#research">Research</a>
<a href="#future">Future Work</a>
<a href="#about">About</a>
<a href="#contact">Contact</a>
</div>
</div>
</nav>
<header>
<div class="wrap">
<div class="hero">
<div>
<h1>Junyi Hu</h1>
<p class="affil">Undergraduate, School of Mathematical Sciences, <b>Fudan University</b></p>
<p class="affil">Research Assistant, <b>New York University</b></p>
<p class="affil">Applying for PhD study, Fall 2027</p>
<p class="tagline">
I study how visual systems can understand <b>interaction in the physical world</b>: how people,
agents, objects and environments influence one another over time.
</p>
<div class="btns">
<a class="btn" href="mailto:jh10472@nyu.edu">
<svg viewBox="0 0 24 24"><path d="M20 4H4a2 2 0 0 0-2 2v12a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2V6a2 2 0 0 0-2-2zm0 4-8 5-8-5V6l8 5 8-5v2z"/></svg>
Email
</a>
<a class="btn" href="https://github.com/junyi2005">
<svg viewBox="0 0 24 24"><path d="M12 .5C5.7.5.5 5.7.5 12c0 5.1 3.3 9.4 7.8 10.9.6.1.8-.2.8-.6v-2c-3.2.7-3.9-1.5-3.9-1.5-.5-1.3-1.3-1.7-1.3-1.7-1-.7.1-.7.1-.7 1.1.1 1.7 1.2 1.7 1.2 1 1.8 2.7 1.3 3.4 1 .1-.7.4-1.3.7-1.6-2.6-.3-5.3-1.3-5.3-5.8 0-1.3.5-2.3 1.2-3.1-.1-.3-.5-1.5.1-3.1 0 0 1-.3 3.2 1.2a11 11 0 0 1 5.8 0C17 4.7 18 5 18 5c.6 1.6.2 2.8.1 3.1.8.8 1.2 1.8 1.2 3.1 0 4.5-2.7 5.5-5.3 5.8.4.4.8 1.1.8 2.2v3.3c0 .4.2.7.8.6 4.6-1.5 7.8-5.8 7.8-10.9C23.5 5.7 18.3.5 12 .5z"/></svg>
GitHub
</a>
<a class="btn" href="assets/Junyi_Hu_CV.pdf">
<svg viewBox="0 0 24 24"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8l-6-6zm-1 1.5L18.5 9H13V3.5zM8 13h8v1.6H8V13zm0 3.4h5.5V18H8v-1.6z"/></svg>
CV
</a>
<a class="btn" href="#publications">
<svg viewBox="0 0 24 24"><path d="M6 2h9l5 5v15H6a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2zm8 1.5V8h4.5L14 3.5zM8 12h8v1.6H8V12zm0 3.4h8V17H8v-1.6z"/></svg>
Publications
</a>
</div>
</div>
<img class="portrait" src="assets/junyi.jpg" alt="Junyi Hu">
</div>
<hr class="rule">
</div>
</header>
<section id="news" class="news">
<div class="wrap">
<h2>News</h2>
<ul>
<li><span class="when">Jun 2026</span><span><a href="#vtamo">VTaMo</a> and <a href="#signnet">SignNet-1M</a> accepted to ECCV 2026.</span></li>
<li><span class="when">Jun 2026</span><span><a href="#safety">Adaptive Safety Margins</a> accepted to IROS 2026.</span></li>
</ul>
</div>
</section>
<section id="research">
<div class="wrap">
<h2>Research</h2>
<p class="statement">
I approach this question from two connected directions. In visual navigation and tracking, I study how a
policy can adapt its behavior to the spatial context instead of following a fixed notion of safety, and
how the dynamics of observer, target and environment can be modeled with explicit causal structure rather
than as one entangled system. In human motion, I study how continuous movement aligns with language, and
what a representation can learn from the temporal structure of movement without labels. The two
directions meet when a robot learns from a person: a human demonstration can be read as a statement of
how the world should change, which the robot then carries out with its own body. In the long term, I want
to make interaction an organizing principle for visual understanding, and to build models around that
idea.
</p>
<div class="dirlabel" id="publications"><span class="idx">DIRECTION I</span><h3>Visual navigation and tracking</h3><span class="bar"></span></div>
<!-- ── Adaptive safety margins ───────────────────────────── -->
<article class="pub" id="safety">
<div class="pub-top">
<a class="teaser" href="https://junyi2005.github.io/safety-critic/">
<img loading="lazy"
src="https://raw.githubusercontent.com/junyi2005/safety-critic/main/docs/assets/img/architecture.png"
alt="A diffusion policy proposes candidate trajectories from RGB-D; a learned safety critic scores and selects one.">
</a>
<div class="pub-head">
<h3><a href="https://junyi2005.github.io/safety-critic/">Learning Adaptive Safety Margins for Visual Navigation</a></h3>
<p class="authors"><span class="me">Junyi Hu</span>, Shuaihang Yuan, Geeta Chandra Raju Bethala, Anthony Tzes, Yi Fang</p>
<div class="meta">
<span class="tag venue">IROS 2026</span>
<span class="tag role">First Author</span>
<span class="tag">Agent ↔ Environment</span>
</div>
</div>
</div>
<p class="blurb">
How much room a robot should leave depends on where it is, and a margin fixed in advance is too
cautious in one place and too tight in another. We let the robot read it from what it sees: a diffusion
policy proposes paths from RGB-D, and a critic trained with map geometry picks the one whose clearance
fits the scene, while the deployed system needs no map. On HM3D it reaches <b>0.783</b> success against
0.710 for NavDP, and it runs on a Unitree G1.
</p>
<div class="links-row">
<a href="https://junyi2005.github.io/safety-critic/">Project page</a><span class="sep">·</span>
<a href="https://arxiv.org/abs/2607.18200">arXiv</a><span class="sep">·</span>
<a href="https://github.com/junyi2005/safety-critic">Code</a>
</div>
</article>
<!-- ── CST-WM ────────────────────────────────────────────── -->
<article class="pub" id="cstwm">
<div class="pub-top">
<a class="teaser" href="https://junyi2005.github.io/cst-wm/">
<img loading="lazy"
src="https://junyi2005.github.io/cst-wm/static/teaser.png"
alt="CST-WM separates target evidence, robot and observation, and removes the direct edge from action to target evidence.">
</a>
<div class="pub-head">
<h3><a href="https://junyi2005.github.io/cst-wm/">CST-WM: A Causally Structured World Model for Embodied Visual Tracking</a></h3>
<p class="authors"><span class="me">Junyi Hu</span>, Shuaihang Yuan, Jiazhao Liang, Yi Fang</p>
<div class="meta">
<span class="tag preprint">Under Review</span>
<span class="tag role">First Author</span>
<span class="tag">Causal Structure</span>
</div>
</div>
</div>
<p class="blurb">
A robot that follows a person sees the world move whenever it moves, and a world model trained on that
footage takes a shortcut: it comes to believe its own steering is what moves the person. CST-WM gives
the model the causal structure of the scene, with separate branches for target, robot and observation
and no direct path from the action to the target, so an action can change what the robot sees of the
target only by moving it. On a Unitree Go2 it follows a person through occlusion, distractors and fast
motion in <b>20 of 30</b> trials, against 14 for TrackVLA.
</p>
<div class="links-row">
<a href="https://junyi2005.github.io/cst-wm/">Project page</a><span class="sep">·</span>
<a href="https://arxiv.org/abs/2609.06302">arXiv</a><span class="sep">·</span>
<a href="https://junyi2005.github.io/cst-wm/cst-wm.pdf">PDF</a><span class="sep">·</span>
<a href="https://junyi2005.github.io/cst-wm/#demo">Video</a>
</div>
</article>
<div class="dirlabel"><span class="idx">DIRECTION II</span><h3>Human motion</h3><span class="bar"></span></div>
<!-- ── VTaMo ─────────────────────────────────────────────── -->
<article class="pub" id="vtamo">
<div class="pub-top">
<a class="teaser" href="https://github.com/junyi2005/vtamo">
<img loading="lazy"
src="https://raw.githubusercontent.com/junyi2005/vtamo/main/assets/pipeline.png"
alt="The VTaMo pipeline: visual encoder, optimal-transport local alignment, orthogonal global alignment, and the language decoder.">
</a>
<div class="pub-head">
<h3><a href="https://github.com/junyi2005/vtamo">VTaMo: Video–Text Alignment Model for Sign Language Translation</a></h3>
<p class="authors"><span class="me">Junyi Hu</span>, Zhewen He, Haomian Huang, Aoxiang Yang, Yi Fang</p>
<div class="meta">
<span class="tag venue">ECCV 2026</span>
<span class="tag role">First Author</span>
<span class="tag">Motion ↔ Language</span>
</div>
</div>
</div>
<p class="blurb">
VTaMo achieves <b>unprecedented alignment between video and text</b>. Rather than leaving the
correspondence between a continuous stream and a sentence to attention, it matches the two explicitly
with optimal transport, word by word; because each word is assigned a stretch of frames, the alignment
also cuts the stream into the pieces that carry each word, with no frame-level labels. Experiments are
on four sign-language translation benchmarks.
</p>
<div class="links-row">
<a href="https://arxiv.org/abs/2607.09126">arXiv</a><span class="sep">·</span>
<a href="https://github.com/junyi2005/vtamo">Code</a>
</div>
</article>
<!-- ── SignDino ──────────────────────────────────────────── -->
<article class="pub" id="signdino">
<div class="pub-top">
<a class="teaser" href="https://junyi2005.github.io/signDEG/">
<img loading="lazy"
src="https://junyi2005.github.io/signDEG/static/insight_geo.png"
alt="Spatial self-distillation on one frame, temporal self-distillation on one tracked body part, and SignDEG, which adds the hand geometry of every frame as a pre-training target.">
</a>
<div class="pub-head">
<h3><a href="https://junyi2005.github.io/signDEG/">DINO Eats Geometry: Self-Supervised Sign Representations that Absorb Articulated Hand Geometry</a></h3>
<p class="authors"><span class="me">Junyi Hu</span>, Zhewen He, Haomian Huang, Zhenhua Li, Zhifei Li, Shuaihang Yuan, Yi Fang</p>
<div class="meta">
<span class="tag preprint">Under Review</span>
<span class="tag role">First Author</span>
<span class="tag">Geometry-Aware Self-Supervision</span>
</div>
</div>
</div>
<p class="blurb">
What a hand expresses lies largely in how it is articulated: which fingers are selected, how they bend,
where the palm faces. An encoder trained on appearance alone sees it only indirectly, because
appearance also follows the viewpoint and the person. SignDEG uses hand geometry <b>only as a
pre-training target</b>: frozen DINOv3 features of tracked hands and faces feed per-stream temporal
Transformers trained by temporal self-distillation, while one block learns to predict the hand’s 3D
articulation at every frame and the last block learns the relations between frames of different videos
that this geometry induces. The reconstruction model and geometry heads are then discarded, so the
deployed encoder runs on video alone. Experiments are on sign-language video.
</p>
<div class="links-row">
<a href="https://junyi2005.github.io/signDEG/">Project page</a><span class="sep">·</span>
<a href="https://arxiv.org/abs/2609.06296">arXiv</a><span class="sep">·</span>
<a href="https://github.com/junyi2005/signDEG">Code</a>
</div>
</article>
<!-- ── Real-time translation ─────────────────────────────── -->
<article class="pub" id="realtime">
<div class="pub-top">
<a class="teaser" href="assets/realtime-slt.pdf">
<img loading="lazy"
src="assets/realtime-slt-pipeline.png"
alt="Three-stage pipeline: confidence-aware masked pose encoding, discriminative VQ tokenization, and a strictly causal streaming pipeline of retrieval, past-only voting and translation.">
</a>
<div class="pub-head">
<h3><a href="assets/realtime-slt.pdf">Real-Time Sign Language Translation by Prototype-Assisted Causal Streaming</a></h3>
<p class="authors"><span class="me">Junyi Hu</span>, Zhewen He, Haomian Huang, Zhenhua Li, Yi Fang</p>
<div class="meta">
<span class="tag preprint">Under Review</span>
<span class="tag role">First Author</span>
<span class="tag">Real-Time Streaming</span>
</div>
</div>
</div>
<p class="blurb">
The <b>first real-time translation pipeline built for the real world</b>, and both fast and accurate.
Online systems so far simply run a recognizer over sliding windows; this one is designed for noisy live
input: it trusts each joint only as far as the pose detector does, recognizes from a short window of
motion, and commits each word once, as soon as it is sure, without ever reading future frames.
Experiments are on five sign-language translation benchmarks.
</p>
<div class="links-row">
<a href="assets/realtime-slt.pdf">PDF</a>
</div>
</article>
<!-- ── SignNet-1M ────────────────────────────────────────── -->
<article class="pub" id="signnet">
<div class="pub-top">
<a class="teaser" href="https://signnet.chatsign.ai/">
<img loading="lazy"
src="https://signnet.chatsign.ai/assets/overview.png"
alt="Overview of the SignNet-1M pipeline: novel-view rendering, scene editing and performer substitution.">
</a>
<div class="pub-head">
<h3><a href="https://signnet.chatsign.ai/">SignNet-1M: Large-Scale Multilingual Sign Language Video Dataset with Downstream Benchmarks</a></h3>
<p class="authors">Zhewen He, <span class="me">Junyi Hu</span>, Haomian Huang, Zhenhua Li, Yu-Shen Liu, Yi Fang</p>
<div class="meta">
<span class="tag venue">ECCV 2026</span>
<span class="tag role">Co-Author</span>
<span class="tag">~1M Clips · 2,058 Hours</span>
</div>
</div>
</div>
<p class="blurb">
Recorded motion video comes from a few cameras, rooms and people, so a model trained on it breaks when
any of them changes, and a benchmark score does not say which one did it. SignNet-1M re-renders
existing video with new viewpoints, backgrounds and performers while keeping the motion, and tags every
clip with what was changed and by how much: about one million clips (2,058 hours), built from seven
sign-language corpora.
</p>
<div class="links-row">
<a href="https://signnet.chatsign.ai/">Project page</a><span class="sep">·</span>
<a href="https://arxiv.org/abs/2606.24361">arXiv</a><span class="sep">·</span>
<a href="https://github.com/openhe-hub/SignNet-1M">Code</a><span class="sep">·</span>
<a href="https://drive.google.com/drive/folders/1Wc43peMNvbkyQ_VGgqi9vdYjA_0Fc3HX">Dataset</a>
</div>
</article>
<div class="dirlabel"><span class="idx">WHERE THEY MEET</span><h3>From a human demonstration to a robot’s execution</h3><span class="bar"></span></div>
<!-- ── PEWAM ─────────────────────────────────────────────── -->
<article class="pub" id="pewam">
<div class="pub-top">
<a class="teaser" href="https://junyi2005.github.io/pewam/">
<img loading="lazy"
src="https://junyi2005.github.io/pewam/static/teaser.png"
alt="Copying the demonstrator fails on a held-out task, after a push, on another arm and under a new grasp; programming the effect succeeds in all four.">
</a>
<div class="pub-head">
<h3><a href="https://junyi2005.github.io/pewam/">Keep the Effect, Drop the Actor: Programmable Effect-to-Execution World-Action Models</a></h3>
<p class="authors"><span class="me">Junyi Hu</span>, Zhewen He, Yi Fang</p>
<div class="meta">
<span class="tag preprint">Under Review</span>
<span class="tag role">First Author</span>
<span class="tag">Human → Robot</span>
</div>
</div>
</div>
<p class="blurb">
A demonstration shows two things at once: what happened to the objects, and how one particular arm made
it happen. PEWAM keeps only the first. It compiles a demonstration into an effect program, which
records where each object travels, where it is held and how the scene ends, with the demonstrator’s
body removed, and a world-action model works out, closed loop, how its own body can produce that
effect. A person can edit the program by hand, the same program runs on four different arms without any
retraining, and on a real Franka arm programs taken from single human videos complete <b>36 of 40</b>
trials, against 22 for the same model given a goal image.
</p>
<div class="links-row">
<a href="https://junyi2005.github.io/pewam/">Project page</a><span class="sep">·</span>
<a href="https://junyi2005.github.io/pewam/pewam.pdf">PDF</a><span class="sep">·</span>
<a href="https://junyi2005.github.io/pewam/#video">Video</a>
</div>
</article>
<div class="dirlabel"><span class="idx">OTHER WORK</span><h3>Surface representation</h3><span class="bar"></span></div>
<!-- ── HeatAtlas ─────────────────────────────────────────── -->
<article class="pub" id="heatatlas">
<div class="pub-top">
<a class="teaser" href="assets/heatatlas.pdf">
<img loading="lazy"
src="assets/heatatlas-overview.png"
alt="HeatAtlas overview: local charts unfolded from the mesh, short-time heat kernel weights, chart content, and composition.">
</a>
<div class="pub-head">
<h3><a href="assets/heatatlas.pdf">HeatAtlas: Surface Textures as Learnable Atlases of Local Heat Kernels</a></h3>
<p class="authors">Zhewen He, <span class="me">Junyi Hu</span>, Yi Fang</p>
<div class="meta">
<span class="tag preprint">Under Review</span>
<span class="tag role">Co-Author</span>
<span class="tag">Surface Representation</span>
</div>
</div>
</div>
<p class="blurb">
Heat-kernel textures describe appearance on a surface through how heat spreads over it, but evaluate
every kernel through a global spectral basis of the whole mesh. HeatAtlas evaluates each kernel where
it lives, on a local chart unfolded from the mesh, and makes those charts the representation itself,
which removes all global preprocessing. On a set of 32 structurally hard objects it reaches <b>32.29
dB</b>, against 26.82 dB for the strongest baseline.
</p>
<div class="links-row">
<a href="assets/heatatlas.pdf">PDF</a>
</div>
</article>
</div>
</section>
<section id="future">
<div class="wrap">
<h2>Future Work</h2>
<div class="prose">
<p>
A human demonstration is usually consumed as a trajectory to imitate. <a href="#pewam">PEWAM</a> reads
it instead as a statement of <b>how the world should change</b>, but I chose the terms of that
statement: rigid boxes, two contact points, an end state. Effects in the physical world say more than
that. Objects bend, open, pour and break, tasks have an order and a phase, and one change sets up the
next. I want effect programs that can express these things, and world-action models that find their own
way to carry them out, even on bodies they were never trained on.
</p>
<p>
The same holds for the structure my models rely on. The causal factorization in <a
href="#cstwm">CST-WM</a>, the body-part streams in <a href="#signdino">SignDEG</a> and the effect
program in PEWAM were all mine to design, which does not scale past problems I already understand. Can
such structure be <b>discovered</b> from data rather than designed by hand, and what evidence would
show that a discovered structure is real, rather than merely convenient for the task?
</p>
<p>
I also want to explore <b>new foundational representations</b>. Heat diffusion is one example I have
started to work with: it describes how a quantity spreads from each point to its neighbours over time
using nothing but local geometry, which makes it a natural way to write down how local influence builds
up into global structure.
</p>
</div>
</div>
</section>
<section id="about">
<div class="wrap">
<h2>About</h2>
<div class="prose">
<p>
I am an undergraduate in the School of Mathematical Sciences at <b>Fudan University</b>, on a
First-Class Undergraduate Scholarship (top 5%), and I am now applying for PhD study to begin in Fall
2027.
</p>
<p>
Since July 2025 I have been a research assistant at <b>New York University Abu Dhabi</b>, working with
Prof. Yi Fang, where the work on this page was done. I lead each project end to end, from the question
to the experiments, the writing and the rebuttal, and I build the systems myself: multi-GPU training,
simulation environments, and deployment on real robots, a Unitree G1 and Go2, an xArm7 and a Franka
FR3. My training in mathematics shows in the tools I reach for: optimal transport to match two
sequences, factorizations chosen for what they rule out.
</p>
<p>
Alongside this, I intern on site at Liqing Intelligence, a world-model startup, and run an NSFC
undergraduate project on grading disc degeneration from MRI in collaboration with Shanghai Ninth
People’s Hospital.
</p>
</div>
</div>
</section>
<section id="contact" class="contact">
<div class="wrap">
<h2>Contact</h2>
<p class="mono">
NYU <a href="mailto:jh10472@nyu.edu">jh10472@nyu.edu</a><br>
Fudan <a href="mailto:23307110151@m.fudan.edu.cn">23307110151@m.fudan.edu.cn</a><br>
Phone <a href="tel:+8618155912005">+86 18155912005</a> <span class="note">(same as WeChat)</span>
</p>
</div>
</section>
<footer>
<div class="wrap">
<p>© <span id="yr">2026</span> Junyi Hu</p>
</div>
</footer>
<script>document.getElementById('yr').textContent = new Date().getFullYear();</script>
</body>
</html>