Repository navigation
Open
Description
The nightly ingest of Zhang's ferret hyperscanning data (incoming 000480, standardized 000547) skips the recordings below. All paths are under sourcedata/raw/Ferret_Hyperscanning/ in 000480. Each needs a fix in the source data, or a decision to leave it out.
Waiting on merged ephys files (8 recordings)
These are reported as pending and retried every night. The converter expects each animal's ephys at <recording>.rec/<animal>HS<n>/<recording>_<animal>HS<n>_merged.rec.
| Recording | Folder | Problem |
|---|---|---|
0235HS3_0237HS4_20260320_102109 |
0235-0237/03.20.2026 |
The 0237HS4 file is named …_0237HS4merged.rec, with no _ before merged |
0235HS3_0237HS4_20260325_124806 |
0235-0237/03.25.2026 |
The file in 0235HS3/ is named …_0236HS3_merged.rec, with animal 0236 instead of 0235 |
0235HS3_0237HS4_20260330_150816 |
0235-0237/03.30.2026 |
Both merged files sit at the top of the .rec folder, not in 0235HS3/ and 0237HS4/ |
0235HS6_0237HS6_20260425_132118 |
0235-0237/04.25.2026 |
The 0235HS5/ folder and both merged file names say 0235HS5, while the recording name says 0235HS6 |
0236HS3_0237HS4_20260331_101851 |
0236-0237/03.31.2026 |
No headstage folders or merged files, only the base station .rec |
0236HS3_0237HS4_20260331_110624 |
0236-0237/03.31.2026 |
No headstage folders or merged files |
0236HS3_0237HS4_20260401_130419 |
0236-0237/04.01.2026 |
No headstage folders or merged files |
0236HS5_0237HS6_20260408_155939 |
0236-0237/04.08.2026 |
The 0237HS6 folder is named 0237HS8 |
Never picked up
These are skipped with only a log line, so they don't show as pending.
| What | Where | Count |
|---|---|---|
Session1/Session2 or Session_1/Session_2 subfolders. The two checked each hold a recording and its Videos/ |
0235-0236: 03.18, 04.23. 0235-0237: 03.19, 04.23, 04.28, 05.07, 05.13, 05.20. 0236-0237: 03.18, 04.23, 04.28 |
22 folders |
Calibration videos named CamNN-Calibration-<time>-NNNN.avi. The converter only reads CamNN-cal-<time>-NNNN.avi |
0235-0236: 03.06, 03.08, 03.09, 03.10, 03.13, 03.16. 0235-0237: 03.08, 03.13. 0236-0237: 03.06, 03.08, 03.13, 03.16 |
60 files |
| A recording directly in the pair folder, not in a date folder | 0235-0236/0235HS4_0236HS3_20260327_125659.rec |
1 |
Recording names the converter does not recognize:
| Recording | Problem |
|---|---|
02355HS6_0236HS5_20260408_134217 |
Animal ID 02355, likely 0235 |
0235HS3_0237HS4_Session1_20260308_142653 |
Extra Session1 |
0235HS4_0236HS3_Session1_20260306_154144 |
Extra Session1 |
0236HS3_0237HS4_Session1_20260306_114321 |
Extra Session1 |
0236HS3_0237HS4_Session1_20260308_130347 |
Extra Session1 |
0235HS3_0237HS4_sess1_20260318_111532 |
Extra sess1 |
0236HS3_0237HS4_RecordingTest1_20260223_155333 |
A test recording, maybe left out on purpose |
Also left out, probably on purpose: 0235-0236/03.10.2026/0236HS3_PMC_rawdownsample.set and the README.txt files in 0236-0237/03.06.2026 and 0236-0237/03.08.2026.
Questions for the lab
- Can the merged files in the first table be renamed or moved to the expected layout? Can the missing ones for 03.31 and 04.01 be uploaded?
- Is
0235HS6_0237HS6_20260425_132118recorded on headstage 5 or 6? - Should the
Sessionsubfolders and the oddly named recordings be converted? If so, they can be renamed and moved to the usual layout, or the converter can learn these layouts. - Should
RecordingTest1stay out? - Are the
CamNN-Calibration-…files calibration videos? If so, the converter can accept that spelling.
Metadata
Metadata
Assignees
Labels
No labels
Activity
Read the Zhang lab's naming variants and same-day Session folders
CodyCBakerPhD commentedon Nov 19, 2026
Update from #63, not merged yet. Once it merges, the converter reads:
Session1/Session_1subfolders, the extraSession1,sess1,Sess1orMergein recording names, the missing_beforemerged, and theCamNN-Calibration-…videos.recfolder, as on 03.30correctionstable inlabs/zhang/ferret-hyperscanning/code/config.yamlEach correction was checked against the merged files' headers. The merge copies the recording's start time into both merged files and records the serial of the headstage that recorded each. Those serials are
03103 00106for HS3,00105for HS4,00127for HS5 and00130for HS6.That converts 107 of the 111 recordings on EMBER.
0235-0236/0235HS4_0236HS3_20260327_125659.recis no longer on EMBER. With #63, the nightly download also deletes leftovers like it from the runner.Still needed from the lab
0236-0237/03.31.2026/0236HS3_0237HS4_20260331_101851,0236-0237/03.31.2026/0236HS3_0237HS4_20260331_110624and0236-0237/04.01.2026/0236HS3_0237HS4_20260401_130419. Each has only the base station.rec, which holds no ephys.Renames that would make the names match the contents
These recordings convert through the corrections table, so none of these renames blocks the ingest. Once a file is renamed to the standard name, the converter uses that name instead of its correction. A rename has to cover every name listed for its recording. Otherwise the recording goes back to pending.
0235-0237/03.25.2026/0235HS3_0237HS4_20260325_1248060235HS3/…_0236HS3_merged.recto…_0235HS3_merged.rec0235-0237/04.25.2026/0235HS6_0237HS6_20260425_132118.recto0235HS5_0237HS6_20260425_1321180235HS5/folder and both merged names say HS5, and 0235's file has HS5's serial0236-0237/04.08.2026/0236HS5_0237HS6_20260408_1559390237HS8/to0237HS6/…_0237HS6_merged.recand has HS6's serial. 0237 wore HS8 in the 14:39 recording that day0235-0236/04.08.2026/02355HS6_0236HS5_20260408_13421702355to0235in the recording folder, its base station.recand both merged files0235HS6/folder and its file say 0235, and the serial matches HS60236-0237/03.08.2026/0236HS3_0237HS4_Session1_20260308_1303470236HS3/…_HS30236_merged.recto…_0236HS3_merged.rec, and0237HS4/…_130347_merged.recto…_130347_0237HS4_merged.rec0235-0237/03.30.2026/0235HS3_0237HS4_20260330_1508160235HS3/or0237HS4/Questions
1. March headstage serials. From April on, every merged file's serial matches its HS number, all 138 files. Six files in four March recordings don't:
0235-0236/03.19.2026/0235HS4_0236HS3_20260319_1303210235-0237/03.20.2026/0235HS3_0237HS4_20260320_1021090236-0237/03.19.2026/0236HS3_0237HS4_20260319_1433270236HS3file reports HS4 and the0237HS4file reports HS3. Already published0235-0237/03.18.2026/0235HS3_0237HS4_sess1_20260318_111532Two files can't both come from HS4, so the serial field is sometimes wrong. That's why these recordings are converted under their names. Were the loggers swapped at the merge for the two swapped recordings? If so, the two animals' ephys is swapped in their NWB files.
2. Short partner files. A session's two merged files are usually within 1% of each other in size (85 of 107 sessions). In these six, one animal's file is far smaller. Did that animal's logger stop early? Should these sessions be marked?
0235-0237/04.15.2026/0235HS7_0237HS8_20260415_1647550235-0237/05.19.2026/0235HS5_0237HS6_20260519_1037350235-0237/04.17.2026/0235HS7_0237HS8_20260417_1343020235-0236/04.09.2026/0235HS6_0236HS5_20260409_1510330236-0237/03.08.2026/0236HS3_0237HS4_Session1_20260308_1303470236-0237/04.28.2026/Session_1/0236HS7_0237HS8_20260428_131255Generated by Claude Code
Accepted the Zhang lab's naming variants, corrected mislabeled record…