Skip to content

Add webcam functionality#15

Closed
noahsprent wants to merge 0 commit into
Pioreactor:mainfrom
Change-Bio:main
Closed

Add webcam functionality#15
noahsprent wants to merge 0 commit into
Pioreactor:mainfrom
Change-Bio:main

Conversation

@noahsprent

Copy link
Copy Markdown

No expectations that this should be merged as is, but wanted to open the pull request for better visibility and to have somewhere to discuss.

Basically by running a command like ffmpeg -f v4l2 -i /dev/video0 -vf "scale=640:480" -hls_time 2 -hls_list_size 5 -hls_flags delete_segments webcam.m3u8 on the pi in the /var/www/pioreactorui/data folder, video snippets are saved from a webcam into this folder and refreshed over time. This addition adds a new tab to the UI that can display two of these streams using the hls js library. There may be a better way of doing this but the nice thing about this method is it doesn't require any additional ports, streaming etc. so all the current methods through tailscale etc. should just work.

Some future improvements:

  • Figure out stability issues: sometimes the stream doesn't update
  • Reduce delay: for me it's currently about 25 seconds
  • Show current system time on the webcam page in a non-obtrusive way: Just a small convenience thing to see how delayed the webcam feed is, because one can then have a screen on the feed which displays the system time and you can compare the two numbers
  • Ability (maybe through a plugin?) to run the ffmpeg command to start the streaming from the UI and then to automatically start displaying that, and to detect webcams as well, rather than having to guess /dev/video0, 1, 3 etc.
  • Ability to start recording when an experiment is started, so that webcam recordings are synced with experiment elapsed time to look back through the recording and see what happened at certain time points - should also come with ability to save to some external storage or the cloud.

Current status:
pioreactor_webcams

@CamDavidsonPilon

Copy link
Copy Markdown
Member

Kudos to you for taking this on!

Just some ideas:

Installing via a plugin: I think adding sidebar links can be done via our plugins pattern of adding yaml files. What I'm not sure how to do is adding raw JS: we would have to move the build step to the Pi. That's not too crazy, but not sure what the overall architecture looks like.

Timing webcams to an experiment start might be difficult, or impossible, mostly because simultaneous experiments can be running at once. Hmmmm this is tricky: I almost want to suggest its not tied to experiments, but a separate tool that users can turn on and off independently of experiments. Of course, if a user wants it on during an experiment, they can turn it on right after they start an experiment (not automatically yet...). The data is record in small flat files each minute, and can be gzipped for smaller disk space. Later, a user can ask for all video files between time start-time-of-experiment and some-endtime - maybe? (If we do choose this path, the Webcams link the sidebar should go at the bottom since it would be experiment-independent).

Show current system time on the webcam page in a non-obtrusive way: Just a small convenience thing to see how delayed the webcam feed is, because one can then have a screen on the feed which displays the system time and you can compare the two numbers

What about adding the timestamp into the video feed? Ex: https://superuser.com/questions/1491264/ffmpeg-filter-to-add-timestamp-on-video

@noahsprent

Copy link
Copy Markdown
Author

Sorry made it a separate branch on our fork so I could more easily sync upstream changes. Re-opened here: #16

@noahsprent noahsprent mentioned this pull request Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants