Using multiple threads in CLI?

[ibrila]

Hi,

I'm trying to use batdetect2 to process thousands of files for a massive project and I am using CLI on a supercomputer. I would be able to use multiple threads to speed things up, but I am not sure if it is supported for batdetect2 or how hard it would be to implement. Do you think this is something that would be possible?

I also have a few smaller questions or suggestions, can I send an e-mail or is posting some "issues" here better?

Thanks,
Ilze

[mbsantiago]

Hi @ibrila!

Thanks for the issue.

I’ve finally published version 2.0.0b1, which introduces new batch processing capabilities. It now supports using multiple workers to load audio in parallel and assembles larger batches to send to the model at once. This should significantly speed things up if you have a GPU and can handle large batch sizes.

Note that it still uses a single process to run the model. If you're on a GPU, running multiple processes with separate model copies likely won't work well. However, if you're using a CPU, you could try listing your files in a text document, splitting that list, and running batdetect2 process file_list on each segment. I'm not entirely sure how that will perform, but if you're keen to experiment with it, I'd be very interested to hear your results!

Hope this helps!
Cheers,
Santiago

[Jeff37]

Labrīt Ilze, hi batdetect2 guys !

A preliminary feedback of mine after upgrading to BD2.0.0b1 (I've been using quite a lot v1 these last years and I'm so happy you issued this brand new version!).

I did some testing on my desktop (24 cores CPU + NVIDIA RTX 3050 GPU) and my laptop (8 × AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx + AMD Radeon Vega 8 Graphics). My OS in Linux Ubuntu 24.04.
First of all, I'd like to understand better this message shown at startup of batdetect2 CLI (I should admit that I didn't dig in a lot yet!):

You are using a CUDA device ('NVIDIA GeForce RTX 3050') that has Tensor Cores. To properly utilize them, you should set torch.set_float32_matmul_precision('medium' | 'high') which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
Where should I set up the torch.set_float32_matmul_precision() parameter ? A quick exchange with an AI tells me to create a config.py file ... but I'm not conviced at all...

The startup message also advised to "increase the value of the num_workers argumenttonum_workers=23in theDataLoader`" so I've added the flag "--workers 23" to batdetect2 process ... command and it work very well!! But note that on my system monitor I see that all 24 cores are working, whatever the value of the --worker flag is set !?

1. Test on my desktop with a dataset of 1522 wav files (from an Audiomoth device), each file is 5 second long and the sampling rate is 384kHz
   -> defaults settings : 1522/1522 [03:05<00:00, 8.18it/s]
   -> with --workers 23 : 1522/1522 [01:07<00:00, 22.71it/s]

2. Test on my laptop with a dataset of 3295 wav (from Batlogger S2), files last from 2 to 15 sec and the sampling rate is 312.5kHz
   -> defaults settings : 3295/3295 [1:16:03<00:00,  0.72it/s]
   -> with --workers 7 : 3295/3295 [1:14:39<00:00,  0.74it/s]

==> Interestingly, the flag --workers 7 changed nothing on the efficiency of the processing on the laptop. On the desktop it flagging 23 workers divided the processing time by 3.
All together, I find the version 2 very easy to install and pilot. The processing is efficient on my desktop (would be worth to compare with other local classificators). I'll share other thoughts on the chiropterological point of view in a separate post.

Thanks again for this amazing work!
Best.

Jean-François Godeau (mailto: jfgodeau-at-gmail.com)

[mbsantiago]

Hi @Jeff37 ,

Wow that was quick, thanks for all the feedback.

Regarding the tensor core messages, to be honest I am not 100% sure about it, it is emited by pytorch lightning which is a framework we're using to manage working across devices, training processes, etc. I haven't tested the recommendation myself, but I believe you could activate it if you write a short python script:

from batdetect2 import BatDetect2API
import torch

torch.set_float32_matmul_precision('medium')

api = BatDetect2API.from_checkpoint()

# do whatever you want ...

In principle this should make it faster, but potentially degrade detection/classification performance a bit. Let me know what you find if you try it out.

Regarding the number of workers: these are separate processes only used for data loading, independent of the main process. So seeing 24 cores being used when running with --workers 23 is ok.

Good to see the reduction in processing time with 23 workers. This means that data was not being loaded fast enough for the model (or the model was eating data more quickly than it was being fed). But only getting 3x speedup from 23 parallel data loading workers makes me think that you are now loading faster than the model can process. So at this point, you can make things faster by increasing the batch size, since each item in the batch is processed in parallel. Try increasing the batch size with --batch-size to something like 32 (default is 8), or even higher.

For the laptop, it's likely that the bottleneck is model processing rather than data loading, so increasing the number of workers does not help much. Try increasing the batch size there too, that may help.

Cheers,
Santiago

[Jeff37]

Hi,
A quick follow-up on performance analysis with batdetect2 CLI.
I performed the following tests on the same data set on my desktop (see previous msg for spec): dataset of 1522 wav files (from an Audiomoth device), each file is 5 second long and the sampling rate is 384kHz

1. batdetect2 process directory /my/folder/ --detection threshold 0.4 : 1522/1522 [03:54<00:00, 6.48it/s]
2. batdetect2 process directory /my/folder/ --detection threshold 0.4 --batch-size 32 : 381/381 [01:45<00:00, 3.60it/s]
3. batdetect2 process directory /my/folder/ --detection threshold 0.4 --workers 23 : 1522/1522 [01:05<00:00, 23.28it/s]
4. batdetect2 process directory /my/folder/ --detection threshold 0.4 --batch-size 32 --workers 23 : 381/381 [00:59<00:00, 6.43it/s]
5. batdetect2 process directory /my/folder/ --detection threshold 0.4 --batch-size 23 --workers 23: 530/530 [00:58<00:00, 9.09it/s]
6. batdetect2 process directory /my/folder/ --detection threshold 0.4 --batch-size 64 --workers 23: 191/191 [00:56<00:00, 3.38it/s]

The way it defines number of items is a bit puzzling... anyways, with batch size of 64 and workers 23 the initial computing time dropped from 234 sec to 56 sec, ie. 24% of the defaults value. Worth giving try :-)