Hi, thanks for the minimal implementation of BYOL. This may be not relevant anymore since it is a relatively old model, but I find that training is quite slow with GPU utilization staying low, due to heavy data loading and multi-view augmentations.
This is a recurring problem with self-supervised models that use this training paradigm, and the NVIDIA DALI library may be a solution, since the augmentation and data processing steps are moved to the GPU.
I am opening a pull request that implements the data loading operations using NVIDIA DALI and integrates them into the dataloader with little modifications to the main code.
Hi, thanks for the minimal implementation of BYOL. This may be not relevant anymore since it is a relatively old model, but I find that training is quite slow with GPU utilization staying low, due to heavy data loading and multi-view augmentations.
This is a recurring problem with self-supervised models that use this training paradigm, and the NVIDIA DALI library may be a solution, since the augmentation and data processing steps are moved to the GPU.
I am opening a pull request that implements the data loading operations using NVIDIA DALI and integrates them into the dataloader with little modifications to the main code.