Feasibility study on integrating a supervised Image-to-Image learning model for visual data prediction in numerical simulation codes to accelerate computation times.
This project aims to implement and evaluate various supervised learning models using pre-trained models from TensorFlow, Mistral AI, and Stability AI, in order to assess their accuracy and the quality of their simulated prediction images.
Pix2Pix is an image-to-image translation model using a conditional Generative Adversarial Network (cGAN). It learns the mapping from input images and converts them into output images.
Pixtral-12B is a text-to-image and image-to-text model designed to analyze, describe, and transform images. It encodes both textual prompts and visual inputs into a shared representation, enabling it to generate coherent image descriptions as well as new images conditioned on prompts.
Stable Diffusion is a latent diffusion model that combines a variational autoencoder (VAE) to embed images into a compressed latent space and a CLIP-based text encoder to process textual prompts. By progressively denoising latent representations, the model reconstructs high-resolution images that align with the semantic meaning of the input text. This process allows efficient generation of detailed, high-quality synthetic images from natural language prompts.
link to the originel tutorials:
https://www.tensorflow.org/tutorials/generative/pix2pix?hl=fr
https://mistral.ai/fr/news/unlocking-potential-vision-language-models-satellite-imagery-fine-tuning
https://huggingface.co/radames/stable-diffusion-v1-5-img2img
1- Download dependencies : retrieve the required libraries (compiled PyPI packages) from the internet into a dedicated local folder
my_pip_download () {
pip download --only-binary=:all: "$@"
}
# NB : --only-binary=:all: force le pip à ne télécharger que les fichiers binaires (fichiers .whl) et pas les fichiers sources (fichiers .tar.gz)
my_pip_download -r requirements.txt2- Create and initialize a Python virtual environment
python -m venv my_env
source my_env/bin/activate3- Install all libraries from the local folder
export REPO_HOME="<path/to/folder>"
my_pip_install(){
pip install -f $REPO_HOME/ --no-index "$@"
}
my_pip_install -r requirements.txt1- Transfer packages: Copy the folder containing the compiled PyPI packages to the supercomputer’s storage
2- Module set up : load the compatibility stack of GNU(compiler)/CUDA(GPU acceleration)/MPI(distributed or parallel execution)/Python
3- Resource Management: Use a job submission script to request correct allocation of CPUs, GPUs, and memory
4- Job scheduling: Use the supercomputer’s scheduler to submit training and evaluation jobs
5- Set up environment: Create and initialize a Python virtual environment
6- Install dependencies : install all libraries on the supercomputer from the transferred folder.
export REPO_HOME="<path/to/folder>"
my_pip_install(){
pip install -f $REPO_HOME/ --no-index "$@"
}
my_pip_install -r requirements.txt1- Cloning the Repository : retrieve the project source from the remote Git repositiry
2- Navigate into the cloned project directory to access scripts and configuration files
3- Use sshfs to mount a remote directory from the supercomputer onto the local machine
1- Data Preprocessing : Clean, format, and prepare datasets for training.
2- Model Fine-Tuning : Apply fine-tuning techniques to optimize model performance. Compare different hyperparameter configurations using methods such as Ray (for distributed tuning) or LoRA (Low-Rank Adaptation).
3- Training : Train the selected models with the optimized configurations. Monitor convergence, stability, and resource usage.
4- Performance Evaluation : Assess the models on relevant metrics (e.g., accuracy, image quality, coherence). Compare results across models and fine-tuning strategies.