MANO is a human hand model which represent human hand as 48D(here we only take into account 45D, without palm) pos. In this work, we reduce the dimension of the MANO pose dataset, using VAE encoder to represent the 45D hand joints representation into 9D latent, then reconstruct the 9D latent back to 45D hand joints.
This project uses MANO pose data processed from the GRAB dataset.
Due to GitHub file size limits, the dataset is hosted externally.
Download the dataset from Google Drive: https://drive.google.com/file/d/XXXX/view?usp=sharing
After downloading, create a file named "hand_dataset" parellal to directory "Compress", extract the zip file into "hand_dataset"
We use GRAB data set to complete this object. To derive data of MANO Pose from GRAB dataset, we need to prepare our dataset in the first place.
run "data_extract.py", you'll get the npz file: "grab_mano_theta_all_norm.npz" which contains millions of data from MANO pose (shape: N*45).
VAE structure can be seen in file "vae.py", and for VAE loss, it's in file "loss.py". You can design your own VAE by modifying these two files. To train the VAE, run "train_vae.py", this will save a model called "hand_pose_vae_all_z9.pt". By default the latent of VAE is equal to 9.
run "eval_recon.py", then visualized evaluation matrices will shown in the new file.
Example:
