This is the repository for paper Automatic Personalized Limbed Robot Design from Media Inputs. (under review).
Authored by Gang Chen*, Moji Shi*, Yu Xing, Marija Popovic, Javier Alonso-Mora, Lei Zhang, Jiangmiao Pang
Our system automatically design a robot from 3D mesh generated by text/image. The output contains a URDF file with motor choices and 3D printable mesh components.
"*" Equal contributions.
This project is released for academic research and non-commercial use only.
Tested Environment: Ubuntu 20.04
(Using Mamba in the following. Change to Conda if you prefer conda).
- If you don't have Mamba installed, refer to Fresh installation in Page.
- Create a Mamba environment and activate
mamba create --name anything2robot python=3.9
mamba activate anything2robot
- Install Open3d and scikit-learn. Please make sure that the version of open3d is 0.18.0.
python3 -m pip install --user open3d==0.18.0 scikit-learn==1.3.2 scikit-image
- Install UI related libs
pip install -U kaleido==0.2.1 vtk pyvista trimesh
mamba install conda-forge::qt anaconda::pyqt conda-forge::pyvistaqt
- Install openscad, CGAL, solidpython. Run the following command
sudo apt-get install openscad libcgal-dev
pip install solidpython
By default this will install CGAL 5.0.2-3. when using Ubuntu 20.04. A different CGAL version may result in error.
- Install meshio by
pip install meshio[all]
- Install pinochhio
conda install conda-forge::pinocchio==2.7.0
- Install Ansys 2023 R2 (Note: Student edition can't be installed in Ubuntu. Make sure you have the license). To install Ansys on ubuntu, first, install the dependencies shown below. Then follow the official instructions of Ansys to install it in Ubuntu. NOTE: Use the default
/root/ansys_incfolder to install. Install only MAPDL is sufficient. Finally, install PyAnsy.
sudo apt install ubuntu-desktop alien freeglut3 libxcb-xinerama0 lsb xterm libmotif-common libmotif-dev
Install MAPDL (Structural Mechanics) on your ubuntu system. Use the default installation path otherwise the installation may fail.
python -m pip install pyansys
- Clone the code to your PC and compile the C++ part.
git clone git@github.com:g-ch/anything2robot.git
# Then compile the code that uses C++ with the following commands
cd anything2robot/metamaterial_filling
mkdir build
cd build
cmake ..
make
- Make a data folder. Put your stl file/folder in the data folder.
cd metamaterial_filling
mkdir data
This repo assumes a '.stl' 3D model has been acquired by AI text/image to 3D model tools like Meshy. Make sure the model is scaled to a reasonable size, such as 20 cm - 100 cm. You can use metamaterial_filling/build/scaleMesh to scale the model.
The command is
./scaleMesh <path_to_stl_file> <path_to_scaled_stl_file> <scale_factor>
The joints should either 1) be acquired by human/animal pose estimation tools; 2) manually added by you in the UI that will be shown when you run the auto design script later. For the first case, the joints must be transformed into the format used in this repo. Check auto_design/modules/quadruped_pose_to_pkl.py for examples. Note the name of contact point to the ground is treated as a joint should contain string "foot" in its name. Each link should have at least two joints. Axis of a fixed joint, e.g. foot, is (0,0,0).
The UI looks like the following:
NOTE: Currently, the axis can only be one dof or two dofs, for one dof, the choices are (1,0,0), (0,1,0), (0,0,1) or their negations; for two dofs, write the valid x axis first, then y axis, then z axis, e.g. (1,0,0) (0,1,0).
The information of the available motors are stored in motor_param_lib.py. You can add your own motor information to the lib.
To start the auto design, run
python script/auto_design.py --stl_mesh_path <path_to_stl_file> --joint_pkl_path <path_to_joint_pkl_file> --expected_x <expected_x_length_of_the_model in cm> --voxel_size <voxel_size in cm. e.g. 0.5 cm>If you already have a pkl file generated by 1), specify the path to the pkl file using --joint_pkl_path. Otherwise, this file will be saved to the provided path when the UI is used.
- Details of more arguments can be found in the main function of
script/auto_design.py. - To Use FEA, please install Ansys, change the
exec_loc = '/root/ansys_inc/v232/ansys/bin/ansys232'inmetamaterial_filling/script/pyansys_fea/mapdl_msh_analysis.pyto the path of the Ansys executable in your system, and set--do_fea_analysistoTruewhen running the auto design script.
When the auto_design process is started, you will see a UI. You can use the UI to manually add/modify joints. A webpage will be shown to visualize added joints positions and axes. Refresh the webpage to see the latest changes.
Click "save" to save the joint pkl file and "Start Design" to start the auto design. When the process is finished, you will see the result (a folder named with the stl file name) in the result folder.
The mesh file path in the result URDF file is a ROS format path. If you want to show the urdf directly with VSCode plugin "URDF Visualizer" or online visualization tools, you need to remove the path. You can also do this by running auto_design/modules/urdf_motor_adding.py, which will add the motors (based on the motor size given in motor_param_lib.py) to the urdf and save to a new folder. This new urdf file uses the relative path for direct URDF visualization.
- 1: Failure Code 1. The motor cost is too high.
- 2: Failure Code 2. The mesh is destroyed during the optimization.
- 3: Failure Code 3. The mesh is not watertight.
- 4: Failure Code 4. The mesh is not feasible in FEA.
When one of the failure is triggered, the auto design process will try to scale up the model and try again until the model is feasible or the maximum trial round is reached.
The common reason that can cause the failures:
- The model is not scaled to a reasonable size or the motors in the
motor_param_lib.pyare not suitable for the model. - The joint is too close to the surface of the mesh so that no enough space for the motor.
- The mesh body is too thin.
This project is released for academic research and non-commercial use only.
Any commercial use, including but not limited to use in commercial products, services, or for-profit research, is strictly prohibited without explicit permission.
If you use this work in your research, please cite our paper.