Skip to content

Commit 3875399

Browse files
jianglongyejianglongye
committed
init: add code
0 parents  commit 3875399

File tree

147 files changed

+622093
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

147 files changed

+622093
-0
lines changed

‎.gitignore‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
output
2+
*.pdf
3+
4+
# IDE
5+
.idea
6+
.vscode
7+
8+
# Byte-compiled / optimized / DLL files
9+
__pycache__/
10+
*.py[cod]
11+
*$py.class
12+
13+
# C extensions
14+
*.so
15+
16+
# Distribution / packaging
17+
.Python
18+
build/
19+
develop-eggs/
20+
dist/
21+
downloads/
22+
eggs/
23+
.eggs/
24+
lib/
25+
lib64/
26+
parts/
27+
sdist/
28+
var/
29+
wheels/
30+
pip-wheel-metadata/
31+
share/python-wheels/
32+
*.egg-info/
33+
.installed.cfg
34+
*.egg
35+
MANIFEST
36+
37+
# PyInstaller
38+
# Usually these files are written by a python script from a template
39+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
40+
*.manifest
41+
*.spec
42+
43+
# Installer logs
44+
pip-log.txt
45+
pip-delete-this-directory.txt
46+
47+
# Unit test / coverage reports
48+
htmlcov/
49+
.tox/
50+
.nox/
51+
.coverage
52+
.coverage.*
53+
.cache
54+
nosetests.xml
55+
coverage.xml
56+
*.cover
57+
*.py,cover
58+
.hypothesis/
59+
.pytest_cache/
60+
61+
# Translations
62+
*.mo
63+
*.pot
64+
65+
# Django stuff:
66+
*.log
67+
local_settings.py
68+
db.sqlite3
69+
db.sqlite3-journal
70+
71+
# Flask stuff:
72+
instance/
73+
.webassets-cache
74+
75+
# Scrapy stuff:
76+
.scrapy
77+
78+
# Sphinx documentation
79+
docs/_build/
80+
81+
# PyBuilder
82+
target/
83+
84+
# Jupyter Notebook
85+
.ipynb_checkpoints
86+
87+
# IPython
88+
profile_default/
89+
ipython_config.py
90+
91+
# pyenv
92+
.python-version
93+
94+
# pipenv
95+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
96+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
97+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
98+
# install all needed dependencies.
99+
#Pipfile.lock
100+
101+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
102+
__pypackages__/
103+
104+
# Celery stuff
105+
celerybeat-schedule
106+
celerybeat.pid
107+
108+
# SageMath parsed files
109+
*.sage.py
110+
111+
# Environments
112+
.env
113+
.venv
114+
env/
115+
venv/
116+
ENV/
117+
env.bak/
118+
venv.bak/
119+
120+
# Spyder project settings
121+
.spyderproject
122+
.spyproject
123+
124+
# Rope project settings
125+
.ropeproject
126+
127+
# mkdocs documentation
128+
/site
129+
130+
# mypy
131+
.mypy_cache/
132+
.dmypy.json
133+
dmypy.json
134+
135+
# Pyre type checker
136+
.pyre/

‎README.md‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# Continuous Grasping Function
2+
3+
This repository is the official implementation of the paper:
4+
5+
> Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrations
6+
> [_Jianglong Ye_](https://jianglongye.com/)<sup>\*</sup>, [_Jiashun Wang_](https://jiashunwang.github.io/)<sup>\*</sup>, [_Binghao Huang_](https://binghao-huang.github.io/), [_Yuzhe Qin_](https://yzqin.github.io/), [_Xiaolong Wang_](https://xiaolonw.github.io/)
7+
> RA-L 2023, IROS 2023
8+
9+
[Project Page](https://jianglongye.com/cgf/) / [ArXiv](https://arxiv.org/abs/2207.05053) / [Video](https://www.youtube.com/watch?v=mJ2d7ZfNyUw)
10+
11+
<div align="center">
12+
<img src="assets/images/interpolation.jpg" width="90%" />
13+
</div>
14+
15+
## Environment Setup
16+
17+
(Our code has been tested on Ubuntu 20.04, python 3.10, torch 1.13.1, CUDA 11.7 and RTX 3090)
18+
19+
To set up the environment, follow these steps:
20+
21+
```sh
22+
conda create -n cgf python=3.10 -y && conda activate cgf
23+
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia -y
24+
25+
pip install six numpy==1.23.1 tqdm pyyaml scipy opencv-python trimesh einops lxml transforms3d viser
26+
pip install git+https://github.com/hassony2/chumpy.git
27+
pip install git+https://github.com/hassony2/manopth
28+
pip install git+https://github.com/facebookresearch/pytorch3d.git
29+
pip install git+https://github.com/UM-ARM-Lab/pytorch_kinematics
30+
```
31+
32+
## Data Preparation
33+
34+
We provide processed data for training CGF which can be downloaded from [here](https://drive.google.com/file/d/1O5FUeulQqM1ylXhU0WyjoIRyBvVZw27k/view?usp=sharing) and unzip it under `PROJECT_ROOT/data/`. The final file structure should look like this:
35+
36+
```
37+
PROJECT_ROOT
38+
├── ...
39+
└── data
40+
├── ...
41+
└── processed
42+
├── ...
43+
├── meta.json
44+
├── pose_m_aug.npz
45+
└── pose_m_object_coord.npz
46+
```
47+
48+
Alternatively, you can prepare the data by following the instructions below.
49+
50+
1. Download DexYCB data from the [official website](https://dex-ycb.github.io/) to `DEXYCB_DATA_ROOT` and organize the data as follows:
51+
```
52+
DEXYCB_DATA_ROOT
53+
├── 20200709-subject-01/
54+
├── 20200813-subject-02/
55+
├── ...
56+
├── calibration/
57+
└── models/
58+
```
59+
2. Run the following script to process the data:
60+
```sh
61+
python scripts/prepare_dexycb.py --data_root DEXYCB_DATA_ROOT
62+
python scripts/retargeting.py --mano_side left
63+
python scripts/retargeting.py --mano_side right
64+
```
65+
The processed data will be saved to `PROJECT_ROOT/data/processed`. Retargeting results can be visualized by running the following command:
66+
```sh
67+
python scripts/visualize_retargeting.py
68+
```
69+
![retargeting](assets/images/cgf-retargeting.gif)
70+
71+
## Running
72+
73+
### Training
74+
75+
To train the CGF model, run the following command:
76+
77+
```sh
78+
# cd PROJECT_ROOT
79+
python scripts/main.py
80+
```
81+
82+
### Sampling
83+
84+
To sample from the trained CGF model, run the following command (replace `TIMESTAMP` with the timestamp of the checkpoint in the `PROJECT_ROOT/output` directory):
85+
86+
We provide a pre-trained model which can be downloaded from [here](https://drive.google.com/file/d/1enUS9p2-_pUFyMtwWzb3mJgq1l0Tjy9a/view?usp=sharing) and unzip it under `PROJECT_ROOT/output/`.
87+
88+
```sh
89+
# cd PROJECT_ROOT
90+
python scripts/main.py --mode sample --ts TIMESTAMP
91+
```
92+
93+
The sampled results will be saved to `PROJECT_ROOT/output/TIMESTAMP/sample`. They can be visualized by running the following command:
94+
95+
```sh
96+
python scripts/visualize_sampling.py --ts TIMESTAMP
97+
```
98+
99+
![sampling](assets/images/cgf-sampling.gif)
100+
101+
To evaluate the smoothness of the sampled grasps (table 1 in the paper), run the following command:
102+
103+
```sh
104+
python scripts/eval_generation.py --ts TIMESTAMP
105+
```
106+
107+
Note that the evaluation results may vary slightly due to the randomness in the sampling process.
108+
109+
### Simulation
110+
111+
Our simulation environment is adapted from [dex-hand-teleop](https://github.com/yzqin/dex-hand-teleop). To setup the simulation environment, follow these steps:
112+
113+
```sh
114+
pip install sapien==2.2.2 gym open3d
115+
```
116+
117+
To run the simulation, run the following command (replace `SAPIEN_SAMPLE_DATA_PATH` with the path to a sample data in the `PROJECT_ROOT/output/TIMESTAMP/sample/result_filter_sapien` directory):
118+
119+
```sh
120+
python scripts/sim.py --data_path SAPIEN_SAMPLE_DATA_PATH
121+
```
122+
123+
## Citation
124+
125+
```bibtex
126+
@article{ye2023learning,
127+
title={Learning continuous grasping function with a dexterous hand from human demonstrations},
128+
author={Ye, Jianglong and Wang, Jiashun and Huang, Binghao and Qin, Yuzhe and Wang, Xiaolong},
129+
journal={IEEE Robotics and Automation Letters},
130+
year={2023}
131+
}
132+
```

‎assets/images/cgf-retargeting.gif‎

180 KB
Loading

‎assets/images/cgf-sampling.gif‎

137 KB
Loading

‎assets/images/interpolation.jpg‎

665 KB
Loading

‎assets/misc/cube.obj‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Blender v2.93.4 OBJ File: ''
2+
# www.blender.org
3+
mtllib untitled.mtl
4+
o Cube
5+
v 1.000000 1.000000 -1.000000
6+
v 1.000000 -1.000000 -1.000000
7+
v 1.000000 1.000000 1.000000
8+
v 1.000000 -1.000000 1.000000
9+
v -1.000000 1.000000 -1.000000
10+
v -1.000000 -1.000000 -1.000000
11+
v -1.000000 1.000000 1.000000
12+
v -1.000000 -1.000000 1.000000
13+
vt 1.000000 1.000000
14+
vt 0.000000 1.000000
15+
vt 0.000000 0.000000
16+
vt 1.000000 0.000000
17+
vt 1.000000 0.000000
18+
vt 1.000000 1.000000
19+
vt 0.000000 1.000000
20+
vt 0.000000 0.000000
21+
vt 1.000000 1.000000
22+
vt 1.000000 0.000000
23+
vt 0.000000 1.000000
24+
vt 1.000000 1.000000
25+
vt 1.000000 0.000000
26+
vt 0.000000 1.000000
27+
vt 0.000000 0.000000
28+
vt 0.000000 0.000000
29+
vn 0.0000 1.0000 0.0000
30+
vn 0.0000 0.0000 1.0000
31+
vn -1.0000 0.0000 0.0000
32+
vn 0.0000 -1.0000 0.0000
33+
vn 1.0000 0.0000 0.0000
34+
vn 0.0000 0.0000 -1.0000
35+
usemtl Material
36+
s off
37+
f 1/1/1 5/2/1 7/3/1 3/4/1
38+
f 4/5/2 3/6/2 7/7/2 8/8/2
39+
f 8/8/3 7/7/3 5/9/3 6/10/3
40+
f 6/11/4 2/12/4 4/5/4 8/8/4
41+
f 2/13/5 1/1/5 3/14/5 4/15/5
42+
f 6/16/6 5/2/6 1/1/6 2/13/6

‎assets/misc/table_map.jpg‎

39.9 KB
Loading

0 commit comments

Comments
 (0)