 |
 |
 |
| KManipSoloArm environment has one 7dof arm with a 1dof gripper. KManipSoloArmVision has a gripper cam, a head cam, and an overhead cam. | KManipDualArm environment has two 7dof arms with 1dof grippers. KManipDualArmVision has 2 gripper cams, a head cam, and an overhead cam. | KManipTorso environment has a 2dof head, two 6dof arms with 1dof grippers. KManipTorsoVision has 2 gripper cams, a head cam, and an overhead cam. |
clone and install dependencies
git clone https://github.com/kscalelabs/gym-kmanip.git && cd gym-kmanip
conda create -y -n gym-kmanip python=3.10 && conda activate gym-kmanip
pip install -e .run tests
pip install pytest
pytest tests/test_env.pyvisualize the mujoco scene
python gym_kmanip/examples/1_view_env.pyrecord a video of the mujoco scene
python gym_kmanip/examples/2_record_video.py🤗 K-Scale HuggingFace Datasets
data is recorded via teleop, this requires additional dependencies
pip install opencv-python==4.9.0.80
pip install vuer==0.0.30
pip install rerun-sdk==0.16.0start the server on the robot computer
python gym_kmanip/examples/4_record_data_teleop.pystart ngrok on the robot computer.
ngrok http 8012open the browser app on the vr headset and go to the ngrok url
data is visualized using rerun
rerun gym_kmanip/data/test.rrd✅ solo arm w/ vision
✅ dual arm w/ vision
✅ torso w/ vision
✅ inverse kinematics using mujoco
⬜️ tune and improve ik
⬜️ recording dataset via teleop
⬜️ training policy from dataset
⬜️ evaluating policy on robot
- Gymnasium is used for environment
- MuJoCo is used for physics simulation
- PyTorch is used for model training
- Rerun is used for logging datasets
- HuggingFace is used for dataset & model storage
- Vuer teleop only is used for visualization
- ngrok teleop only is used for networking
helpful links and repos
@misc{teleop-2024,
title={gym-kmanip},
author={Hugo Ponte},
year={2024},
url={https://github.com/kscalelabs/gym-kmanip}
}