sagarchotalia / Pick-and-Place-Robot-Eklavya

This project aims to build/use an effective bin packing algorithm and simulate it using a robotic arm in CoppeliaSim.

Pick and Place Robot using Bin Packing

This project aims to use and simulate an efficient bin packing algorithm to pick and place boxes into bins(containers) in CoppeliaSim.

Table of Contents

• About the Project
  • Tech Stack
  • File Structure
• Getting Started
  • Prerequisites
  • Installations
  • Execution
• Theory and Approach
• Results and Demo
• Future Work
• Troubleshooting
• Contributors
• Acknowledgements and Resources
• License

About The Project

Bin packing is one of the most interesting problems in combinatorics. It has multiple applications, such as packaging boxes in containers, loading trucks with weight capacity constraints, creating file backups in media and technology mapping in FPGA semiconductor chip design. This project aims to implement a bin packing algorithm that packs variable sized boxes into containers and simulate the same in a CoppeliaSim Scene.

Tech Stack

• CoppeliaSim
• Spyder
• OpenCV

File Structure

 ┣Algorithms                           # ALgorithms written and tested
 ┃ ┣1D Algorithms                      # 1 Dimensional Bin packing algorithms
 ┃ ┃ ┣BestFitAlgorithm.cpp             
 ┃ ┃ ┣FirstFitAlgorithm.cpp            
 ┃ ┃ ┗NextFitAlgorithm.cpp             
 ┃ ┣2D Algorithms                      # 2 Dimensional Bin packing algorithms
 ┃ ┃ ┗ShelfNextFit.cpp
 ┃ ┗3D Algorithms                      # 3 Dimensional Bin packing algorithms
 ┃ ┃ ┣3DBinPacking.py
 ┃ ┃ ┣3dalgo-dragos.py
 ┃ ┃ ┗3dalgo-pseudo-initial.py         # Pseudocode used to give the general outline of the algorithm
 ┃ ┗Resources                          # Resources referred for Algorithms
 ┃ ┣RectangleBinPack.pdf
 ┃ ┣3D_Bin_Packing_by_erick_dube.pdf
 ┃ ┗master_thesis on_Bin_Packing.pdf
 ┃
 ┣Assets                          # Contains simulation result videos
 ┃
 ┣Script                          # Contains the scripts used to run the scene
 ┃ ┣zmqRemoteApi                  # Contains the ZMQ API client files
 ┃ ┣Scene_Script.py
 ┃ ┣bp3d_greedy.py
 ┃ ┗bp3d.py
 ┃ 
 ┣Simulation_Scene                                # CoppeliaSim Simulation Scenes 
 ┃ ┣EKLAVYA_PICK_PLACE.ttt
 ┃ ┣Simulation_Multiple Boxes.ttt
 ┃ ┣Simulation_four_boxes.ttt
 ┃ ┗scene_ortho_view.ttt
 ┃
 ┣include                              # Essential add-on files needed for ZMQ-API 
 ┃ ┣cbor.lua
 ┃ ┣simAddOnZmq remote API.lua
 ┃ ┗simZMQ.lua
 ┗README.md

Getting Started

Prerequisites

• Used CoppeliaSim version 4.2.0
• Tested on Windows 10 and MacOS versions 10.14.6 and 10.15.7
• Used Anaconda for Spyder IDE.

Installations

1. Clone the repo

git clone https://github.com/sagarchotalia/Pick-and-Place-Robot-Eklavya.git

2. For CoppeliaSim v4.2, Clone ZMQ API repository in the CoppeliaSim/programming folder.

3. Open the include folder of this repository. Add the simAddOnZMQ remote API.lua file to your main CoppeliaSim directory (for the add-on).

4. Then navigate to the CoppeliaSim/lua folder. Paste the scripts cbor.lua and simZMQ.lua in them.

5. If there are any files with the same name as the above, simply replace them with the files provided in this repository, otherwise the API will show an error.

6. Install OpenCV in Spyder Terminal.

Execution

• Open Spyder IDE using either the directly downloaded application or through Anaconda Navigator.
• Open the scene in Coppeliasim. Don't click on the Play icon, the script contains a function that will run the scene for you.
• Run the Scene_Script.py script

Theory and Approach

The main idea of the project is that an item comes packaged inside a box on the conveyor belt, and then gets detected by two specific cameras used for vision sensing (one for the length and breadth and the other for the height and length). Then, using this image data captured by the cameras, the three spatial dimensions of the box are determined by using image processing. After this happens, the conveyor stops and the box is picked up by the gantry components and placed at the desired position coordinates as determined by the bin packing algorithm.

In the script, first we connect with the ZeroMQ API at port 23000. Then, the object handles are accessed (this is required in order to manipulate the specific parts of the robot). After that, the simulation needs to be started and then, inside object_dimensions function OpenCV functions need to be used to detect the box dimensions as soon as they come into the frame of the camera. Once the dimensions are determined the bin packing algorithm is called and the coordinates returned are used so that the robot places the box at the desired position by using the pick_place function.

Flow of Program

Link to Project Report

Project Report

Results and Demo

Future Work

• Add the feature of rotation of boxes for more efficient packing
• Improve and test approaches to Bin Packing Algorithm
• Improve Simulation by using Suction Pad for picking boxes, Trajectory Planning

Troubleshooting

• Refer CoppeliaSim Documentation for ZMQ API.
• Check for NumPY and OpenCV installation.

Contributors

• Ayush Kaura
• Sagar Chotalia

Acknowledgements and Resources

• SRA VJTI
• Leopoldo Arnesto YouTube
• A great 3D Bin Packing Algorithm Implementation
• Erick Dube's Research Paper along with Enzo Ruiz's insightful repository
• Our seniors and mentors Dhairya Shah and Karthik Swaminathan

License

MIT License