The examplary task for reducing electronic waste is a good chance to showcase modern solution to robotic tasks, including perception, control and motion planning. Our solution to the Robothon 2023 Challenge is built on three core principles:
(Uncertainty-aware) Interactive Imitation Learning (IIL) is a cutting-edge approach in robot manipulation that allows robots to learn from human demonstrations while taking into account uncertainty in the data and in the learned model. This approach is particularly useful in scenarios where the environment is dynamic and the robot must adapt to changes in its surroundings. It combines the strengths of reinforcement learning, imitation learning, and human guidance to teach robots complex tasks. By incorporating uncertainty into the learning process, the robot can better handle novel situations and avoid catastrophic failures. In contrast to traditional imitation learning, where the robot is simply copying the actions of the human demonstrator, IIL actively seeks feedback and adjusts its behavior accordingly. This two-way interaction between the robot and the human teacher allows for a more efficient learning process. IIL has shown promising results in a variety of tasks, including pick-and-place tasks, assembly tasks, and disassembly tasks. One key advantage of IIL is its ability to learn from human demonstrations that may be imperfect or incomplete. As robots become increasingly integrated into our daily lives, approaches like IIL will become essential for enabling safe and efficient interactions between humans and machines.
Using learning from demonstration, complex skills can be easily learned and applied o new situations. The human demonstrator moves the robot's end-effector transferring implicit knowledge of the given task.
To overcome uncertainty in the localization or the controller, we use different feebacks to increase robustness and reliability. Using a simple spiraling logic helps inserting the probe.
Safe coexistance of humans and robots can only be achieved using compliant control, i.e. a robot that is controlled using torque signals rather position or velocity signals. Using that concept, the robot can always be held back if needed.
5 minute video presentation of our approach
5 rollouts for the Robothon competition
Our philosophy to robotics in human-shared environments requires the robot to have a torque interface for each motor. That means, the robot must have torque sensors. One such robot is Franka Emika Panda. To reproduce our solution, we suggest to use the same robot, although other robots with that property can be used as well. The robot used in this challenge are
And that is it! You are ready to teach your robot any skill!
Our solution to the robothon 2023 challenge is made fully open-source to allow the community to build on top and further improve our solution. All components are stored Platonics Delft. You can find all required software components and links to their installation guides below. The first repo is the controller in Cpp and the second one is the python manipulation package. You can follow the instructions for the installation in the README.md of the two repositories.
roslaunch franka_robothon_friendly_controllers cartesian_variable_impedance_controller.launch robot_ip:=ROBOT_IP
.
Then we start the camera with
roslaunch box_localization camera.launch
.
Then we record a template with
python3 manipulation_tasks_panda/trajectory_manager/record_template.py "name_template"
.
You can now record a demonstration with:
python3 manipulation_tasks_panda/trajectory_manager/recording_trajectory.py "name skill"
.
For executing the skill you can run
python3 manipulation_tasks_panda/trajectory_manager/playback_trajectory.py "name skill"
.
To run all the skill in a row, you can modify the manipulation_tasks_panda/trajectory_manager/playall.py according to the name of your skills. You can run
python3 manipulation_tasks_panda/trajectory_manager/playall.py 0
if you don't want to have the active localizer or
python3 manipulation_tasks_panda/trajectory_manager/playall.py 1
, otherwise.
The second command requires that you also launch the localization service with
roslaunch box_localization box_localization.launch template:='name_template'
During demonstration or during execution, it is possible to give feedback to the learned skill using the computer keybord.
Gripper commands:
Camera feedback:
Haptic feedback:
The motivation of explictly labeling or disenabling the haptics and local camera feedback is because during a long trajectory the user can explictly teach the robot to use or not that skill. For example, it makes sense to have the haptic feedback only active when doing insertion tasks, such that the robot will not start spiraling when external forces are perceived but they are not due to critical insertion tasks. It is worth noticing, that if, no insertion task is perfomed, in case of large force, the robot would temporarly stop the execution of the motion, until the distrubance is gone. This increases safety when interacting with humans.
Directional feedback
This feedback is useful when, after the demonstration, the robot will, for example, not press the button hard enough. However, by pressing j, the trajectory is locally modified and when executing again, the robot will apply enough force.
At the end of every play back, the computer will ask if to overwrite the old trajectory or not. If the changes are accepted, the robot will remember the feedback in the next executions.
The additional material, such as the website's source code and
presentation can be found in the same github organizition
Platonics Delft: