To establish the Reinforcement Learning (RL) for Robotics project for the AI for Manufacturing lab, the machine learning methodology had to be developed (as discussed here), and the real-world experimental task had to be set up and tested. Setting up the experiment involved installing the robotic arm, procuring and mounting sensors, purchasing a workstation, installing a real-time Linux-Ubuntu kernel, establishing communication between each piece of equipment, creating a Gazebo model, and developing a control strategy.

The robotic system includes a 7DoF Panda robot (Franka Emika), which is connected via ROS to an over-the-shoulder Azure Kinect DK RBGD sensor (Microsoft). The workstation purchased for the project includes a Nvidia RTX 3070 GPU and an AMD Ryzen 9 3900 CPU.

For control interface testing, I set up a Gazebo simulation model of the robotic agent in an ODE physics simulation. As shown in the diagram below, the real robotic arm (or the simulated Gazebo model), is controlled by the Franka Control Interface (FCI) and libfranka.

Control commands are sent to the robotic control box (FCI) through the C++ libfranka control package. Libfranka communicates with ROS control packages through the franka_ros control interface.

The communication node graph for the ROS control system can be seen below. The two strategies for sending ROS control signals include sending direct joint commands through the robot state publisher or sending position or joint commands through the MoveIt planner server.

MoveIt is a C++ library which has pre-built robotic control packages that can be quickly and efficiently implemented for robotic control. MoveIt has a visualization package (RViz) which can be used to interact with the robotic arm and test various motion planners. The Moveit control interface is more advanced than most other planners, as it incorporates a collision detection pipeline prior to allowing action execution.

The process for controlling the robotic arm using a pre-trained RL agent which sends commands over MoveIt can be seen in the diagram below.  

In a collaboration with the Non-Destructive Testing (NDT) team in the AI for Manufacturing Lab, I had the opportunity to test the control system for the robotic arm. As shown in the video below, the robotic arm was implemented to manipulate the NDT transducer for measuring defects in an aluminum block. The test validated the tuning of the controller and proved that the robotic arm is ready for real-world RL testing.