In April’s previous mobile grasping application nuclear industrial robot case sharing, I introduced the nuclear robot team from the University of Texas at Austin on the Husky platform with the help of ClearPath robot integration experts as early as 2015. Two UR5 robotic arms were created and a unique industrial-grade dual-arm system named "VaultBot" was created for the recycling and processing of nuclear materials and nuclear contamination inspection. We have been following the research progress of the team, and now they have some new research results, let us find out.

Guy Zaidner and Dr. Mitch Pryor are researchers at the University of Texas at Austin. They developed Vaultbot, a Husky-based dual-arm mobile robot designed to improve the situation in remote operation of dual-arm mobile robots in dangerous and uncertain environments Perception

Challenges of Robot Teleoperation

For dangerous and uncertain environments, using robots instead of humans seems to be a good idea. However, due to the lack of appropriate visual feedback, indirect remote control operations (remotely controlling the actions of the robot) are often very challenging. For example, video return is usually low resolution, low FPS, limited field of view, and high latency.

Remote control robots have many challenges. First, operators need to be equipped with various wearable devices, such as head-mounted displays (HMD) and force and tactile feedback devices. Although these devices can increase the amount of information that the operator can receive, the information can also confuse the operator.

Another problem is that operating the robot using only standard 2D monitors may be complicated due to the lack of perceived depth and field of view.

The last problem outlined in the study is the delay or round-trip delay between the operator and the robot, which may cause the robot's control to be unstable. Waiting times higher than 50-100 ms may make the operator think that the system is not responding and then re-execute the command, which will lead to overreaction.

All these problems are the cause of the situational awareness problem (Situational Awareness SA), which the team defines as "the perception of environmental elements and events in time or space, the understanding of their meaning, and the prediction of their future state."

Side view of one of the remote control Vaultbot arms

Rescue robot perception and planning

By creating a task-based graphical user interface (GUI) that combines SA theory and the psychological principles behind motion perception, the research team has been able to solve these common problems.

For remote navigation, the integrated screen can be divided into two levels to display the environment: the first level, using a 360° camera, the operator can explore and view the environment from the perspective of the "driver" of the spherical image. Second, by zooming out the view, the operator can see the robot on the map (a bird's-eye view). When moving the robot, the user will see the speed command that has been sent to the robot, so the user may want the robot to move more autonomously after a period of time. By switching to the operating mode, the user can see another screen that displays the image from the perspective of the grabber. The dynamic overlay that constitutes the virtual gripping plane only provides depth perception in the relevant area of interest. When the object is close enough for grabbing, the object will cover the virtual surface.

For kinematics, the researchers used "point mode" to display the relative position of the end effector and the robot base relative to the kinematic boundary to prevent singularity. The image of the gripper changes according to the rotation and translation of the gripper to achieve better spatial perception. After grasping the object, the user will see an indication of the force exerted on the object, which allows the user to adjust the contact force.

In order to solve the delay problem, the user can simultaneously see the speed command from the input device and the simulated arm aligned with the real arm moving with a small delay. 

Linear perspective view

Grabber view with 2D depth and force sensitivity

Technical composition of Vaultbot

Vaultbot is built on Husky UGV and has two integrated UR5 collaborative robot arms. The robot also has the following sensors: UM6 inertial measurement unit (IMU), SICK LMS511 2D LIDAR, NetFT force torque sensor, Intel RealSense D435 camera and two 4K Kodak PixPro cameras.

The robot runs on two synchronized ROS melodic masters, which run on different computers with Ubuntu 18.04 OS. One main controller is responsible for the entire robot except the camera, while the other main controller is completely responsible for the on-board camera. Because Vaultbot is designed to work in hazardous environments, the purpose of this setup is to provide a safe solution. If the robot computer needs to be restarted remotely due to a malfunction, the camera will remain on and continue to work.

Vaultbot is equipped with UM6 inertial measurement unit (IMU), SICK LMS511 2D lidar, NetFT force torque sensor, two integrated UR5 collaborative robot arms, Intel RealSense D435 camera and two 4K Kodak PixPro cameras

Why did they choose to develop on Husky

Researchers need a reliable mobile platform because it needs to work in dangerous environments and it must be robust so that they can add more sensors and devices, both of which HUSKY UGV has.

"The cooperation with Husky is a very valuable experience. The hardware and software are stable for a long time. The integration of ROS software and sensors has become simple." Zaidner said. In addition, Clearpath has proven itself to be a reliable and stable supplier, ensuring that it can use its commercial hardware to replicate any developed functions. Clearpath helped UT Austin develop a dual-arm mobile robotic arm from commercial components and obtained long-term support.

Widely used Vaultbot

Vaultbot has been used to support funding projects from Los Alamos National Laboratory, Savannah River National Laboratory, and multiple O&G industrial partners including Woodside. Vaultbot is also introduced in various publications related to the nuclear environment and is included in IEEE Spectrum.

As the authorized agent and the largest sales platform of many internationally renowned intelligent robot companies in China, such as Japan's Softbank Robotics Company, Denmark Universal Robots Company, Canada Clearpath Company, South Korea ROBOTIS Company, etc., Wuhan Jingtian Electric has become the most capable of technical integration and after-sales service. The strength of agents.