The main hardware of the program includes: 1 UR10e manipulator, a Haption Virtuose™ 6D six-degree-of-freedom force feedback manipulator. The URcap plug-in allows the two to be perfectly combined, combined with the teleoperation program to achieve remote master-slave teleoperation.

【Experimental teaching goal 】

1. PROPOSAL DESCRIPTION

2.1 General introduction

The main hardware of the program includes: 1 UR10e manipulator, and a Haption Virtuose™ 6D six-degree-of-freedom force feedback manipulator. The URcap plug-in allows the two to be perfectly combined, combined with the teleoperation program to achieve remote master-slave teleoperation.

TeleRobotics EXtender (TREX) is a stress feedback remote operation application. It is an innovative solution suitable for dangerous, difficult or other restricted environments, which can provide protection for workers and avoid them from any risks. Operators can use TREX to control remote UR e series robots through the Haption arm. The UR robot can truly reproduce the movement of the main arm in real time. The operator can feel the power of the robot and the force exerted by the robot through the main arm with stress feedback. Stress perception can significantly reduce the risk of damage to the environment or the robot itself. By using TREX, operators can apply their know-how and manual experience to UR robots without any offline programming.

Virtuose 6D is a force feedback system with 6 degrees of freedom, specially designed for virtual reality working environment. Due to its large working space and high bearing capacity, it can be interactively simulated with CAD models.

2.2 Detailed design 

Ø System design

The use of the main arm with force feedback allows the operator to feel the power of the robot as well as the force exerted by the robot. The feeling of force is essential to reduce the risk of damaging the environment or the robot itself. The UR robot faithfully reproduces the movement of the main arm as if it were an extension of the operator.

Ø Installation platform

Haption size: The base is 40cm wide and 30cm long. When fully deployed, the height is 75cm and 12kg. It can be carried by one person without any assistance. The working space is 1330 x 575 x 1020 mm. The installation platform can meet the above requirements.

Ø Use

The user works comfortably and retains all manual capabilities without being exposed to dangerous or restrictive environments. Thanks to the main arm, the user can feel the power of the robot, the weight of the object being lifted, and possible collisions. A set of buttons allows it to change the intensity felt and the speed of the action.

Ø Control interface

2.3 Software solution

The software package is based on UR's PolyScope teaching programming software. PolyScope is a graphical user interface (GUI). You can operate the robot arm and control box, execute the robot program, and create a new program easily. The teach pendant plug-in developed based on URcap allows the TREX teleoperation system to be implanted into the UR5e teach pendant panel with one key to develop teleoperation functions.

Program case

✓ User overview 

International: Massachusetts Institute of Technology, University of Waterloo, Carnegie Mellon University, Georgia Institute of Technology...

Domestic: Tsinghua University, Peking University, Hong Kong Polytechnic University, Wuhan University, Huazhong University of Science and Technology, Shanghai Jiaotong University, Southeast University, Central South University, Shenyang Institute of Automation, Chinese Academy of Sciences, Guangdong University of Technology...

2.4 Universal connection example

Host: Haption force feedback machinery, computer, monitor, communication converter;

Slave machine: UR5e robot arm and control cabinet, camera.

Connect as shown in the figure below to form a master-slave robotic arm control system with force feedback.

TREX系统连接示意图

3. APPLICATION NOTICE AND RESULT 

3.1 Plan description 

Simple and easy-to-use robot master-slave remote operation system. We provide standard solutions, which eliminates the need for additional expensive technical support. With TREX, operators can apply their know-how and manual expertise through UR robots without any offline programming.

3.2 Program advantages and results

Ø Advantage

(1) Reduce the risk of users

(2) Precisely control gestures and enhance flexibility

(3) Utilize users’ manual skills and knowledge

(4) Let humans make decisions

Telerobotics Extender (TREX) is a forced feedback remote operation application. It provides innovative solutions for all situations where people work in dangerous, difficult or otherwise constrained environments, enabling them to perform tasks without facing any risk. Using TREX, the operator can control the remote UR e series robot through the Haption arm. All products include complete solutions: monitors, processing software, computers, cameras and necessary cables, etc.

【Main instruments and equipment 】

The main hardware of the program includes: 1 UR10e manipulator, and a Haption Virtuose™ 6D six-degree-of-freedom force feedback manipulator. The URcap plug-in allows the two to be perfectly combined, combined with the teleoperation program to achieve remote master-slave teleoperation. 

【Experimental project setup 】

Related papers catalog

Haption's paper

1) Force control and remote operation realization method

Gosselin F, Riwan A. Design of Virtuose 3D: A new haptic interface for teleoperation and virtual reality[C]//Proc. ICAR. 2001: 205-212.

2) vr capture

Stergiopoulos P, Fuchs P, Laurgeau C. Design of a 2-finger hand exoskeleton for VR grasping simulation[J]. Eurohaptics, Dublin, Ireland, 2003: 80-93.

3) Advantages of mechanical reverse drive in force control medical applications

Perret J, Vercruysse P. Advantages of mechanical backdrivability for medical applications of force control[C]//Workshop on Computer/Robot Assisted Surgery (CRAS). 2014.

4) Master-slave operating system

Goubot J M, Garrec P. STeP: an innovative teleoperation system for decommissioning operations[C]. 2003.

5) Expand the working space of the 6-DOF tactile device

Gosselin F, Andriot C, Bergez F, et al. Widening 6-DOF haptic devices workspace with an additional degree of freedom[C]//Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07) . IEEE, 2007: 452-457.

6) Innovative transparent exoskeleton for upper limbs

Garrec P, Friconneau J P, Measson Y, et al. ABLE, an innovative transparent exoskeleton for the upper-limb[C]//2008 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2008: 1483-1488.

Control

1) Design of mobile robot motion controller

Röhrig C, Heß D, Künemund F. Motion controller design for a mecanum wheeled mobile manipulator[C]//2017 IEEE Conference on Control Technology and Applications (CCTA). IEEE, 2017: 444-449.

2) Real-time motion control for upper limb tracking

Omarali B, Taunyazov T, Bukeyev A, et al. Real-Time Predictive Control of an UR5 Robotic Arm Through Human Upper Limb Motion Tracking[C]//Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction . 2017: 237-238.

3) Use UR to imitate human writing

Miatliuk K, Wolniakowski A, Diaz M, et al. Universal robot employment to mimic human writing[C]//2019 20th International Carpathian Control Conference (ICCC). IEEE, 2019: 1-5.

Kinematics and dynamics

1) Calibration of UR5 manipulator based on kinematics model

Liang B, Cheng Y, Zhu X, et al. Calibration of UR5 manipulator based on kinematic models[C]//2018 chinese control and decision conference (CCDC). IEEE, 2018: 3552-3557.

2) UR5 dynamic modeling

Kebria P M, Al-Wais S, Abdi H, et al. Kinematic and dynamic modelling of UR5 manipulator[C]//2016 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, 2016: 004229-004234.

3) Based on singularity-robust multitask priority inverse kinematics framework

Moe S, Antonelli G, Pettersen KY, et al. Experimental results for set-based control within the singularity-robust multiple task-priority inverse kinematics framework[C]//2015 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 2015: 1233-1239.

4) New method of inverse kinematics solution

Moe S, Antonelli G, Teel AR, et al. Set-based tasks within the singularity-robust multiple task-priority inverse kinematics framework: General formulation, stability analysis, and experimental results[J]. Frontiers in Robotics and AI, 2016, 3: 16.

.Vision, calibration, perception

1) Image vision automatic calibration system based on UR

Jian B L, Tsai C S, Kuo Y C, et al. An image vision and automatic calibration system for universal robots[J]. Journal of Low Frequency Noise, Vibration and Active Control, 2019: 1461348419874925.

2) New planning method based on UR

Tatsubori M, Munawar A, Moriyama T. Design and Implementation of Linked Planning Domain Definition Language[J]. arXiv preprint arXiv:1912.07834, 2019.

3) Hand-eye calibration

Li J, Li X, Dun A, et al. Hand-eye calibration for flexible manipulator[C]//Journal of Physics: Conference Series. IOP Publishing, 2019, 1187(3): 032097.

4) Adaptive learning based on UR

Inoue T, Chaudhury S, De Magistris G, et al. Transfer learning from synthetic to real images using variational autoencoders for robotic applications[J]. arXiv preprint arXiv:1709.06762, 2017.

5) Visual tracking based on UR

Ramachandruni K, Jaiswal S, Shah S V. Vision-based control of UR5 robot to track a moving object under occlusion using Adaptive Kalman Filter[M]//Proceedings of the Advances in Robotics 2019. 2019: 1-6.

Planning

1) Automatic adjustment and configuration of path planning algorithm

Burger R, Bharatheesha M, van Eert M, et al. Automated tuning and configuration of path planning algorithms[C]//2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2017: 4371-4376.

2) Evaluation method of robot task planning

Paxton C, Jonathan F, Hundt A, et al. Evaluating methods for end-user creation of robot task plans[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018: 6086-6092 .

3) Predictive control of deep learning for human motion tracking

Agravante D J, De Magistris G, Munawar A, et al. Deep learning with predictive control for human motion tracking[J]. arXiv preprint arXiv:1808.02200, 2018.

4) Time optimal planning based on UR collaborative robot

Zhenyi C. Joint Trajectory Time Optimization of Cobot Based on Particle Swarm Optimization[C]//IOP Conference Series: Materials Science and Engineering. IOP Publishing, 2019, 616(1): 012015.

5) Multi-manipulator trajectory planning algorithm

Tavares P, Lima J, Costa P, et al. Multiple manipulators path planning using double A[J]. Industrial Robot: An International Journal, 2016.

Medical treatment

1）基于UR的康复机器人

Kyrkjebø E, Laastad M J, Stavdahl Ø. Feasibility of the UR5 Industrial Robot for Robotic Rehabilitation of the Upper Limbs After Stroke[C]//2018 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2018: 1-6.