Mobile manipulators are robotic systems that can both move within its environment and interact with objects. They are increasingly being adopted in various fields such as manufacturing, logistics, healthcare and service industries. 

The Learning Systems & Robotics Lab from the University of Toronto and the Technical University of Munich have a mission to enhance the performance, safety, and autonomy of robots by enabling them to learn from past experiments and from each other. They aim to build the next generation of robots empowered by machine learning to work in human-centered environments. Their current project aims to develop a ‘waiter robot’ capable of balancing objects on a tray, navigating dynamic environments, avoiding collisions and overall being a top-tier server. 

Apart from the novelty and excitement surrounding robot servers, this innovation notably reduces errors and boosts productivity and efficiency. Their project involves controlling a robot’s ability to balance objects on a tray while navigating its surroundings and avoiding obstacles much like a traditional restaurant waiter would. This is often referred to as the ‘waiter’s problem’  by the robotics community. Their aim is to have a system that is fast enough to react to sudden changes in the environment.

“Imagine a restaurant customer suddenly backing out their chair into the waiter’s path,” says Adam Heins, PhD Candidate, University of Toronto. “Ideally, the waiter would be able to avoid colliding with the person without dropping any food or drink from their tray.”

The Waiter’s Problem

At present, most mobile manipulators navigate their surroundings at a relatively slow pace, often alternating movement between the mobile base and manipulator arm(s). Up to this point, the waiter’s problem has mostly been tackled by fixed-base arms. To make a functional, more efficient waiter robot, the team would have to mount a robot arm on a mobile base. Their project aims to resolve the waiter’s problem by accomplishing rapid, seamless full-body motion with a mobile manipulator. 

The challenge with mobile manipulation systems, unlike fixed-base robot arms, is the potential vibrations and disturbances at the robot’s end effectors while driving, making it challenging to balance objects on a tray. Additionally, controlling a mobile manipulation system is computationally more demanding than a simpler, fixed-base arm. 

“Furthermore, a fixed-base robot arm is considerably less compelling as a “waiter robot”: imagine a waiter who is stuck in one place with only their arms moving. They could not serve very many tables!”

A Five-Star Service With the Ridgeback Waiter Bot

To create an efficient waiter robot, the team required an indoor, omnidirectional platform that they could use as a mobile base. They chose the Ridgeback because of its easy to use ROS interface and capability of carrying heavy payloads. 

Clearpath Robotics was able to mount UR10 on the Ridgeback which ultimately helped hold and manipulate the objects on the tray. The team used internal joint encoders of the arm as well as a Vicon motion capture system to determine the position of the robot as well as any moving obstacles in the scene.

“Clearpath is the go-to solution for mobile research robots in Canada. We found it extremely valuable that Clearpath could integrate the UR10 arm with the Ridgeback base to make a combined mobile manipulation system. With Clearpath taking care of the hardware, we can focus on our research on the software and control aspects of the system.”

To address the vibrations and other disturbances while the robot maneuvers, the team developed a whole-body Model Predictive Controller (MPC) to control the motion of the combined mobile manipulation system comprising both the mobile base and robot arm in a unified manner. This allowed the team to generate motion to move swiftly to desired locations while simultaneously ensuring compliance with constraints, such as maintaining the balance of the objects on the tray and preventing collisions with obstacles in the environment.

Watch this video where the team showcases the Ridgeback waiter robot balancing objects on a tray with the MPC and avoiding collisions with static and dynamic obstacles: 

Precision Served: Future of Waiter Bots

Currently, the team assumes that they have accurate estimates of various parameters of balanced objects such as their masses and centres of mass. Their goal is to enable their robots to balance objects despite facing uncertainty regarding these parameters. Additionally, they aim to implement a system where the robot can learn and modify these values in real-time using data obtained from cameras and measurements from force-torque sensors. 

The team’s work was also published in IEEE Robotics and Automation letters. Click here to access the article.

The team members involved in this project consist of Angela Schoellig, Professor, University of Toronto and Adam Heins, PhD Candidate, University of Toronto. 

If you would like to learn more about the Learning Systems & Robotics Lab, you can visit their website and LinkedIn page. 

You can learn more about Ridgeback here.