The picture shows us an apple harvesting robot that will automatically navigate the apple orchard and accurately pick fruit from the tree

Apple picking robot project background

Amir Degani (AD) is an assistant professor at the Technion Institute of Technology, and Avi Kahnani (AK) is the CEO and co-founder of Fresh Fruits Robotics (FFR), an Israeli robotics startup company. Together, they developed an apple picking robot that can automatically navigate in the apple orchard and accurately pick fruit from the tree.

How is the apple picking robot project realized?

AD: Since cost is very important in agriculture, we try to lower the price and find the best robot to perform a specific task. We actually do different tasks, even though they look very similar to us, such as apple picking, orange picking or peach picking. If you look at the kinematics of robots, they are actually very different. We may need different joints, different lengths, etc. Therefore, we need to collect data and model fruit trees for optimization, and we are looking for the best robot for a specific task. This is something we have been working on for several years. As part of this work, we must not only design the best robot, but also focus on the design of fruit trees and find the best tree to further simplify the robot.

Information on how the robot works and its components

AD: The FFR robotic arm has a vision system that can segment and detect apples. After finding the apple, it will provide the controller with posture information in space, and then the robotic arm will reach out to get the fruit. It has an under-actuated gripper at the end with a single motor that can grab the apple and rotate it about 90 degrees to release it from the fruit tree. The arm will not return to the original position, just retract a little, let go, and then put the apple in the container. In the laboratory, we are now focusing on the automation of the mobile robot itself. We have some mobile robots such as Warthog, Husky and Jackal. Fruit Fresh Robotics is mainly dedicated to the design of manipulators.

What kind of vision sensor is being used?

AD: On Warthog, we have an IMU, a SICK LIDAR at the front, a stereo camera on the Pant-Tilt unit, and a GPS. The arm uses Kinect for sensing, and FFR is also studying TOF sensors to provide better robustness in the field.

The robot is ready to pick an apple from the tree at Microsoft's Think Next exhibition

What would you say is the most challenging part of the automated apple picking process?

AD: I think the most difficult thing is to make cost-effective and stable equipment. I think the best way is not only to think about robots, but also to train trees to make it simpler. Now, this work has been completed in the past one or two centuries to simplify the harvesting work of mankind. In extreme cases, you will see almost flat fruit walls. They did this to make the human harvester simpler. This is even more important for robotic harvesting. Trying to get a robot to pick apples from a completely natural tree is very complicated. You probably need a robot with 6-7 degrees of freedom to do it. As far as vision is concerned, perception will be very difficult. People have tried in the past to make the whole thing too expensive for farmers to use.

By picking simpler trees (the trees have been trained and trimmed in cooperation with Fresh Fruit Robotics), you can actually use a three-degree-of-freedom robot (the simplest Cartesian robot) for picking. However, I think you can even further shape the tree into a shape that is more suitable for the robot, such as a quarter circle. This may be bad for humans, but it may be perfect for robots, and may allow us to use simple robots that only require two degrees of freedom. Therefore, enhancing the system while reducing costs is the most difficult part, for which you must also be on the tree.

How do you train trees?

AD: Training systems such as fruit walls require high-density planting while ensuring that the trunks and branches are not too thick. For this, you must use grids and other engineering support systems to support them. I want to design them best so that all energy and water can flow to the fruit, not the tree itself. It has been completed for a while now, and we are basically optimistic about it. If we want robots to pick and harvest, then for robots, simplifying trees is even more important for robots than for human work.

Can the robot autonomously navigate and patrol the apple orchard?

AD: Not at the moment. We are now doing this on all robots. At present, we are trying to conduct a full SLAM on the orchard in order to conduct patrols to harvest.

How do you compensate for the impact of the weather?

AD: In Israel, the weather is relatively mild, so the problem usually lies in the sun and wind rather than snow and rain. The main problem caused by the weather is the perception of robots, which must compensate for changes in light and object positions. To overcome this problem, FFR used a tent-like facility to cover trees and robots. If there is wind, fast closed-loop control must be used, because if the target starts to move after being sensed, the system must always track the target so that the grabber can accurately grab the target instead of where it was 10 seconds ago.

Why did you choose Warthog for this project?

AD: We have been with Husky UGV for some time. We also have some older robots from 10 years ago, for example, we modified them into semi-automatic tractors. However, none of these vehicles are strong enough to perform practical work while being easy to control. We want to fill boxes with fruits weighing more than half a ton, and we don't have robots that can do this. In addition, I want my students to learn something and apply these algorithms to real-size robots that can work in the field. We started learning ROS from TurtleBot and then moved to Husky. Then, I want something that I can scale up and actually move to the site. That is what we are doing now.

How did you integrate?

AD: Mechanical integration is very easy, and we have not fully completed the electrical integration. We have two separate controllers and two separate electrical systems. Later, since the electrical system uses more or less the same voltage, it is relatively easy to plug the electrical system into the robot. But now it is decoupled.

Can you quantify the efficiency and cost benefit of manual picking robots?

AK: The system we designed can pick about 10,000 high-quality fruits per hour-without damaging the fruits during picking. Assuming that it only works during the day, the robot can save farmers 25% of harvesting costs. It is important to understand that by replacing the end effector (fixture) and software module, the same system will be able to pick other fresh fruits. This option can harvest a variety of fruit types, which greatly improves the efficiency of the system.

The picked apples are placed in a basket on the top of Jackal UGV and handed to people in the exhibition

What is the next step for CEAR Labs?

AD: We will continue to study the theoretical part of robotic arm optimization. We are studying new ideas about the reconfigurability of the robotic arm-let the robotic arm perform different tasks, and it is easy to switch from one task to another. With the help of Warthog, we are working on autonomous navigation of the apple orchard, and we are also working on non-agricultural projects related to search and rescue. We designed a suspension system on the Grizzly and installed a stretcher on top of it. The motivation for this is to be able to automatically evacuate people injured by injuries in rough terrain.

Relevant research results display

The Husky UGV used by the robotics team at West Virginia University for the robotic plant pollination project. To watch the pollination robot in action, please click on the video link below:

The mobile grasping robot is conducting plant pollination operations：https://mp.weixin.qq.com/s/6Q8ggHykXh17J9tOFhHZag

Reference page:

https://www.ffrobotics.com/

http://www.producegrower.com/article/nasa-award-winning-robot-artificial-pollination-blackberries-west-virginia-university/