Human and robot team up for high-tech fruit harvest

PROSSER, Wash.− Fall is in the air and it’s harvest time for Washington apple growers. With another bumper crop expected this season many Washington tree fruit growers dream of a day when automated technology helps bring in the harvest. Manoj Karkee, assistant professor with the Center for Precision and Automated Agricultural Systems at WSU, believes that day will soon be upon us.

Conceptual human-machine interface for collaborative fruit identification. Image credit: Manoj Karkee and Mark De Kleine.
Conceptual human-machine interface for collaborative fruit identification. Image credit: Manoj Karkee and Mark De Kleine.

Karkee and his team of WSU scientists recently won a $548,000 USDA grant to develop tree fruit harvesting technology where robot and human work side by side. “Due to the complexity of fruit identification in an orchard environment, collaboration between human and machine is very important. This is what’s unique,” Karkee said. “When the robot can’t deliver, humans will step in and vice versa.”

When apples are in clusters or obscured by leaves and branches, a robot requires complex algorithms and long computational time to identify them. Humans, on the other hand, can very quickly identify fruits in these situations. Working together in a mobile system in the field, the fruit is identified in real time faster than by human or machine alone.

Karkee will develop specialized robotic methods to harvest fruit with consideration for things like the delicacy of the fruit and the dynamics of picking fruit by hand.

To develop a prototype, Karkee and his team, which includes Karen Lewis, Changki Mo and Qin Zhang, will determine how best to detach fruit from the tree – pull, rotate, twist and pull? The team will study growth patterns of various types of apples, and record and analyze videos of hand motions taken during manual picking as well as analyze force and pressure data recorded by sensors placed on the hand. This knowledge will be transferred to the robotic hand for a highly efficient fruit removal system. A complimentary project directed by Karkee will identify materials that will best mimic the human hand in order to create a robotic hand that won’t damage the fruit.

 Sensors measure force and pressure during hand picking of apple. Photo credit: Long He
Sensors measure force and pressure during hand picking of apple. Photo credit: Long He

The cost of seasonal labor is increasing and the availability of a semi-skilled labor force continues to become more uncertain. But will growers embrace robotic fruit harvesting? “Growers are very, very interested in this technology and enthusiastically waiting for it. In three to five years we hope to have a prototype to demo in the field and in another five years be able to point to where growers can adopt the technology,” Karkee said.

Funding for the research was awarded through the National Robotics Initiative, a joint program of the National Science Foundation, the USDA National Institute for Food and Agriculture, National Institutes of Health, and National Aeronautics and Space Administration.