Harvesting blackberries is complicated, with or without humans being involved in the process.

The fruits must be picked quickly to avoid spoiling and becoming a sweet snack for hungry wildlife. Unlike other crops, the delicate structure of the fruit means farmers can't use heavy machinery and must rely on humans to gently pick the berries.

“Blackberries are a soft, fragile fruit that needs to be hand-picked at the shiny, black stage of ripeness,” said Dr. Renee Threlfall, a food scientist at the University of Arkansas.

For blackberry farmers and other specialty crop producers, a shrinking workforce combined with rapidly rising labor costs is making it more difficult and expensive to grow fresh fruits. Researchers at the University of Arkansas and Georgia Tech are hoping that an autonomous, robotic hand could be the answer.

Threlfall and others are in the process of creating a fully autonomous harvesting system that is delicate enough to pick fruits. Once completed, the technology could help farmers in the often overlooked sector of specialty crops expand their production while keeping costs down.

“One of the major challenges with fresh-market blackberries is a shortage of human labor for harvesting them,” Threlfall explained. “Finding a solution for labor issues could increase U.S. production.”

A hands-on approach

Threlfall teamed up with researchers in Arkansas and Georgia in 2020, including Professor Yue Chen, an assistant professor in the department of biomedical engineering at Georgia Tech and Emory University.

Their goal was to create an autonomous robot that can pick berries as gently and accurately as a human. The team recently received grant funding from the National Institute of Food and Agriculture's Cyber-Physical Systems program in collaboration with the National Science Foundation.

Chen specializes in biomedical engineering with a focus on minimally invasive robotic surgery. He found that picking delicate fruit is similar to performing sensitive surgery.

“Both rely on accurate motion planning, robot control, navigation, and end effector manipulation," he said. "So, I extended the medical robotics principles to agriculture.”

The research process began with a few fundamental questions: How many fingers does a human use to pick blackberries? How much force does it take, and what level of pressure will damage the fruit?

To find answers, the team connected biometric sensors to silicone finger sleeves that tracked force and movement as graduate students picked thousands of blackberries. The data revealed that humans use just three fingers when picking and apply approximately 0.5 newtons of force.

Utilizing these findings, Chen and his graduate student, Anthony Gunderman, designed a robotic system that picks berries gently enough to prevent damage 95% of the time, a rate higher than most human pickers achieve. This three-fingered "hand" uses a guitar string as a tendon to grab, stabilize and harvest each berry.

Giving the hand a brain

The next challenge was teaching the robot which berries were ready for harvest. In other words, they needed to give it artificial intelligence. That’s something specialty crop producers are only starting to grapple with, and there’s no simple solution.

A form of AI called computer vision can analyze the branch structures to determine what to pick, but environmental hazards and the amount of skill needed to operate robots is still hampering a widespread transition to automation among specialty crop producers.

“An open-ended environment like berry bushes complicates the process of using cameras to perceive fruit and branches," said As Alan Fern, professor of computer science at Oregon State University and co-principal investigator at the AgAID Institute, a collaborative effort to solve complex agricultural problems using AI. "And then, controlling robot manipulators requires high degrees of dexterity with minimal room for error.”

For Chen, the primary challenge was obtaining a database comprehensive enough to teach the robot what ripe berries looked like and where to find them.

“In 2019, Gunderman took over a thousand pictures of blackberries at various times throughout the day. In each image, there were over 30 blackberries,” Chen explains. “This allows us to use over 30,000 segmented berries to teach an algorithm what a ripe berry looks like.”

While obtaining quality images and segmenting them proved to be time-consuming, it gives the robot enormous versatility in the long run.

“Once these images are obtained, and the neural network has learned what a ‘ripe’ specialty crop is, it can ideally be used in various scenarios," Chen said, "similar to how a human can identify a blackberry, whether in Arkansas or California.”

Chen and Gunderman created a test model of the robot in 2022 that utilized a new version of the soft robotic gripper and a stereotactic camera powered by their algorithm. It identifies ripe berries through an onboard, infrared sensor that assesses berry locations and ripeness status.

“This process can be nearly real-time,” says Chen. “Then, using our previous modeling methodology, we can place the gripper and harvest blackberries.”

Hope for an uncertain future

Beyond combating labor shortages, AI-powered autonomous robots may provide an advantage for specialty growers grappling with climate change. Increasingly erratic weather patterns could further shrink harvest windows. But unlike people, robots don’t need breaks and can work at night and even through rainstorms.

Chen believes that augmenting the labor supply through robotics will significantly improve production and increase energy efficiency for a lower carbon footprint for specialty crops.

“By maximizing productivity and output, we can reduce wasted effort that can contribute to climate change," he said. "In other words, we can produce more with less.”

Despite the challenges climate change will bring, Threlfall believes blackberries themselves will be resilient.

“There are enough locations, cultivars, and trellising/production systems that I don’t think climate change would have a major impact," she said. "In fact, it might expand production in U.S. regions where low temperatures are a challenge.”

Perhaps all the more reason to bring robotic help to the harvest.