Why scaling robotics is challenging in agriculture?
Ag Robotics is difficult af
One of my favorite fictional robots is Marvin, the paranoid android from The Hitchhiker’s Guide to the Galaxy by Douglas Adams.
Marvin is incredibly smart, has a "brain the size of a planet" capable of extremely complex mathematics, can solve difficult problems and operate high-tech devices.
Despite being so smart, Marvin is typically made to perform menial tasks and labor such as escorting people, opening doors, picking up pieces of paper, and other tasks well beneath his skills. Due to this, Marvin is extremely frustrated and depressed.
If we compare the current state of robotics to Marvin (albeit a fictional robot), it reminds one of the 2013 lament by Peter Theil
“They promised us flying cars, but all we got was 140 characters.” (Tweets back then had a 140 character limit)
—-
Strawberry picking (if you are going to pick a basket or two) can be a fun activity. The process of picking strawberries by hand, with your kids, can be a fun activity. The weather is great, you are out with your family, and you get to eat fresh strawberries, which you picked moments ago.
But picking strawberries at scale, and picking them consistently requires practice and skill. You have to judge if the strawberry is at the right ripeness for picking, grab the stem, twist it, and put the strawberry in your basket, without bruising or damaging the strawberry. Human beings are good at following the process to find strawberries with the right ripeness, grab the stem, twist it, and put it in a container.
Try to build a robot which can do the same job with human or better efficiency, and it is difficult af.
We desire Marvin, and we get a slow version, compared to Marvin or a human strawberry picker.
—
PIcking strawberries using a robot is a very specific application. It requires hardware to “see” strawberries, software logic to decide which strawberry to pick, to move an arm to the right spot in 3D space to cut the stem of the strawberry, to position the basket correctly, and to cut the stem. The robot has to do it fast enough to make economic sense.
It is expensive and difficult to build a hardware and software system which can meet strawberry picking requirements.
Tortuga has developed a bot to save on labor for picking hydroponic strawberries. PIcking strawberries using humans is an expensive activity, as growers have to pay upto $ 35,000 per year in California (UC Davis study), which amounts to 40% of the total cost per acre for a strawberry operation.
Tortuga sells a subscription model with a flat cost per robot picked box of strawberries at a rate comparable to human worker wages. Today the Tortuga robot goes as fast as a slow human, but it could get to human speed in the near future. The biggest bottleneck today is skill sets required to service and operate the robots, manage their downtime etc.
Now take the same agriculture robot to pick apples, and it will be more depressed and frustrated than Marvin from Hitchhikers. The strawberry robot has no chance to pick an apple as apples grow on trees.
—
The example above highlights some of the challenges with the development and adoption of agriculture robotics technology.
Building an application specific robot is expensive and hard, but the application of the robot can be limited due to the diversity of applications needed in agriculture. This limits the size of the business of application specific robots. Due to this, it is difficult to raise VC funding as the total addressable market of the application is limited.
In a recent Agtech So What? Podcast by Sarah Nolet covered the three categories of agriculture robotics autonomy. The three categories mentioned were:
Brand new equipment or after-market attachments
Brand new equipment for row crop or aftermarket attachments for existing equipment to satisfy certain use cases like see & spray etc. This space is currently dominated with investments from incumbent OEMs. For example, investments made by Deere in See & Spray, acquisition of Raven, and Blue River.
Application specific agriculture robots built by startups
Startups building application specific agriculture robots have struggled.
For example, Abundant Robotics (mentioned in the podcast), a Hayward, California based agriculture robotics company was founded in 2016. It focused on robots picking apples. The company was shut down in 2021, as it was unable to develop the market traction necessary to support its business during the pandemic. The company raised a total of $ 12 million over 5 years.
Abundant said it has a “host of IP, including a large body of vacuum manipulation patents (and patent applications), a patented sensory system to allow the vacuum to navigate obstruction, a patented world-class vision system for identifying fruits and their quality, and several software patents for the machine’s automated operations (including a solution to solve for “doubles” that enables the machine to pick multiple fruit at once.
According to the company, they excelled with its top-end vision system, which accurately and quickly identifies optimal routes for the picking arm through voxel mapping (3D imaging) and, in conjunction with hardware, assesses each apple’s ripeness and quality before picking.
By using an expansive collection of imaging data and software, the vision system is able to detect not only which apples are ripe, but the specific ripeness level of each apple.
The apple harvester was able to harvest one apple every two seconds. The company estimated its machine could reach between 50-90% of fruit on trees. The system is designed to augment human labor and could allegedly pick apples every two seconds. Abundant said it was targeting a pick rate of 1.5 seconds for the commercialized version of the robot.
The company planned to broaden the type of fruit it picked in the future.
The company’s product expansion strategy (in their liquidation memo) was highlighted as below. The strategy called for solving the apple harvesting problem to a certain degree, and then move on to other tree fruits (for harvesting) and then other operations like thinning, pruning, and spraying.
Unfortunately the company could not make it past apples, as harvesting apples using robots and making it commercially viable is an extremely hard problem.
Robot platforms
Robot platforms provide generic robotic capabilities for hardware and software, without building application specific capabilities. Building robot platforms is equally hard, as your customer is not the end user of a particular application but the application developer for that application.
Platforms are incredibly powerful as they provide the tools to rapidly build other applications. Imagine how difficult (and inefficient) it would be, if to make accounting software, you had to design and build computer hardware, and an operating system to run on the hardware, before you got to building your accounting software.
There is not a plethora of sophisticated platforms and tools to choose from, especially in agriculture. According to the podcast guest, Andrew Bates (CEO),SwarmFarm Robotics is building a platform (a set of tools and capabilities) to help application developers focus on building their application.
So what does a robot technology stack and operating environmentlook like?
Robot operating environment
Let us begin with a simple schematic diagram of the hardware and software for a robot, and how it interacts with the environment.
Schematic by Rhishi Pethe
The sensor and the actuator are the hardware components of the robot, For reference, a sensor could be a camera or a vision system (in the strawberry and apple examples above). The actuator is the hardware component performing an action (in the strawberry example, it is the robotic arm with scissors to cut the stem of the strawberry).
The Sense, Plan, and Act are software components of the robotic system. The Sense component in our apple and strawberry example determines the ripeness of the apple or berry, and other attributes like distance, angle, obstructions etc. The Plan component creates a plan based on Sense whether to pick the apple or berry or not. The Act component acts on the plan (pick the apple or berry, and move on to the next fruit or move on to the next fruit).
Robot technology stack
There is another way to look at the technology stack for a robot. This is an overly simplified version, but the schematic will be helpful to understand how to think about robot platforms from a technology standpoint.
The schematic shows hardware at the bottom of the stack, with software layered on top of it. For the software component, as you go from the robot operating system up to robot applications, the capabilities become more specific to the application at hand (for example, picking fruits).
Schematic by Rhishi Pethe
The first robot specific layer is the robot base libraries & services layer, as it contains functionality independent from concrete robot applications. It provides generic capabilities like path planning, navigation, and localization.
The top most layer is the most application specific. This is the layer that defines what the robot is actually doing, i.e. picking a strawberry or an apple.
Building a set of generic tools and generic hardware like robot arms, cameras, etc. and having them work together for a set of basic capabilities is not trivial. It is harder to build a business to begin with, as you need a longer cycle of investment, to build the generic tools. Adoption can be harder as you are not solving an end application problem (for example, picking apples), but have to find application developers and help them accelerate their technology and innovation process.
I had illustrated a similar point last year in edition 83 (picks and shovels), between application and tools, and how they create value in the longer term.
As you go from left to right on the X-axis, the offering is more like a tool, rather than an application. As you go from bottom to top on the Y-axis, the value per unit (or customer) is higher, as you control the user experience and provide specific value, though the number of customers is lower.
We see similar development in the field of robotics, especially agriculture robotics.
For example, last year Raven acquired DOT Technology Corp. (CNH acquired Raven late in 2021)
DOT is the developer of the “Dot Power Platform,” an autonomous power unit that has the ability to pair with a wide range of implements which are critical to agricultural production and will offer a variety of labor saving solutions to farmers and Ag Retailers alike. Fusing DOT, and the recently acquired Smart Ag® autonomous perception and path planning technology, with our core technology platforms in guidance, steering, and machine control enables Raven to deliver revolutionary autonomous solutions and greatly accelerate our long-term growth in Applied Technology.
DOT created an autonomous power unit, and abstracted it out to work with a range of implements. This could accelerate innovation on the implement side, as the implement developer can focus on the end application rather than spending their time on an “autonomous power unit.”
A more recent example on the software side is a Y-combinator startup which came out stealth mode just a few days back. Polymath Robotics has launched plug-and-play autonomous software for any industrial vehicle.
“Robots used in warehouses, mining, agriculture and other industrial environments have hyper-specific applications that are structured and are often repeated thousands of times. In other words, the farmer in Iowa, the yard truck operator in Florida and the e-commerce giant with 100 warehouses spread throughout the country have specific needs that no one else does.”
Polymath Robotics have developed, (emphasis is from me)
A plug-and-play software platform and an accompanying SDK that allows companies to quickly and cost efficiently automate industrial vehicles. Think of it as SaaS for industrial robotics.
The Polymath Robotics software platform lets another startup, warehouse owner, farmer or mining company skip the often long process of building out autonomy, a safety layer and front-end app.
(Note: The article mentions a farmer, but it will be another technology company who will use the software platform to build tools for farmers)
The video below shows how you can use the Polymath Robotics platform, create simulations, and then push the code to work in actual physical environments. (I am oversimplifying this, but the concept is still true)
One of the investors in Polymath Robotics said,
“All sorts of business come to Oracle for hardware and software solutions for their business, and with minor configurations can plug it into their business. If you project forward 10 to 15 years where can this end up its an enormously audacious goal, but I think it can also continue to provide value all along the way.”
(Oracle was an early pioneer in relational database technologies, and continues to be an important player in the technology industry.)
Will companies like SwarmFarm Robotics, Polymath Robotics, Sabanto, and others move a lot of the smarts in smart machines to software? Would it be possible to build cheaper and smaller hardware, powered by really smart software? I had explored this question in edition 89. (Android of agriculture),
“Majors like Deere can follow an Apple approach with high end hardware, and software under their brand. They will service the high end large scale row crop farming in the developed world.
Is there space for a generic sub-$10K tractor, where the hardware is more or less a commodity, and the software on it is what makes it interesting and more useful? There are many types of operations depending on crop type, and so there might be different hardware configurations.
Could some manufacturer provide hardware in large volumes at low cost? Can a strong software and technology provider provide an equipment operating system, with room for application developers?”
I am personally excited about the development of software and hardware tools, which can potentially accelerate innovation in agriculture applications of robotics, and create value.