It all started with two software developers and a tomato farmer on a west coast road trip.
The three visited farms to assess their needs and hatched a plan in an apple orchard: build a highly adaptive 3D vision AI system to automate field tasks.
Verdant, based in the San Francisco Bay Area, is developing an AI that promises versatile agricultural support in the form of a tractor device for weeding, fertilizing and spraying.
Founders Lawrence Ibarria, Gabe Sibley, and Curtis Garner—two Cruise Automation engineers and tomato grow managers—use the NVIDIA Jetson Edge AI platform and NVIDIA Metropolis SDKs like TAO Toolkit and DeepStream for this ambitious piece of farm automation.
Founded in 2018, the startup is being used commercially on carrot farms and in trials on apple, garlic, broccoli and lettuce farms in California’s Central and Imperial Valleys and Oregon.
Verdant plans to help organic farming by lowering production costs for farmers, increasing yields and supporting workers. It employs the tractor driver who is trained to manage the AI-controlled devices. The company’s Robot-as-Service model, or RaaS, allows farmers to view metrics of yield improvements and chemical cost reductions, and pay per acre for the results.
“We wanted to do something meaningful to help the environment,” said Ibarria, Verdant’s chief operating officer. “And it not only reduces costs for farmers, but also increases their yield.”
The company recently raised more than $46 million in Series A funding.
Another recent event at Verdant was the hiring of Frank Dellaert as Chief Technology Officer, known for using graphical models to solve large-scale mapping and 4D reconstruction challenges. A faculty member at Georgia Institute of Technology, Dellaert has led work at Skydio, Facebook Reality Labs and Google AI while on leave from research university.
“One of the things that struck me when I joined Verdant was how they measure performance in real time,” noted Dellaert. “It’s a promise to the grower, but also a promise to the environment. It shows if we are actually saving on all the chemicals that are being brought into the field.”
Verdant is a member of NVIDIA Inception, a free program that provides startups with technical training, go-to-market support, and consulting for AI platforms.
Companies around the world—Monarch Tractor, Bilberry, Greeneye, FarmWise, John Deere, and many others—are building the next generation of sustainable agriculture with NVIDIA Jetson AI.
Cooperation with Bolthouse Farms
Verdant is working with Bakersfield, California-based Bolthouse Farms to support the transition of its carrot growing business to regenerative farming practices. The goal is to use more sustainable farming practices, including reducing herbicide use.
Verdant begins with weeding and next expands into precision fertilizer applications for Bolthouse.
Calculating and automating Verdant has enabled Bolthouse Farms to understand how to achieve their sustainable farming goals, according to the farm’s management team.
Riding with Jetson AGX Orin
Verdant sets the Jetson AGX Orin System-on-modules in tractor cabs. The company says Orin’s powerful processing power and availability with ruggedized vendor cases make it the only choice for agricultural applications. Verdant also works with Jetson ecosystem Partners including RidgeRun, Leopard Imaging and others.
The module allows Verdant to create 3D visualizations showing crop treatments for the tractor driver. The company uses two stereo cameras for its field visualizations, for inference and to collect data in the field to train models on NVIDIA DGX Systems with NVIDIA A100 Tensor Core GPUs back at headquarters. DGX performance allows Verdant to use larger training datasets to achieve better model accuracy in inference.
“We’re showing a model of the tractor and a 3D view of each individual carrot and weed and the actions we’re taking so customers can see what the robot is seeing and doing,” Ibarria said, noting that this is all on A single AGX Orin module delivering inference at 29 frames per second in real time.
DeepStream powered Apple Vision
Verdant relies on NVIDIA DeepStream as the framework to run its core machine learning to support detection and segmentation. It also uses custom CUDA kernels to perform a number of tracking and positioning elements of its work.
Verdant’s founder and CEO, Sibley, whose post-doctoral research focused on simultaneous localization and mapping, has brought this expertise to agriculture. This is handy for presenting a logical representation of the farm, Ibarria said. “We can see things and know when and where we saw them,” he said.
That’s important for apples, he said. They can be difficult to treat as apples and branches often overlap, making it difficult to find the best way to spray. The 3D visualizations enabled by AGX Orin allow a better understanding of the occlusion and the correct spray path.
“If you see a bloom on apples, you can’t just spray them when you see them, you have to wait 48 hours,” Ibarria said. “We do that by creating a map, relocating ourselves and saying, ‘That’s the bloom, I saw it two days ago, so it’s time to spray.'”
NVIDIA TAO for 5x model production
Verdant relies on the NVIDIA TAO Toolkit for its model building pipeline. The transfer learning feature in the TAO Toolkit makes it possible to take off-the-shelf models and quickly refine them with images captured in the field. For example, it is possible to switch from detecting carrots to detecting onions within a day. It used to take about five days to build models from scratch to an acceptable level of accuracy.
“One of our goals here is to use technology like TAO and transfer learning to work very quickly in new circumstances,” said Dellaert.
While the production time for model building was reduced by five times, the company was also able to achieve 95% accuracy with its vision systems using these methods.
“Transfer learning is a big weapon in our arsenal,” he said.
