Building Drones for Aerial AI with Flyby Robotics' Cat Orman
Cat Orman is the COO and cofounder of Flyby Robotics, which is building the first American-made medium-lift drone as a platform for edge AI.
.png)
How did Flyby get started?
We started out building drone delivery systems as students at Yale. We were originally building hardware and software on top of off-the-shelf drones for delivery on-demand, and we realized it’s incredibly difficult to develop on top of the drones that were on the market. Commercial drones have a closed architecture — they give you no access to firmware, they have very little compute, and there are a lot of restrictions on what developers can do, from both a hardware and software perspective.
So last year, we realized we needed to build a more powerful drone ourselves. We designed a drone that could lift 6 pounds of modular payload, and we designed it around an extremely powerful GPU, the NVIDIA Jetson Orin NX. We posted it on Reddit to get feedback from other drone pilots, and we were immediately overwhelmed with people wanting to buy it for themselves. We decided to go all in on manufacturing drones.
What are some of the key applications that Flyby Robotics and others are exploring?
Flyby Robotics makes medium-lift commercial drones. That means you put 4-6 pounds worth of sensors on them and fly them around something. They’re incredibly versatile, and they’re behind the scenes of a massive cross-section of American industry.
Today, these drones inspect every major asset in energy and oil and gas. They use thermal sensors to inspect cracks in the containment concrete around nuclear reactors. They fly over and around wind farms, solar panels and hydroelectric dams to detect hot spots and damage. They’re used on substations and on transmission lines to detect vegetation encroachment, where trees can grow into power lines and cause wildfires. They detect seawater corrosion on offshore oil rigs, and carry optical gas imaging sensors to find methane leaks on pipelines.
They carry heavy-duty LiDAR payloads for land surveys, and they do photogrammetry, which is when you take thousands of geotagged pictures of an asset and stitch them together into a 3D model, on large construction sites. They’re used by farmers to carry multispectral cameras that can detect disease in crops, and by mining companies to carry hyperspectral cameras to find deposits.
In the military, these drones carry high-zoom visual and infrared sensors for ISR, or intelligence, surveillance and reconnaissance. That means they look behind enemy lines to coordinate on the battlefield and find targets to direct artillery.
One of the most interesting use cases is something called DFR, or drone-as-first responder. Over 1,500 police departments across the country use these drones today. If you get mugged in Santa Monica, which is where I live, the first police response on the scene will actually be a drone. They take off from the roof of the police department, and fly in a matter of seconds or minutes to the location of the incident, and look at it with a high-zoom visual or thermal camera. We actually visited the SMPD drone program and they told us about an incident where they received a 911 call about a teenager who had a gun, and because they had a drone, they were able to use the zoom camera and see that it was just a lighter that just looked like a gun. So they were able to arrive on the scene knowing there was no threat, and avoid a potentially escalating situation.
What’s the unique potential of high on-edge compute?
Our drones are built around an extremely powerful GPU, a NVIDIA Jetson Orin NX. That GPU is capable of 100T computations per second. That’s so much math you could literally mine crypto while the drone is in flight. Every piece of sensor data — everything the drone sees — and every single piece of telemetry data — the GPS coordinates, pitch, yaw, roll, et cetera — all talk directly to the GPU. The GPU can have full control over the flight controller of the drone, and full control over the gimbal and the camera. It’s incredibly difficult to design a drone around such a powerful GPU and build out the software side to support it, and what we’ve done is unique to Flyby.
That means developers can write applications on our drone that tell the drone how to make decisions in-flight, autonomously. The drone can take in information on what it sees, and use that information to change direction, navigate, look left, look right, zoom in, or move the sensor around.
One of the most compelling use cases for this is in GPS-denied navigation. In contested environments, where the drone’s communication with satellites is constantly jammed, drones have no way of knowing where they are. Our partner Palantir Technologies has built a capability on top of our drones for APNT, or assured positioning, navigation and timing, which allows the drone to look down and navigate by comparing what it sees to a map of reference imagery. We hope to use this capability to save lives in Ukraine.
What’s the deal with American-made drones?
Our mission is to reshore America’s drone fleet. Today, around 80% of the drones in the United States are made by Chinese companies. That’s a scary situation — that potentially gives a foreign adversary access to sensitive data on our critical infrastructure vulnerabilities. These drones literally make API calls to Beijing every time they take off. Several states have banned these Chinese drones for state-funded applications, like public safety or civil infrastructure inspection, and customers are increasingly insisting that pilots don’t use Chinese drones on their assets.
But that leaves American drone pilots in a difficult spot, because American inspection-grade drones on the market are much more expensive, they don’t have the software capabilities or even quality things like IP rating that Chinese drones do, they have a long list of user experience complaints, and there are very few of them to begin with.
We’re changing that. We’re able to price the F-11 at only $17k, compared to $14k for the comparable Chinese-made product. It’s the first affordable American option for our customers.
What were some of the major challenges or hurdles that Flyby Robotics had to overcome to make their drone-as-AI platform vision a reality?
Everything about designing a drone around a very powerful GPU is hard. It’s not as simple as just adding the chip in, you have to build the entire drone around it. We went through several iterative cycles on everything from thermal optimization to weight optimization.
One particularly interesting pain point we solved — some of our most interesting customers are building custom sensors to use on our drone. But to attach a custom physical payload, you have custom hardware needs — access to specific ports on the Jetson, for example, or different mounting hardpoints. We couldn’t make one design that would fit every partner. To accommodate all of the incredible things developers are building on our platform, we were able to introduce a custom housing option, where the top or the bottom of the housing can be 3D printed to customer specifications. It’s a completely new degree of modularity.
Looking to the future, how do you envision the world being impacted as drone-based AI platforms become more widespread and commonplace across different industries and applications?
Drones are the most natural platform through which AI will interact with the physical world. As unmanned systems get adopted more broadly, there will be nothing they can’t do — they’ll be capable of analyzing more complex data to make decisions in flight, they’ll have smarter obstacle avoidance, and they’ll coordinate with each other in real time for swarm action. Drones will be a force in every major American industry. They will improve lives at home and save lives on the battlefield.
a digital agency built for the future.
Haus Biographics is a digital agency in NYC that specializes in marketing and communications for deep tech organizations. Check out our website, follow us on Twitter, or say hello@hausb.io
