Drones with AI: Incorporating Intuitive UAVs into the Workflow

Drones are increasingly being incorporated into workflows to help gather high-quality data about assets. Over time, developers have needed to push drone technology to meet end users' demands for more intuitive and responsive flight, data collection, processing, and analysis.

In the "Drones and AI: Improving Workflows" webinar, Commercial UAV News connected with Yariv Geller from vHive, Toomas Välja from Hepta Airborne, Kabe Termes from Skydio, and Eric Wittner from Intel Geospatial, to discuss how AI and ML improve workflows from data collection, processing, and analysis to the final deliverable. Below is a summarized version of the webinar, so be sure to check the full version for more insights about this topic.

Danielle Gagne: AI and ML are often used interchangeably, but they have some differences. Can you provide your definition of Artificial Intelligence (AI) and Machine Learning (ML)?

Yariv Geller: vHive is the only software solution that enables enterprises to deploy autonomous hives of drones to digitize their assets and generate digital twins. We make a lot of use of AI and ML, and I think the more important thing is what these terms convey, what's behind them. The goal industries are looking for is the sense of automation, as more and more data flows through the pipes of organizations, providing them with tremendous value. But in many cases, organizations are becoming overwhelmed with the amount of data they are getting. Part of the promise of AI is in capturing data, analyzing data, automating processes, and getting to insights much faster, therefore saving the organization from drowning in data and getting to the point much quicker. There is also a notion of systems that can improve in terms of quality over time, so there's a sense of ML, which is another interesting thing we see in different industries.

Toomas Välja: AI is more of a general term, which describes a problem-solving method to solve problems that aren't achievable by conventional methods. It's something that allows using techniques such as ML, a subfield of AI, where the improvement of the algorithm or program is similar to how humans learn. We look towards solving problems and making sense of the data. No matter how we call it, whether it is AI, Deep Learning, Neural Networks, or anything else. If you're not talking to actual data scientists, then, in my experience, those terms are all used interchangeably.

Eric Wittner: AI as a general category is a machine replicating human intelligence to decide or perform a task based on a set of rules or procedures. Those rules can be human-defined, such as the phone automation system that's answering back to you when you talk, or learn like a face detection algorithm and Facebook, which is showing you all your friends. ML is a subset of AI, and it allows the AI to learn the rules themselves based on the provided data. It can improve over time with more data, which is one of the reasons I think drones and UAVs, in general, are so exciting due to the volume of data they provide. These ML algorithms can identify and understand the characteristics of things they see in this imagery, such as finding all the houses in a sort of imagery, determining which ones are damaged, and flagging the ones destroyed in response to a tsunami. Deep learning is sort of the next step where AI derives insight from the data in a more unsupervised fashion and gives you the characteristics it sees. But I think in this space, AI encompasses computer vision, spatial analysis, and spatial prediction. It's all sort of in the service of delivering decisions or delivering actions to customers.

Kabe Termes: I would echo a lot of the sentiment about the terms of AI and ML in terms of the exact science and how they interact with one another, but I like to think more from the operations side of the house. When we're thinking of ML at Skydio, we refer to that as almost a building block of AI. ML can be used in an image classifier to identify the damage, and you can use things like ultrasonic sensors to determine where an obstacle is. But for drones to leverage AI, you need a device that can see, understand the world around them. Today, more and more companies are marketing towards ML features. Still, we at Skydio are firm believers that you need to cross that threshold to get into the AI space before you can have a truly transformative experience for your pilots. This way, you can start to scale these programs and build things out, which hinges on pulling all the different pieces of ML into a truly AI-enabled drone.

When drones were introduced into commercial industries, the focus was on collecting a lot of high-quality data. We soon realized that it created a bottleneck for downstream processes. Since it took more time to ingest, process, analyze, and then produce a deliverable, any efficiencies or gains were lost in the process. Can you provide an example that demonstrates where we are solving these bottlenecks and how AI and ML are an important part of that equation?

Eric Wittner: I think we've seen this challenge repeatedly in the imagery and geospatial world over time. Each time we advance our ability to collect information, we struggle to manage, use, and make decisions from it. In the late 90s and early 2000s, the lidar datasets we got were so large that sometimes they were even challenging to process on the machine. It's really a three-part problem: the total size of the data, and we're seeing a lot of the solutions where we can manage large data in the cloud; then there's making that data usable, like having the tools to manipulate and associate images around a specific; and finally performing analysis to drive insight from that data. That's the framework. When you can manage, visualize all the data together, and associate it with real-world features to analyze and make decisions, that gives us the complete solution.

Toomas Välja: At Hepta Airborne, we had a job to inspect powerlines extending tens of thousands of kilometers. This was quite a stagnant industry where inspections were done on foot, in the forests, running away from bears, losing rubber boots in the swamps, or by helicopters which are expensive. So, we started bringing drones into it and immediately realized that the amount of work to move photos and videos between folders and manage all of that data did diminish the gains you get. To solve that, we started creating a simple web application to manage and consolidate the data to develop a structured approach, but it had nothing to do with AI at that point. Today, we're using AI to speed up manual analysis processes, and while we don't claim to have magical tools which detect everything, the people who are doing this work end up doing it 2, 3, 5 times faster than before.

When we first started collecting all this data, what was the appeal and value that people saw? Why should we be looking at AI to enhance our understanding and outcomes? How does AI close that gap between more and better?

Toomas Välja: As we try to automate infrastructure inspection procedures as much as possible, this often requires looking at each variant of an asset. Conventional methods require a person to look through the images, so we have two ways of making it more efficient: either provide tools that make this selection process faster or eliminate the human component altogether. The latter is a lot harder to achieve, but I think the former is a stepping stone to reach the latter.

Yariv Geller: If we take tower surveying as an example, the current method of capturing data is to send people climbing on top of 300 feet towers, using a measuring tape, cell phones to snap photos, and maybe a marker. It's dangerous. First, it takes a lot of time, and you are not allowed to do it with a single person, because it's working at heights. Even then, they will only take a few sparse points of data, send it back to somebody in a back office, who is going to sit with some CAD software, try to recreate what they captured, and get something interesting out of that. Drones provide a method of getting the same data in a fraction of the cost and time without endangering people and generating data products and insights through automated workflows. Replacing cumbersome and dangerous processes with fully automated ones that give you results in real-time is the primary appeal that we've seen in the industry. Drones need to become a tool; they will not scale up in an enterprise if they're relying on a drone expert to use them. The concept is to have a drone that anybody can launch whenever they want, take it out of their trunk or their backpack and just set it free.

How can drones and AI realize the goals of modernizing, fixing, and maintaining our infrastructure and building jobs around the globe? What kind of gaps or challenges do you envision if AI applications are not part of that equation?

Kabe Termes: If you look at the US today, we have about 620,000 bridges, 4 million miles of roads, 140,000 Miles railway, and something like around 150,000 cell phone towers, 200,000 miles of transmission lines, and there are only about 200,000 pilots in the US today. We're using drones to inspect some of that infrastructure, but drones need to become an asset on an operator's tool belt to keep up with the demand and really ramp the scale. Everyday operators won't need to be drone pilots, and they won't need the high degree of training and maintenance that we put on the current UAV operators. This will be critical, especially over the next ten years, when some of these programs come into play as infrastructures continue to crumble and the need for inspections ramps dramatically. That's where AI comes in. One of our customers in Japan, who did around ten bridge inspections in 2019, switched their program to leverage AI and ML-enabled Skydio drones and inspected around 700 bridges in 2021. They've reduced the amount of training time from about 100 hours to 8 hours total before they can equip these people and get them out into the field. As soon as we reduce the burden on training and the level of skill required for an operator to get out in the field, that will be the transition point.

Yariv Geller: We've had customers suffering from equipment exposed to corrosive environments that collapse if they're not taken care of. In many cases, companies are driven to action only when actual bad stuff starts happening, and we want to preempt that right. Part of the promise of having digital assets is the ability to respond much faster and more intelligently to what's going on in the field. The ability to get insights into infrastructure grows tremendously through these tools and enables a much tighter job. Sometimes, there is a tendency to look at AI and computers as a threat to the workforce because it's taking away jobs. However, the democratization of data and the ability to use drones autonomously is helping people join the digital workforce in many ways, instead of being pushed out. The more we push out technology to people and make it accessible to them, the more opportunities we create to participate in this effort.

With a growing number of AI development toolsets from companies including AWS, Google and C3, it is now easier than ever to develop AI. How will your company maintain a competitive advantage in this growing AI marketplace?

Eric Wittner: That's a tough question. You're going to maintain your competitive advantage by building your tools or improving your routines. From the Intel Geospatial perspective, we're not just a platform for our own AI and ML; we provide a place where other companies can come and host their algorithms and do their analytics. We have a third-party analytics on vegetation encroachment and transmission line assessment that live on our platform, but we do not build them. So, we recognize we're not going to build everything. There are areas where we maintain our leadership in terms of AI and ML, but we want to work with other leaders and bring to customers all the best tools they can use to drive their decision-making and drive their actions within their organizations.

To wrap everything up, where do you see the future of AI and ML in the near and long term? And how will that create value in the enterprise?

Yariv Geller: There are three areas where AI can serve the drone industry. The first is helping drones do a better job flying, not bumping into stuff, understanding where they are, and so on. The other area is in data acquisition, the ability to better plan and understand the types and location of data that needs to be captured to get the right results. And the third area of activity is in more vertical industrial applications. Now that I have the data, how do I get better insights from it? These are the three major areas of activity that I see right now and in the future. You're going to see that happening on a larger scale with more and more use cases.

Eric Wittner: Two things. First, I think insights driven by change over time apply AI and ML as we collect more and more of this information in the same locations and see what we can pick out and determine. Then, a little prediction of what we're going to see in the long term is edge deployment of AI and ML out in devices generating actionable information. Intel owns a company called Mobileye, which builds sensors for autonomous vehicles and for driving assistance, and all the geospatial information that comes out of those will push AI and ML to start moving out onto the edge.