Artificial Intelligence (AI) for Asset Mapping

We're going to focus on critical infrastructure that every single one of us uses, day in and day out. I'm talking about the roads, streets, and sidewalks that enable modern transportation and underpin almost every aspect of our lives. Often taken for granted, this kind of infrastructure represents a huge investment by every taxpayer in the country.

The concepts of a city street are pretty simple. We need something strong and smooth, but maintaining these massive systems comes with overwhelming challenges. For example, how can the city acknowledge, catalog, and prioritize the repair of the pothole you just drove through? How can a highway department do the same for missing or damaged signage and streetlights? How can the municipal works department look at a thousand miles of streets and evaluate the best return on investment for taxpayer dollars? 

Our guest today is Ben Schmidt, the CEO and co-founder of RoadBotics, a company that is focused on aiding data-driven decision making for challenges around transportation infrastructure. This conversation really changed my perspective so I'm excited to share it.

Below are a few excerpts from our conversation. Listen to the full interview! 



Let's start off with the inception of RoadBotics. What was the “aha” moment when you realized that there needs to be a better way to inspect highways and public infrastructure?


The inspiration came from smartphones – everybody's got one in their pocket and they have incredible sensors on them with a great amount of processing power. The roads, bridges, all of that public information, how do you take inventory of that and understand what's going on to make the infrastructure safer?

It was a lofty goal that ended up spinning out into our first products around pavement and pavement management, and then it just kept broadening from there into more and more public infrastructure.


Fascinating. On your website, I saw mention of evaluating the safety and surface conditions on public roads. Was that the initial focus?


That was the first focus and we still do a lot of that today. 

It began broadly with “where are the potholes” and then became more specific to the condition of the pavement, such as cracking. There are a dozen types of cracks and they all mean something.

The first problem with maintaining infrastructure is, as you well know, it’s massive and it’s hard to pay attention to. You just can't do it manually. You need tools and technology in play to make that efficient, so that's where we come in.


Just trying to map out streets of a specific neighborhood and their appropriate signage is challenging enough, let alone an entire municipality. I'm sure there are real challenges with scaling, sifting through, and maintaining that data.


Oh, absolutely. You mentioned the scale problem. Just think about the roads that are owned by an average small town. It's going to take you a few days to travel that entire network in a car.

If you want people to actually pull over and take a few notes, that could take weeks, and if you’re in a city environment, you're talking months. Especially if it's just one person. So just to get a single set of eyes on it and scribble a few notes, there's no other way to do it without new technologies.


It's not just the time investment of collecting the data, but then you need to log it and create action items. Someone needs to sort through that for prioritizing.


Totally. The other thing is, what about next year? When you look at those same roads or signs, all you've got is your memory or a few scribbled notes from the previous year. You don't have the imagery, you don't have the machine outputs of what actually was found and discovered.

You can't do a comparison and even more importantly, it might not have been you who did it last year. That record keeping is equally as critical as having the data and watching it over time and hopefully making the right decisions.


I’m sure it’s useful for establishing budgets, directing traffic, and monitoring the damage caused by big trucks. I'm guessing you see that stuff all the time.


Oh, for sure. When a new big box retailer goes in, suddenly traffic increases and so does the wear and tear on that infrastructure.

It could be new construction or even a water main break – that means you have to dig into the road or the sidewalk. So there are plenty of issues. 

Utilities have those assets too, right? You have the water systems, gas lines, then you get into fiber optic and telecom. So it’s all a massive, massive scale problem. It's really exciting to see advanced technologies making such a dent in these industries. We've been maintaining infrastructure since the dawn of society so it’s an exciting time.


Oh, for sure. And I'd love to dive into the technology that makes that possible here in a minute, but you talked about a number of different damage types.

Give us a little bit of a background on pavement and the damages.


Pavement is an aggregate, right? So it's made up of multiple materials, and I'm loosely talking about both concrete and asphalt, they have different characteristics but the damage is similar. 

So there’s wear and tear of the materials, which you can’t do anything about that. If it’s outside, it’s going to degrade over time. Traffic is a large component of it too. You’re taking thousands and thousands of pounds repeatedly slamming into the pavement. The last one is really about weather related effects. So, the freeze-thaw cycle is basically for us folks up here in the Northeast that get snow. 

When water or moisture gets under the road surface, it will damage the interior part over time. A road’s average lifespan is maybe 15 years on average. If you're lucky, it could be 30 years if you do proper maintenance.

For many communities in an average suburban town, it has maybe a hundred miles of road. Repairing one mile of the road usually costs about a million dollars but the average budget for these towns is around six-hundred thousand dollars. 

Then you remember, wait a minute, didn’t he just say that roads last for fifteen years? If they were built staggered, every year several roads need to be repaired but you have a budget for maybe half of them. That's what puts a lot of communities behind the eight ball.

Your water systems follow the same characteristics, so do your sidewalks, ADA-compliant curbs, and ramps, things like that.

They all cost money. They all degrade. They all need to be maintained. 


Exactly. We come from very different businesses, but in both cases, we're really focused on the reliability of incredibly expensive assets. The issues are degradation and optimizing budgets. 

As you're looking at the kinds of damages typically found in roads, which are the most dangerous? Should you treat those with a higher priority if you find them?


Yes, certainly the absolute worst that everybody knows are potholes. It's dangerous, it’s damaging. When you get into best managing these issues, you have to keep in mind that once you’ve seen a pothole, the road is already forgone. It needs to be fully repaired. You can patch it a little and try to keep it alive, we've all seen that and it always fails. 

The worst version is what's called alligator cracking so this gets its name because it looks kind of like alligator skin. It's little patches that sort of create this lattice work. What it actually means is that the base layer of the road is gone on that piece of pavement, so the pressure on top is starting to collapse the surface. What it means is that eventually, you're going to get a pothole. If you identify it sooner, you can intervene before it becomes a really big problem. It’s all preventative maintenance. 


Wow, I can really see a business case for that. If you can prevent it from becoming a pothole, that's worth the buy-in alone, in my opinion. 

So let’s take a step back. What’s the conventional process? What types of methods are you displacing with your technology?


Largely it is grabbing a pad of paper and pencil, driving around, and taking little notes here and there for several weeks. You still need to travel the roads in order to get the data to feed into the machine learning platforms.

So that same person now, instead of spending six weeks taking notes, spends two days collecting imagery data, video data and then spends the next five and a half weeks forming a maintenance plan.


All right. So we've talked quite a bit about your ability to assess road conditions, but what stuff have you been doing since?


We found that we were doing three things that were neat.

One was the data ingestion using smartphone cameras, kind of a novel concept of data collection and machine vision was another piece. Right? So training an algorithm how to find a pothole or find a crack. The third one is the mapping, which was very unexpected, but it's actually a tough problem, right? Which roads do you own? I took an image of a road, but which road is it? So those three components, data, AI, and mapping are where we really shined. 

Now, we can take data from basically any system and manage it. The communities, civil engineering firms, utility companies all have 360 imagery, drone footage, and photos. We can use that data and place all of it onto a single map, so you can see all the assets you own within a city.

We've begun this transition of RoadBotics as a pavement company to RoadBotics as a mapping company. 


That's fantastic. So if I understand correctly, the asset owner is now the data collector? 


It’s very flexible. You can use our smartphone app, you can get a GoPro or a FLIR camera, whatever you want, and collect your own data at whatever frequency. 

Let's say you've contracted someone else to do large scale data collection. Great. We can use that data too. We're fusing it in one single platform, so as the asset owner, you can have aggregated data and a cohesive understanding.


It seems like there's not a ceiling to this if, for example, you want to catalog your street lights or trees that could impede traffic. Is it as open-ended as it seems?


You are exactly on the right track.

There are hundreds of assets in any given community. Some you can see, some are buried. How do you create one comprehensive catalog of all those things and track it over time? You can’t reasonably do that without this type of technology. 

After you identify and map what you own, you can watch as it changes and decide the most efficient maintenance protocols.

Share this post

Subscribe to our newsletter