Join the Shiny Community every month at Shiny Gatherings

Fastai for solar panel identification hero image

Using fastai to Detect Solar Panels From Orthophotos (2/3)


Welcome to the second post in this three-part ML project on using fastai to detect solar panels. Throughout the miniseries, I’ll be covering the following ML project steps:

  1. Data collection and preprocessing
  2. Training neural network model using fastai
  3. Deploying your app using streamlit

In this blog, we’ll begin building a model to automatically detect solar panels in orthophotos. To do this, we’ll focus on creating an image segmentation model using the fastai library. Although we’ll be using satellite imagery, more specifically – orthophotos, the methods presented here apply to any tasks working with images in fastai.

What is fastai?

fastai is a deep learning library created by Jeremy Howard et al. that gives practitioners both high- and low-level components to quickly create advanced PoC models. It’s an innovative tool that gives us ease of use without sacrificing flexibility or performance. Since we are working on a vision task, we’ll use fastai.vision, a part of the greater fastai ecosystem. But it is important to note that fastai provides convenient interfaces for:

  •  Vision
    • image classification, single and multi-label
    • image segmentation
    • estimating points in the image
  • Text data
    • creating of language models
  • Tabular data
    • convenient creation of embeddings
  • Collaborative filtering
    • recommender systems

Image segmentation as a machine learning task

A quick reminder from where we finished in the previous post. After preprocessing data, we have two folders images and masks. The folders contain our preprocessed patches of images (500 x 500px) as well as a fill_ratio.csv file containing the percentage of pixels occupied by solar panels in every image.

Solar Panels identified in an orthophoto

So let’s load the necessary libraries and start modeling! Note that from fastai.vision.all import * already imports pandasnumpyPath and more.

As you might recall, a lot of images (nearly 85%) don’t contain any solar panels. This is a lot of “useless” data to process. So if we can reduce the target data down to only those of value we can save a lot of time. But how do we do this?

To do so we’ll use a powerful pandas function, query. Check out query documentation, there are a lot of neat things you can do with pandas query!

Since we’re using fastai in this project, we’ll use the L class as well. This is fastai‘s answer to a container that’s most similar to Python list – numpy one-dimensional ndarray. L class supports a lot of useful functionalities like map and filter. You can read about the L class in the documentation.

Looking for effective real-time object recognition? Explore the possibilities of image classification with the YOLO object detection algorithm.

Path on the other hand, comes directly from the Python standard library. This feature introduced in Python 3.4 allows easier working with paths on various systems. Check out this great tutorial on pathlib paths. What’s important for us is that you can create paths by using slash, e.g., Path(‘.’) / ‘images’.

And the paths will be valid on every system Python is running on! Finally, paths in the csv file describe the mask files while we need paths to image files. Function map applies our lambda function to every element of nzero_fill and the second map creates Path objects from strings.

Preparing fastai dataloaders for modeling

Since we have nicely organized data, we would like to use the SegmentationDataLoaders.from_label_func function to create dataloaders. To do so, we need a function that will say which image file corresponds to which mask. All that’s left to do is to create dataloaders and preview an example batch.

Orthophoto showing solar panels, indicated by reddish/brown mask

Segmentation model training using fastai

In general, image segmentation might be a difficult machine learning task to train a model from scratch. Fortunately, we can leverage pre-trained image classification models. With fastai all we have to do is create a Learner with the unet_learner function.

Now we can run the fine_tune method and leave all training-related issues to fastai! Using Nvidia T4 single GPU, each epoch took around 20 minutes. In all, the entire training should last about an hour.

Since we are using large images, we have to create small training batches to fit them into our memory. To compensate for the small batches, we use gradient accumulation. This is very easy in fastai with its callback system.

fastai model stats

To quickly preview the model’s performance let’s run show_results. The parameter figsize allows us to make images larger, which is particularly useful if we want to view the results.

Orthophoto comparing target to prediction of solar panels

fastai model serialization

Usually, after we spend our precious resources training the model, we want to save it. It’s not unheard of that a Jupyter kernel dies unexpectedly and we have to train from scratch. If you’ve been lucky enough to escape this fate, congratulations. But trust me when I say, it’s no fun. We also plan to use this model in part 3 of the miniseries, so we really ought to save it.

Prediction results using trained fastai model

It’s always nice to see the decreasing loss values and some predictions on the validation set. For me, what’s truly impressive about machine learning is its ability for generalization. Let’s take some orthophoto images of Warsaw suburbs and see whether our model will be able to find solar panels in it!

Explore our guide to GPU-accelerated ship recognition using Keras and R

Here we’re simulating usage of the model after its deserialization. Our model returns the predicted mask as a PyTorch tensor. To visualize found solar panels, let’s create an alpha mask that will fade out parts of the image that are not panels.

Alpha mask fading non-solar panel pixels in orthophoto

Next step

In just a few lines of code, we’ve trained a PoC model for solar panel detection using fastai and performed predictions on our files. We’ve also serialized the model for future use.

Need assistance with object detection, image classification, or satellite image analysis? Contact our Computer Vision team. We build models that identify objects, defects, and patterns that human auditors miss. We can provide fast development, going from concept to a working prototype in as little as one week.

Closing notes on fastai

When working with libraries like fastai sometimes you have to do odd things like juggling filenames back and forth to prepare appropriate input for provided functions. But this is only a minor inconvenience when compared to accessing the impressive features of fastai.

Another thing to note, while we trained the model on images 500 x 500px, we’ve made predictions on images of different sizes and it just worked. But this wasn’t magic. fastai took care of everything in the background. There are many beneficial reasons to utilize fastai in your project – the best of which you won’t actively notice. We recommend you begin with fastai’s quick start and explore their tutorials to get a better grasp of what’s possible.

Written under fastai version 2.4.1.