Detect objects with a deep learning pretrained model

Coconuts are an important agricultural production on the island of Tongatapu in Oceania. Plantations have thousands of trees, and inventorying each of these trees by hand would require a lot of time and resources. Deep learning is a powerful approach to detect objects automatically in imagery, but training a deep learning model from scratch can be a time-consuming process. An alternative is to use a model that was already trained for you.

In this tutorial, you will retrieve a pretrained model from the ArcGIS Living Atlas of the World collection and apply it to high-resolution imagery to efficiently detect the palm trees present throughout the landscape in ArcGIS Pro. The resulting feature layer will be useful to estimate the number of palm trees throughout the region, monitor farming practices, or assess upcoming production levels.

This tutorial was last tested on November 10, 2023, with ArcGIS Pro 3.2. If you're using a different version of ArcGIS Pro, you may encounter different functionality and results.

Requirements

• ArcGIS Pro (see options for software access)
• ArcGIS Image Analyst
• Deep Learning Libraries for ArcGIS Pro
• Recommended: NVIDIA GPU with a minimum of 4 GB of dedicated memory.
• For more guidance, visit the Get ready for deep learning in ArcGIS Pro tutorial.

Detect objects with a deep learning pretrained model

Set up the project and explore the data

To get started, you'll download a project that contains all the data for this tutorial and open it in ArcGIS Pro.

1. Download the Palm_Tree_Detection package.

   A file named Palm_Tree_Detection.ppkx is downloaded to your computer.

   Note:

   A .ppkx file is an ArcGIS Pro project package and may contain maps, data, and other files that you can open in ArcGIS Pro. Learn more about managing .ppkx files in A guide to ArcGIS Pro project packages (.ppkx files).

2. Locate the downloaded file on your computer. Double-click Palm_Tree_Detection.ppkx to open it in ArcGIS Pro. If prompted, sign in with your ArcGIS account.

   Note:

   If you don't have access to ArcGIS Pro or an ArcGIS organizational account, see options for software access.

   A map appears, centered on the village of Kolovai on the Tongan island of Tongatapu. An image layer displays on top of the topographic basemap.

   

3. Zoom in and pan to examine the imagery.

   Observe the numerous palm trees throughout the landscape.

The imagery was captured by a drone and orthorectified to remove any distortions. It is high resolution—each pixel represents about 9 by 9 centimeters on the ground—and shows palm trees and other features quite clearly. It is in the TIFF format with three bands: red, green, and blue, which together form a natural color picture.

Identifying all the palm trees in this image manually would be very time consuming. Instead, you’ll use deep learning to detect them automatically.

Detect palm trees using a pretrained deep learning model

You want to use deep learning to detect palm trees from the imagery. If you don't already have a deep learning model available, this first requires training a model from scratch, feeding it large numbers of examples to show the model what a palm tree is. High-performing models can require being exposed to tens of thousands of examples. An alternative is to use a model that was already trained for you. You will retrieve such a model and apply it to your imagery.

Note:

Using the deep learning tools in ArcGIS Pro requires that you have the correct deep learning libraries installed on your computer. If you do not have these files installed, save your project, close ArcGIS Pro, and follow the steps delineated in the Get ready for deep learning in ArcGIS Pro instructions. In these instructions, you will also learn how to check whether your computer hardware and software are able to run deep learning workflows and other useful tips. Once done, you can reopen your project and continue with the tutorial.

First, you’ll open the Detect Objects Using Deep Learning tool.

1. On the ribbon, on the View tab, in the Windows group, click Geoprocessing.

   

2. In the Geoprocessing pane, in the search box, type Detect Objects Using Deep Learning. In the list of results, click the Detect Objects Using Deep Learning tool to open it.

   

3. Set the following Detect Objects Using Deep Learning parameter values:
   • For Input Raster, choose Kolovai_imagery.tif.
   • For Output Detected Objects, type Detected_Palm_Trees.
   • For Model Definition, click the Browse button.

   

   You will now retrieve the Palm Tree Extraction pretrained model from ArcGIS Living Atlas of the World.

   Note:

   ArcGIS Living Atlas of the World is Esri's authoritative collection of GIS data. It includes a growing library of over 50 pretrained deep learning models to detect various objects from buildings to ships to agricultural field boundaries.

4. In the Model Definition window, under Portal, click Living Atlas. In the search box, type Palm Tree Detection. Select Palm Tree Detection and click OK.

   

   Note:

   You can learn more about the Palm Tree Detection model by retrieving it on the ArcGIS Living Atlas site. You will learn that the model is meant to detect palm trees on high resolution imagery (5 to 15 centimeters), such as drone imagery. The imagery is expected to have three bands: red, green, and blue (RGB). This model is a good match for your imagery.

   Once you select the deep learning model, the model arguments load automatically.

5. Under Arguments, keep all the default values.

   

   Two arguments are of particular interest:

   • Threshold—A threshold of 0.5 means that an object detected will only be added to the output dataset if the model has at least a 50 percent confidence that the object is indeed a palm tree. The optimal threshold value can be found by trial and error.
   • Batch_size—Deep learning object detection cannot be performed on the entire image in one go. Instead, the tool will cut the image into small pieces known as chips. A batch size of 4 means that the tool will process four image chips at a time. As you run the tool, you may get an out of memory error because your computer doesn’t have enough memory for that level of processing. In that case, try decreasing the Batch_size value from 4 to 2 or even 1. Decreasing this value will not affect the quality of the model, only the efficiency of the model’s detection process.

   For now, you’ll keep the default values for these two arguments.

   Sometimes the model detects an object more than once. Non Maximum Suppression (NMS) is an optional process that suppresses some of the detected objects when there is duplication. The object that was detected with the highest confidence is kept, the other objects are removed. In the following example image, the palm tree was detected twice. With NMS, only one detected object will be kept.

   

6. Check the box next to Non Maximum Suppression.
7. For Max Overlap Ratio, type 0.5.

   

   • The Max Overlap Ratio parameter determines how much overlap there must be between two detected objects for them to be considered duplicates and for NMS to be applicable. A value of 0.5 means that the overlap must be of 50 percent or more. In the following example image, the overlap between the two detected objects is less than 50 percent, so they don’t represent the same object and NMS will not be applied.

     

8. Click the Environments tab.

   • 

9. For Processor Type, choose GPU.

   • 

     Note:

     This tutorial assumes that your computer has the recommended GPU capabilities. If you don't have a GPU, you can still do the process with your CPU, but it will take longer to process the data. In that case, choose the CPU option.

     At this point, you could run the tool as is: it would proceed to detect palm trees over the entire Kolovai_imagery.tif image, which could take 20 minutes to 1 hour based on your computer’s specifications. For the brevity of this tutorial, you will only detect palm trees in a small subset of the input image.

10. On the ribbon, on the Map tab, in the Navigate group, click Bookmarks and choose Detection area.

    

    The map zooms in to a smaller area of Kolovai.

    

11. In the Geoprocessing pane, on the Environments tab, under Processing Extent, click the Current Display Extent button.

    

    The Top, Left, Right, and Bottom coordinates update to match the current extent showing on the map.

12. Accept all other default values and click Run.

    You can monitor the process progress below the Run button, and you can click View Details to see more information.

    

    After a few minutes, the result layer, Detected_Palm_Trees, appears in the Contents pane and on the map. It is a feature layer in which each polygon represents a palm tree.

    Tip:

    If you get an out of memory error, try decreasing the batch_size value from 4 to 2 or even 1 and run the process again.

    You successfully detected palm trees in an area of Kolovai using a deep learning pretrained model.

    

    Note:

    The color is assigned at random and may vary.

Style the result layer and examine it

You’ll change the style of the Detected_Palm_Trees layer and review it.

1. In the Contents pane, click the Detected_Palm_Trees symbol to display the Symbology pane.

   

2. In the Symbology pane, if necessary, click the Properties tab.

   

3. Under Appearance, set the following parameters:
   • For Color, choose No color.
   • For Outline color, choose a bright yellow, such as Solar yellow.
   • For Outline width, select 2 pt.

   

4. Click Apply.

   The layer updates to the new symbology.

   • 

5. On the map, zoom in and inspect the Detected_Palm_Trees layer.

   

   You can observe the model was successful in detecting most palm trees, and there are very few cases of false positives—where the model mistakenly found a palm tree where there is none.

6. Optionally, run the Detect Objects Using Deep Learning tool again on a different extent of your choice or on the entire image. To do that, zoom and pan to the new extent on the map, and click Current Display Extent on the Environments tab. Then click Run.

   Note:

   Running the tool on the entire image will take 40 minutes or longer depending on your computer’s specifications.

   Alternatively, a map showing the result layer for the full extent was included in your project. To view it, go to the Catalog pane, expand Maps, right-click Map_results and choose Open.

   Over 12,500 palm trees were detected over the entire image.

7. Optionally, experiment with changing some of the arguments and parameters, such as Threshold and Max Overlap Ratio.
8. On the Quick Access toolbar, click the Save button to save your project.

   

Apply a pretrained model to your imagery

To apply a pretrained deep learning model to your own data, first try using the workflow you learned in this tutorial. This is the simplest approach, and you may immediately obtain high quality results.

However, you may not be fully satisfied with your first results. In that case, there are several techniques that can be applied next. Most importantly, you can do the following:

• Modify your imagery to ensure it better suits the pretrained model’s expectations (for instance, if the model was trained on red-green-blue imagery, your imagery should only have the red, green, and blue bands present, as opposed to infrared or other spectral bands).
• Train the deep learning model further by giving it some examples of your own data.

You can learn about these powerful approaches in the Improve a deep learning model with transfer learning tutorial.

In this tutorial, you used deep learning with a pretrained model from ArcGIS Living Atlas to detect palm trees in an image. The resulting feature layer can be used to estimate the number of palm trees throughout the region, monitor farming practices, and evaluate upcoming production.

You can find more tutorials in the tutorial gallery.

Acknowledgements

• Kolovai_imagery.tif: This drone imagery comes from the OpenAerialMap data repository.
• World Topographic Map sources: Esri, HERE, Garmin, FAO, NOAA, USGS, OpenStreetMap contributors, and the GIS User Community
• World Hillshade layer sources: Esri, Airbus DS, USGS, NGA, NASA, CGIAR, N Robinson, NCEAS, NLS, OS, NMA, Geodatastyrelsen, Rijkswaterstaat, GSA, Geoland, FEMA, Intermap, and the GIS user community