Configure your system
Before you can run the deep learning model, you need to check your computer system, install the latest drivers for your graphics card, and ensure that you have installed the deep learning libraries for use with ArcGIS Pro.
Check your graphics card
The processes that are performed when using deep learning take a large amount of memory from your computer. To perform a deep learning process, an NVIDIA GPU with a minimum dedicated memory of 8 GB is recommended. To check your graphics card's dedicated GPU memory, you will use the NVIDIA SMI utility. Learn more about what GPU is and how it works.
- Click the Start menu and type command. Click Command Prompt to open it.
The Command Prompt window appears. The first command you will run is to change directories to where the NVIDIA-SMI utility is located.
- In the Command Prompt window, type cd, add a space, type or copy and paste C:\Windows\system32, and press Enter.
The path of the executable can be different on your machine depending on how the graphics cards are installed. NVIDIA could also be installed to this folder: C:\Program Files\NVIDIA Corporation\NVSMI. You can search your C drive for nvidia-smi.exe and in the Command Prompt, change the directory to that folder, using the cd command. Once you are in the correct folder, you can run the nvidia-smi.exe file.
Next, you will run an executable to find the dedicated GPU memory.
- In the Command Prompt window, type or copy and paste nvidia-smi.exe and press Enter.
The command runs and shows the maximum amount of dedicated GPU memory for your computer.
Depending on your computer, the displayed information may be different. In this example, this computer meets the minimum recommendation for this lesson of 8 GB of dedicated memory in the GPU.
If you have more than one graphics card, the deep learning geoprocessing tools will automatically choose the best GPU in the training and inferencing process.
- Make a note of the name of the graphics card and close the Command Prompt window.
In this example, the name of the graphics card is Quadro RTX 4000.
Now that you know more about your graphics cards, you will download and install the latest drivers from NVIDIA.
Update NVIDIA drivers
Next, you will update the drivers for your graphics card. An out-of-date GPU driver will cause deep learning tools to fail, so it is good practice to check system requirements and update the drivers before running the tools.
- Go to the NVIDIA Driver Downloads page.
- In the NVIDIA Driver Downloads section, choose the options that match your graphics card. In this example, the graphics card is Quadro RTX 4000.
- Click Search, click Download, and click Agree & Download.
- On your computer, locate and run the installer file. Click Yes to allow changes to the system and click OK to run the installer.
- In the NVIDIA Installer window, click Agree and Continue, accept the default installation options, and click Next.
- When the installation is finished, click Close.
Now, you have the necessary drivers for processing imagery with deep learning in ArcGIS Pro. However, you also need to have the deep learning libraries installed on your computer for ArcGIS Pro.
Using the deep learning tools requires that you have the correct Deep Learning Libraries installed on your computer. If you do not have these files installed, save your project, close ArcGIS Pro, and follow the instructions to install deep learning frameworks for ArcGIS. Once installed, you can reopen your project and continue with the lesson.
Download and explore data
Next, you will download three .zip files that contain data that you will use throughout the lesson for training and classification of point cloud data. Once you download and extract the data, you will explore and symbolize LAS dataset files in ArcGIS Pro.
Download the data
All the data required to complete the lesson is in three .zip files that you will now download.
- On your computer, open the File Explorer.
- In your C:\ drive, create a folder named LearnArcGIS. Within the LearnArcGIS folder, create another folder named DL.
The C:\LearnArcGIS\DL folder is where you will store all the data and outputs for the lesson.
- Download the following .zip files:
- smalldata.zip—Contains a small training dataset and the training boundary and a small validation dataset and the validation boundary.
- testdata.zip—A test dataset and the processing boundary.
- outputs.zip—Training results from the large dataset that you can use instead of training the model for the large dataset.
- When the files are downloaded, extract each .zip file to the LearnArcGIS/DL folder.
To extract a .zip file, right-click it, choose Extract All, and browse to your LearnArcGIS\DL folder.
The folders contain various information that you will use for preparing and training data. Some of the main files in the folder are LAS files. A LAS file is an industry-standard binary format for storing airborne lidar data. The LAS dataset allows you to examine LAS files, in their native format, quickly and easily, providing detailed statistics and area coverage of the lidar data contained in the LAS files. The results folder contains a trained model using the large dataset that was performed on a computer with 24 GB GPU. In the final section of this lesson, you will use this trained model later in the lesson to classify a LAS dataset.
Next, you will review and familiarize yourself with the downloaded data in ArcGIS Pro.
View the training data and validation data
You will create an ArcGIS Pro project and view the small training data and validation data.
- If necessary, start ArcGIS Pro.
- Sign in to your ArcGIS organizational account or into ArcGIS Enterprise.
If you don't have an organizational account, you can sign up for an ArcGIS free trial.
- Under New and Blank Templates, click Local Scene.
- On the Create a New Project dialog box, for Name, type Powerline Classification with DL. For Location, click Browse, browse to C:\LearnArcGIS, click the DL folder, and click OK. Uncheck Create a new folder for this project.
- Click OK.
- In the Catalog pane, expand Folders and expand DL.
If the Catalog pane is not visible, on the ribbon, click View, and in the Windows group, click Catalog Pane.
You can see the data folders that you downloaded and extracted to this location.
First, you will explore the files in the small_data folder and update the statistics on the dataset to ensure that the proper class codes display.
- In the Catalog pane, expand the small_data folder. Right-click train_small.lasd and choose Properties.
- In the LAS Dataset Properties window, click the Statistics tab.
The dataset contains several Classification Codes:
- 1- Unassigned represents low vegetation or objects above the ground, but lower than high vegetation and buildings.
- 2- Ground represents the ground.
- 5- High Vegetation represents vegetation, such as trees.
- 6- Building represents buildings and other structures.
- 7- Noise represents low points.
- 14- Wire-Conductor represents actual power wires.
- 15- Transmission Tower represents the power line towers.
You are interested in the classification code 14 for Wire Conductor. Your goal is to train a model to detect lidar points that are the power lines so you can better assess wildfire risks for power lines in close proximity to trees.
At the bottom of the window, there is a message that states that one file has out-of-date or no statistics. You will fix this issue now so that you can display the correct class codes in the scene.
- Click Update.
A message appears confirming that the statistics are up to date.
Now that the statistics are up to date, you can style the layer according to classification codes.
- Click OK.
- In the Catalog pane, right-click train_small.lasd and choose Add To Current Map.
The LAS dataset layer appears in the scene.
Now that you have updated the statistics for the small dataset LAS file, you will do the same for the LAS dataset that will be used to validate the model, val_small.lasd.
- In the Catalog pane, open the properties for val_small.lasd, update the statistics, and click OK. Add val_small.lasd to the current map.
- In the scene, press C and V while dragging your pointer to tilt it and pan the scene to explore the datasets in 3D.
You will use train_small.lasd to train the model and val_small.lasd for validating the trained model to prevent overfitting during the training process.
Next, you will update the symbology for each of the LAS layers to show different colors by the classification codes. You'll start with the train_small.lasd layer.
- In the Contents pane, click the train_small.lasd layer to select it. On the ribbon, click the Appearance tab.
- On the Appearance tab, in the Drawing section, for Symbology, click the drop-down menu and choose Class.
The Symbology pane appears and the train_small.lasd layer is symbolized by its Classification Codes.
If you have white spaces in your LAS dataset, this indicates a caching issue. You can go to the properties of the LAS dataset, choose Cache, and click Clear Cache to fix this, or it will be fixed automatically when you close and open ArcGIS Pro.
- In the Contents pane, click val_small.lasd. On the Appearance tab, click Symbology and choose Class.
The val_small.lasd layer is now symbolized by its Classification Codes.
LAS datasets can be large and take more time to display than other layers. To speed up the display, you can build pyramids using the Build LAS Dataset Pyramid tool. This tool creates or updates a LAS dataset display cache, which optimizes its rendering performance.
- Above the ribbon, on the Quick Access toolbar, click Save to save the project.
Next, you will train a model using some of the datasets that you explored.
Train a model using a small dataset
In this section, you will use the small datasets to walk through the process of training a classification model. Because this dataset does not contain as many points needed for a more accurate classification, it is likely to produce a less accurate classification model. This section shows you the process of training a model and then you will use a trained model from a larger sampling of points later in the lesson to classify a LAS dataset and visualize the power lines.
Prepare the training dataset
LAS datasets cannot be used directly to train the model. The LAS datasets must be converted into smaller training blocks. You will use the Prepare Point Cloud Training Data geoprocessing tool in ArcGIS Pro to export the LAS files to blocks.
Your goal is to train the model to identify and classify the points that are power lines. Not every point in the LAS data cloud is necessary to review. Only the points within the surrounding area of power lines need to be reviewed. You will use prepared boundary data, bnd_train_small and bnd_val_small, to specify which points should be converted into training blocks.
- On the ribbon, click the Analysis tab and click Tools.
The Geoprocessing pane appears.
- In the Geoprocessing pane, search for and open the Prepare Point Cloud Training Data tool.
Note:This tool requires the ArcGIS 3D Analyst extension.
- In the Prepare Point Cloud Training Data tool, set the following parameters:
- For Input Point Cloud, choose train_small.lasd.
- For Training Boundary Features, click Browse, browse to C:\LearnArcGIS\DL\small_data\boundaries.gdb, choose bnd_train_small, and click OK.
- For Validation Point Cloud, choose val_small.lasd.
- For Validation Boundary Features, click Browse, browse to C:\LearnArcGIS\DL\small_data\boundaries.gdb, choose bnd_val_small, and click OK.
You have entered all the training, boundary, and validation layers.
- For Output Training Data, click Browse and browse to C:\LearnArcGIS\DL\results. For Name, type training_data_small.pctd.
The output file extension .pctd stands for point cloud training data.
- Click Save.
- In the Prepare Point Cloud Training Data tool, continue entering the following parameters:
- For Block Size, type 82. For Unknown, click the drop-down menu and choose Feet as the unit.
- For Block Point Limit, keep the default of 8192.
Block Size and Block Point Limit control how many points are in one block. To determine this value, it is important to consider the average point spacing, the objects of interest, available dedicated GPU memories, and the batch size when setting these two parameters. A general rule is that the block size should be large enough to capture the objects of interest with the least amount of subsampling, or reducing the size of the original data.
You will start with the Block Size of 82 feet (about 25 meters) and the Block Point Limit of 8192. 82 feet is a proper block size to capture the power line’s geometry. You will examine the output to see if 8,192 is an appropriate block point limit. If most blocks have more than 8,192 points, you will need to increase the block point limit to reduce subsampling.
For the large data, it is recommended that you use the LAS Point Statistics As Raster tool to generate the histogram to determine the proper block size and block point limit before running the Prepare Point Cloud Training Data tool. To use the tool, choose Point Count as the method and Cell Size as the sampling type and test with different values for Block Size.
- Click Run.
A confirmation message appears at the bottom of the tool pane when the tool completes.
- In Windows, open the File Explore and browse to C:\LearnArcGIS\DL\results. The results folder now contains a training_data_small.pctd folder.
- Double-click training_data_small.pctd to open the folder.
The output file contains two subfolders, train and val, which contain the exported training and validation data, respectively.
- Open each folder to review their contents.
In each folder, you will see a Statistics.json file, a ListTable.h5, a BlockPointCountHistogram.png file, and a 0 folder containing Data_x.h5 files.
The ListTable.h5 and Data_x.h5 files include the information of points (xyz and attributes such as return number, intensity, and so on), which are organized into blocks.
- In the train folder, double-click BlockPointCountHistogram.png.
The BlockPointCountHistogram.png shows a histogram of the block point count based on the block size you specified, which was 82.
The histogram shows that most of the blocks have a point count of less than 8,192, which confirms that a Block Size of 82 feet and the default Block Point Limit of 8,192 are proper for this dataset.
Now that you have prepared the training data, you can use the blocks to train a classification model with deep learning.
Train a classification model
Next, you will use the Train Point Cloud Classification Model geoprocessing tool to train a model for power line classification using the small training dataset. The results may vary based on different runs of the tool and different computer configurations.
- In ArcGIS Pro, in the Geoprocessing pane, click the back arrow. Search for and open the Train Point Cloud Classification Model tool.
- In the Train Point Cloud Classification Model tool, set the following parameters:
- For Input Training Data, click Browse. Browse to C:\LearnArcGIS\DL\results, and double-click training_data_small.pctd.
- For Pre-trained Model, leave it blank.
- For Attribute Selection, click the drop-down menu and choose Intensity.
The intensity of power lines is lower when compared with vegetation and ground, so it is an effective attribute for distinguishing power lines.
- For Output Model Location, browse to the results folder. Click it and click OK.
- For Output Model Name, type Powerline_classification_model_small_data.
- For Minimum Points Per Block, type 2000.
By setting the Minimum Points Per Block to 2000 during the training, blocks whose points are less than 2,000 will be skipped. Blocks with a small number of points are likely to be located at the boundaries where there are no power line points. Skipping these blocks in the training will make the training faster and make the model's learning more efficient. This parameter is only applied on the training data blocks, not the validation data blocks.
- Expand Manage Classes.
- Under Class Remapping, for Current Class, choose 14. For Remapped Class, choose 14.
Class 14 represents the wire conductors that you want to classify and locate in the point cloud.
- In the next row, for Current Class, choose OTHER and from Remapped Class, choose 1.
By specifying the Class Remapping, classification code 14 will remain unchanged, meaning points in the LAS dataset that already classify as power lines will remain power lines. All other classification codes will be remapped to the class code of 1. The resulting layer will only have two symbolized classes, 1 and 14, thus making the power lines easier to distinguish in the scene.
- For Class Description, accept the populated class codes and descriptions.
- Expand Training Parameters. For Model Selection Criteria, accept the default of Recall.
Model Selection Criteria specifies the statistical basis that will be used to determine the final model. The default of Recall will select the model that achieves the best macro average of the recall for all class codes. Each class code's recall value is determined by the ratio of correctly classified points (true positives) over all the points that should have been classified with this value (expected positives). For each class, its recall value is the ratio of correctly predicted points of this class to all the reference points of this class in the validation data. For example, the recall value of class code 14 is the ratio of correctly predicted power line points to all the reference power line points in the validation data.
- For Maximum Number of Epochs, type 10.
An epoch is the complete cycle of the entire training data learned by the neural network (in other words, the entire training data is passed forward and backward through the neural network one time). You will train the model for 10 epochs to save time.
- For Iteration Per Epoch (%), accept the default of 100.
Leaving the Iteration Per Epoch (%) parameter at 100 ensures that all the training data will be passed per epoch.
You can also choose to pass a percentage of the training data at each epoch. Set this parameter to something other than 100 if you want to reduce the completion time for each epoch by randomly selecting fewer batches. However, this can lead to more epochs before the model converges. This parameter is useful is you want to quickly see the model metrics at the tool’s messages window.
- For Learning Rate, leave it blank and let the tool pick the optimum learning rate for you.
You can experiment with different learning rates. A small value can lead to slow training of the model, while a large value can prevent the model from converging.
- For Batch Size, use the default value of 2.
Batch Size specifies how many blocks are processed at a time. The training data is split into batches. For example, if the Batch Size value is set to 2, 1,000 blocks are split into 500 batches, and each of the 500 batches are processed in one epoch. By splitting the blocks into batches, the process will consume less GPU memory.
- Uncheck Stop training when model no longer improves.
Unchecking this option will allow the training to run for 10 epochs. If checked, the training will stop when the model is no longer improving after several epochs, regardless of the maximum number of epochs specified.
- Click Run.
Depending on your system, the time it takes to run this tool may vary. On the NVIDIA Quadro RTX 4000 GPU with 8 GB of dedicated memory, this tool will take about 30 minutes to run. With a GPU that has 4 GB of memory, it may take around 50 minutes. As the tool runs, you can monitor its progress.
- At the bottom of the Geoprocessing pane, click View Details.
The Train Point Cloud Classification Model (3D Analyst Tools) window appears, displaying the Parameters tab, which shows the parameters that were used for running the tool.
- Click the Messages tab.
The geoprocessing messages populate as the tool runs. When the tool is completed, the geoprocessing message shows the results for each epoch.
The information in your messages may be different from the example shown, depending on your GPU and system setup.
The tool reports the following information:
- GPU used in the training.
- Count of the training data blocks used in the training (only training data block containing more than 2,000 points are used in the training).
- Count of the validation data blocks used in the validation—all the validation data blocks are used in the validation.
- Iteration per epoch—the count of the training data blocks divided by the batch size.
The tool first calculates the optimal learning rate and then reports Training Loss, Validation Loss, Accuracy, Precision, Recall, F1-Score, and Time spent for each epoch.
As each epoch progresses, you can see the Training Loss and Validation Loss values decrease, which indicates that the model is learning. In the final epoch, the Training Loss is low, but the Validation Loss is high, which means the training is overfitted for the training data (in other words, the trained model does not generalize well on new, unseen data).
In the final epoch, the Recall value is over .80.
The lower Recall value indicates that it is not an accurate model, which is expected because the model was trained with a small dataset. These results highlight the need to have more sample points in separate datasets so that you can achieve better results.
Examine the training outputs
Next, you will look at the results of training the classification model.
- In the File Explorer, browse to C:\LearnArcGIS\DL\results.
In this folder, there are two subfolders. One is the model folder, and another is the checkpoints folder.
- Expand the Powerline_classification_model_small_data folder and expand the Model Characteristics folder.
The Model Characteristics folder contains a loss graph and ground truth and predictions results graph.
- In the Model Characteristics folder, double-click loss_graph to view a graph of the model's loss.
- Go back to the results folder. Open the Powerline_classification_model_small_data.checkpoints folder and view its contents.
The Powerline_classification_model_small_data.checkpoints folder includes the models folder, which contains data for each of the epoch checkpoints and two files versions of statistics, one in HTML and the other in CSV format. When the training tool was running, one checkpoint was created after each epoch. Each checkpoint contains a .pth file and an .emd file.
- In the Powerline_classification_model_small_data.checkpoints folder, open the Powerline_classification_model_small_data_Statistics Microsoft Excel file to see the statistics.
The CSV file opens in Excel.
The statistics file includes the precision, recall, and f1-score values for each class after each epoch. In certain cases, the model with the best overall metrics may not be the model that performed the best in classifying a specific class code. If you are only interested in classifying certain class codes, you may consider using the checkpoint model associated with the best metrics for that class code. Viewing the statistics in Excel allows you to sort the columns and easily locate the epoch with the highest recall value. You have trained a model using a smaller sampling of points. Next, you will use a model that was trained using a larger sampling of points to classify a LAS dataset.
Classify a LAS dataset using the trained model
You will classify the LAS dataset containing more than 2 million points using a trained model. Classifying the LAS dataset using a trained model allows you to locate power lines in the study area for risk assessment analysis. When the model was trained, two classification codes were designated, Unassigned and Wire Conductor. By classifying points in the point cloud to wire conductor and the rest as unassigned, it will make the dataset more valuable, as it will clearly identify power lines.
Explore model results and choose the best epoch
You will explore the results of a provided model training that used the large dataset. You will look for the best recall value for wire conductors.
- Open the File Explorer and browse to C:\LearnArcGIS\DL\results.
The folders contain results of training the small and large data using a computer with 24 GB of dedicated GPU memory.
The parameters and progress files are not the results from training: they are a screen shot of the parameters used and the epoch statistics, respectively.
- Open the Powerline_classification_model_large_data.checkpoints folder and double-click Powerline_classification_model_large_data_Statistics to view the statistics in Excel.
The file opens in Excel.
- In Excel, double-click the column dividers to expand them so that you can see all the text.
You will sort the Recall column and find the largest value for Wire Conductor and use that epoch for classifying the LAS dataset. The largest Recall value for the Wire Conductor indicates the epoch that you should use in the classification.
- Click the Recall column header to highlight the entire column.
- On the ribbon, in the Editing section, click Sort & Filter and choose Sort Largest to Smallest.
- In the Sort window that appears, click Sort.
The rows in the spreadsheet are sorted so it is easy to see the highest value for Wire Conductor.
- Locate the first row of CLASS_CODE with a value of Wire Conductor (row 25).
This is the highest recall value of 0.923336055, for Wire Conductor, which occurred in EPOCH 16.
You will use epoch, or checkpoint 16, when you classify the LAS dataset, as it has the highest recall value for the power line class code and will provide the best results.
You will use the trained model that is provided for you to classify the LAS dataset. You could also use the model that you trained, if you performed that operation. You would follow the same steps in the statistics spreadsheet to find the epoch that has the highest recall value for wire conductor and use that epoch file to classify the data.
Classify power lines using the trained model
Next, you will use a trained model to classify the power lines from a test dataset. You will apply processing boundaries to only classify points within the boundaries.
- Restore ArcGIS Pro.
- In the Catalog pane, expand Folders, expand DL, and expand testdata. Right-click test.lasd, choose Add To New, and choose Local Scene.
The test.lasd LAS dataset appears in a local scene.
- If necessary, in the Contents pane, click test.lasd to select it. On the ribbon, click the Appearance tab, click the drop-down arrow for Symbology and choose Class.
The test.lasd LAS dataset is now rendered using class codes.
All points are gray because currently, the layer displays only with a class a code of 1, or Unassigned.
- In the Geoprocessing pane, search for and open the Classify Point Cloud Using Trained Model tool.
- For Target Point Cloud, click the drop-down menu and choose test.lasd.
- For Processing Boundary, browse to C:\LearnArcGIS\DL\testdata\boundaries.gdb and double-click bnd_test.
By selecting a Processing Boundary, the tool will use the trained model you provide to classify only the points within the boundary.
Because the power line classification model was trained using points within the surrounding boundary of the power lines, a similar boundary should also be applied on the test dataset to be classified.
- For Input Model Definition, click Browse. Browse to C:\LearnArcGIS\DL\results\Powerline_classification_model_large_data.checkpoints\models\checkpoint_2021-10-05_18-54-53_epoch_16. Click checkpoint_2021-10-05_18-54-53_epoch_16.emd and click OK.
For Input Model Definition, you can choose an .emd file, a .dlpk file, or the URL to the model shared at ArcGIS Online or ArcGIS Living Atlas of the World.
The Target Classification parameter appears after selecting a model definition file.
- Under Target Classification, uncheck 1 Unclassified.
By unchecking 1 Unclassified, when the model runs, the points are predicted as power lines and will have their class codes assigned as 14; otherwise, their class codes are kept as 1.
- Click Run.
The tool runs. Next, you will update the symbology to show the results of the classification model.
- Open the Symbology pane. Next to Values, click More and click Add all values.
The symbology updates and the points in the point cloud that are wire conductors display in yellow.
- Use the navigation tools and shortcuts to examine the classification results in the local scene.
The classification results are accurate using the trained model. Most of the power line points are correctly classified. Notice that the power poles are still Unassigned.
In this lesson, you learned the workflow of point cloud classification using deep learning technology. You gained experience with deep learning concepts, the importance of validation data in the training process, and how to evaluate the quality of the trained models. Going further, you can use the Extract Power Lines From Point Cloud tool to generate 3D lines to model the power lines. You could also use the Locate LAS Points By Proximity tool to find lidar points that are within a certain distance from the power lines. These points, mostly trees, are too close to the power lines, which could cause a power outage or spark a fire. These tools can go further to provide important information for insurance risk assessments.
You can find more lessons in the Learn ArcGIS Lesson Gallery.