In the previous lesson, you classified an image to show impervious surfaces. In this lesson, you'll assess the accuracy of your classification by statistically comparing it to the original image. After confirming that the classification has an acceptable accuracy, you'll calculate the area of impervious surface per parcel and symbolize the parcels accordingly.
Create accuracy assessment points
Visually comparing the classified image to the original doesn't provide a statistical measurement of the classification's accuracy. With storm water bills being determined from your analysis, you'll perform a more rigorous assessment by creating randomly generated accuracy assessment points throughout the image. You'll then compare the classified value of the image at the location of each point with the actual land-use type, or ground truth, of the original image.
- If necessary, open the Surface Imperviousness project in ArcGIS Pro. In the Catalog pane, expand the Tasks folder and open the Calculate Surface Imperviousness task.
- In the Tasks pane, expand the Assess Classification Accuracy task group. Double-click the Create accuracy assessment points task to open it.
The first step of the task opens the Create Accuracy Assessment Points tool. This tool generates random points throughout an image and gives the points an attribute based on the classified value of the image at the point's location. The accuracy assessment points will also have a field for the ground truth of the original image, which you'll manually fill in for each point.
- For Input Raster or Feature Class Data, choose the Louisville_Impervious layer.
- For Output Accuracy Assessment Points, click the Browse button. Browse to the Neighborhood_Data geodatabase and save the output layer as My_Accuracy_Points.
Next, you'll determine the characteristics of the points. The Target Field parameter determines whether the attribute table of the points describes the classification value or the ground truth value. Your input image is the classified raster, so the points should contain the classification values. The Number of Random Points parameter determines how many points are created. For a small image with only two classes, a relatively small number of points is acceptable.
Lastly, the Sampling Strategy parameter determines how points are randomly distributed across the image. The points can be distributed proportionally to the area of each class, equally between each class, or absolutely randomly. Because your primary interest is in the accuracy of impervious surfaces (the smaller of the two classes), you'll equally distribute the points between each class to better represent impervious surfaces in the assessment.
- Change the remaining parameters:
- Target Field: Classified
- Number of Random Points: 100
- Sampling Strategy: Equalized stratified random
- Click Run.
One hundred accuracy points are added to the map (they may be difficult to see in the example image) and the task continues to the next step. The tool also added attributes to the points. Specifically, the points attribute table contains the class value of the classified image for each point location. You'll now use the accuracy points data to compare the classified image to the ground truth of the original image.
- In the Contents pane, right-click the My_Accuracy_Points layer and choose Attribute Table.
The attribute table opens.
Other than the ObjectID and Shape fields, the points have two attributes: Classified and GrndTruth (or Ground Truth). The Classified field has values that are either 20 or 40. These numbers represent the classes in the image: 20 is impervious; 40 is pervious. For the GrndTruth field, however, every value is -1 by default. You'll edit the GrndTruth attributes to either 20 or 40 depending on the type of terrain that the point covers in the original image.
- In the Contents pane, uncheck all layers except My_Accuracy_Points and Louisville_Neighborhood.tif.
- In the attribute table, click the row header (the small gray square) next to the first record to select the feature. Right-click the row header and choose Zoom To.
The map zooms to the selected point. (Your point will be in a different location than the point in the example.)
In this example, the point appears to be on either grass or bare earth. Either way, the surface is pervious. You would change the GrndTruth attribute for this point to 40 for pervious. If your first point appears to be on an impervious surface such as roads or roofs, you'll change the GrndTruth attribute to 20 for impervious.
Depending on your map extent and the location of the point, you may not have zoomed close enough to the point to determine what kind of ground cover it is on. Feel free to zoom closer to better determine the point's ground truth.
- In the attribute table, in the GrndTruth column, double-click the value for the selected feature to edit it. Replace the default value with either 40 or 20, depending on the point's location, and press Enter.
- Select the next point in the attribute table. Right-click the point and choose Pan To.
The map pans to the corresponding point.
- Depending on the location of the point, change the GrndTruth value to either 20 or 40.
It may be difficult to tell the ground truth for some of the points due to ambiguous features on the map. The most rigorous accuracy assessment would involve on-site verification of accuracy assessment points, but in many cases traveling to the actual location being analyzed is infeasible. Edit each point with your best guess based on the imagery.
- Repeat the process for the first ten points.
Under normal circumstances, you would need to examine and edit each accuracy point. However, to save time in this lesson, you will not continue to repeat this process for the rest of your points. The data that you downloaded at the beginning of the project includes an accuracy assessment point feature class with the GrndTruth field populated for you. You'll use the provided feature class for subsequent analysis in this lesson.
- Close the attribute table. In the Contents pane, right-click the Louisville_Neighborhood layer and choose Zoom To Layer.
The map returns to the full extent of the imagery.
- In the Tasks pane, click Next Step.
Although you'll use the provided accuracy points for the remainder of the project, you'll still save the edits that you made to your own points.
- Click Run. In the Save Edits window, click Yes to save all edits.
- In the Tasks pane, click Finish.
Compute a confusion matrix
After creating accuracy assessment points and populating their attributes with ground truth data, you'll use the points to create a confusion matrix. A confusion matrix is a table that compares the Classified and GrndTruth attributes of accuracy assessment points and determines the percentage of accuracy between them. If the areas that were classified as impervious actually represent impervious areas in the original imagery, the confusion matrix will have a high percentage and indicate high accuracy of the classification.
- In the Tasks pane, double-click the Compute a confusion matrix task to open it.
The task opens the Compute Confusion Matrix tool. The tool has only two parameters: an input and an output.
- For Input Accuracy Assessment Points, click Browse. Browse to the Neighborhood_Data geodatabase and select Accuracy_Points.
- For Output Confusion Matrix, click Browse. Save the output in the Neighborhood_Data geodatabase as Confusion_Matrix.
- Click Finish.
The tool runs and the confusion matrix is added to the Contents pane. Because the confusion matrix is a table with no spatial data, it does not appear on the map.
- In the Contents pane, under Standalone Tables, right-click Confusion_Matrix and choose Open.
Because you have so many layers in the Contents pane, you may need to scroll down to find the confusion matrix. If you want to reduce the amount of space that the imagery layers take up in the Contents pane, click the arrows next to the layer name to collapse the layer symbology.
The confusion matrix opens.
The values in the ClassValue column serve as row headers in the table. C_20 and C_40 correspond to the two classes in the classified raster: 20 for impervious surfaces and 40 for pervious surfaces. The C_20 and C_40 columns represent points with a ground truth of 20 or 40, while the C_20 and C_40 rows represent points that were classified as 20 or 40. For instance, when using the example points, 47 points that had a ground truth of 20 were also classified as 20, while one point with a ground truth of 20 was misclassified as 40. Out of a total of 100 points, four were misclassified (three were misclassified as impervious, and one was misclassified as pervious).
U_Accuracy stands for user's accuracy. It represents the fraction of pixels classified correctly per total classifications. P_Accuracy stands for producer's accuracy and represents the fraction of pixels classified correctly per total ground truths. For instance, 50 pixels were classified as impervious, of which 47 were classified correctly, leading to a user's accuracy of 0.94 (or 94 percent). Meanwhile, 48 pixels had a ground truth of impervious, of which 47 were classified correctly, leading to a producer's accuracy of approximately 0.98 (or 98 percent).
The final attribute is Kappa. Based on the total user's and producer's accuracies, it gives an overall assessment of the classification's accuracy. In the example above, the Kappa is 0.92, or 92 percent. While not perfect, an overall accuracy of 92 percent is fairly reliable. If you used your own accuracy points instead of the example points, you might receive different values. For the purposes of this lesson, you'll assume your classification was fairly accurate.
If your Kappa is below 85 to 90 percent, your classification may not be accurate enough. There are two parts of the workflow that may contribute to classification error. The first is segmentation. If your segmentation parameters generalize the original image too heavily or not enough, features may be misclassified. Try tweaking the segmentation parameters for a better segmentation. Alternatively, the majority of error may have been caused by your training samples. Having too few training samples, or training samples that cover too wide a variety of spectral signatures, may also lead to classification error. Adding either more samples or more classes may increase the accuracy.
- Close the confusion matrix.
Tabulate the area
Now that you've assessed your classification's accuracy, you'll determine the area of impervious surfaces within each parcel of land in the neighborhood. You'll first calculate the area and store the results in a stand-alone table. Then, you'll join the table to the Parcels layer.
- In the Tasks pane, expand the Calculate Impervious Surface Area task group. Double-click the Tabulate the area task to open it.
The first step of the task opens the Tabulate Area tool. This tool calculates the area of classes within zones that can be defined by an integer raster or a feature layer.
- For Input raster or feature zone data, choose the Parcels layer. Confirm that the Zone field parameter populates with the Parcel ID field.
The zone field is an attribute field that identifies each zone for which area will be calculated. You want the zones to correspond to the parcel features, so you'll use a zone field that is unique for each parcel. The Parcel ID field has a unique identification number for each feature, so you'll leave the parameter unchanged.
- For Input raster or feature class data, choose the Louisville_Impervious layer.
- For Class field, choose Class_name.
The class field determines the field by which area will be determined. You want to know the area of each class in your reclassified raster (pervious and impervious), so the Class_name field is appropriate.
- For Output table, confirm that the output location is the Neighborhood_Data geodatabase and change the output name to Impervious_Area.
The final parameter, Processing cell size, determines the cell size for the area calculation. By default, the cell size is the same as the input raster: half a foot (in this case). You'll leave this parameter unchanged.
- Click Run.
The tool runs and the table is added to the Contents pane. The task continues to the next step and opens the Join Field tool. Before you continue, you'll take a look at the table that you created.
- In the Contents pane, right-click the Impervious_Area table and click Open.
The table has a standard ObjectID field, as well as three other fields. The first is the Parcel_ID field from the Parcels layer, showing the unique identification number for each parcel. The next two are the class fields from the Louisville_Impervious raster. Impervious shows the area (in feet) of impervious surfaces per parcel, while Pervious shows the area of pervious surfaces.
- Close the table.
You now have the area of impervious surfaces per parcel, but only in a stand-alone table. Next, you'll join the stand-alone table to the Parcels attribute table. A table join updates the input table with the attributes from another table based on a common attribute field. Because you created the Impervious_Area table with the Parcel_ID field from the Parcels layer, you'll perform the join based on that field.
- In the Tasks pane, for Input Table, choose the Parcels layer.
- For Input Join Field, choose Parcel ID.
- For Join Table, choose the Impervious_Area table.
- For Output Join Field, choose Parcel_ID.
With the final parameter, Join Fields, you can choose specific fields from the join table to include in the result. If left empty, all fields from the join table will be included. The join table has only three fields, so there's no reason not to add them all.
- Click Finish.
The tool runs and the task ends.
- In the Contents pane, open the attribute table for the Parcels layer. Confirm that the attribute table includes the following fields:
- Close the table.
Symbolize the parcels
Now that the tables have been joined, you'll change the field aliases to be more informative. Then, you'll symbolize the parcels by impervious surface area to depict the area attribute on the map.
- In the Tasks pane, double-click the Clean up the table and symbolize the data task to open it.
The first step of the task is to clean up the Parcels attribute table.
- In the Contents pane, click the Parcels layer to select it (it may already be selected). In the Tasks pane, click Run.
The Fields view for the Parcels attribute table opens. With the Fields view, you can add or delete fields, as well as rename them, change their aliases, or adjust other settings. First, you'll remove the redundant Parcel_ID_1 field.
- Right-click the gray square to the left of the Parcel_ID_1 field and choose Delete.
Next, you'll change the field aliases of the two area fields to be more informative.
- Change the alias of the Impervious field to Impervious Area (Feet).
- Change the alias of the Pervious field to Pervious Area (Feet).
- On the ribbon, on the Fields tab, in the Changes group, click Save.
The changes to the attribute table are saved.
- Close the Fields view. In the Tasks pane, click Next Step.
The second step of the task is to symbolize the Parcels layer. First, however, you need to turn the Parcels layer back on. You'll also turn off layers that are unnecessary for visualizing the data.
- In the Contents pane, uncheck the My_Accuracy_Points layer to turn it off. Check the Parcels layer to turn it on and confirm that the layer is selected.
- In the Tasks pane, click Run.
The Symbology pane for the Parcels layer opens. Currently, the layer is symbolized with a single symbol. You'll symbolize the layer so that parcels with high areas of impervious surfaces appear differently than those with low areas.
- In the Symbology pane, for Primary symbology, choose Graduated Colors.
A series of parameters becomes available. First, you'll change the field that determines the symbology.
- For Field, choose Impervious Area (Feet).
The symbology on the layer changes automatically. However, there is little variety between the symbology of the parcels because of the low number of classes.
- Change Classes to 7. Change the Color scheme to Yellow to Red.
The layer symbology changes again.
The parcels with the highest area of impervious surfaces appear to be the ones that correspond to the location of roads. These parcels are very large and almost entirely impervious. In general, larger parcels tend to have larger impervious surfaces. While you could symbolize the layer by the percentage of area that is impervious, most storm water fees are based on total area, not percentage of area.
- Close the Symbology pane. In the Tasks pane, click Finish.
- Save the project.
In this project, you classified an aerial image of a neighborhood in Louisville, Kentucky, to show areas that were pervious and impervious to water. You then assessed the accuracy of your classification and determined the area of impervious surfaces per land parcel. With the information that you derived in this lesson, the local government would be better equipped to determine storm water bills. While your classification was not perfect, it was accurate enough that the local government could have reasonable confidence in your results.
You can use the tasks and tools in this project with your own data. As long as you have high-resolution, multispectral imagery of an area, you can classify its surfaces. This ArcGIS Pro task is designed to quickly replicate the workflow described by these lessons.
Try some of the other lessons in the Learn ArcGIS Gallery to discover more capabilities of ArcGIS.