Prepare multidimensional pollution data

First, you will explore a mosaic dataset of pollution data. You will use the mosaic to create a multidimensional layer and a space-time cube. The multidimensional layer and space-time cube data structures are different ways of representing temporal data, and are formats that are compatible with many different analytical tools.

Download and explore the data

You will start by exploring global raster data measuring levels of fine particulate matter pollution. These are air pollution particles with a diameter of 2.5 microns or smaller, referred to as PM 2.5. You can read about fine particulate matter and other outdoor air pollution at the World Health Organization's website.

  1. Download the InvestigatePollutionPatterns project package.
  2. Locate the downloaded InvestigatePollutionPatterns.ppkx file on your computer. Double-click the file to open it in ArcGIS Pro. If prompted, sign in to your ArcGIS account.
    Note:

    If you don't have ArcGIS Pro or an ArcGIS account, you can sign up for an ArcGIS free trial.

    The project has three maps (Part 1, Part 2, Part 3.1) and a local scene (Part 3.2). The Part 1 map is active.

  3. If necessary, zoom out until you can see the whole world.

    Part 1 map with the Equal Earth projection

    The map is drawn using the Equal Earth projected coordinate system. This coordinate system is appropriate for global analysis because it does not distort area.

  4. In the Contents pane, check the box next to PM25 to turn on visibility for the layer.

    Contents pane with the PM25 mosaic layer turned on

    PM25 is a multidimensional mosaic. Multidimensional refers to data that is captured at multiple times, depths, or heights. In this case, the extra dimension is time. A mosaic dataset in ArcGIS Pro consists of three parts: a boundary, footprint, and image.

    Part 1 map with PM25 mosaic drawn on top

  5. In the Contents pane, right-click Footprint and choose Attribute Table.

    Attribute Table option in the context menu of the footprint

    The attribute table appears below the map. It contains 19 rows.

    PM25: Footprint attribute table showing 0 of 19 selected at the bottom

    The mosaic references 19 .tif raster files. The rows in this table represent the footprints of the 19 rasters, dated from 1998 to 2016. The 19 rasters are stacked on top of each other within the PM25 mosaic. Each raster records the average level of fine particulate matter pollution (PM 2.5) for a year.

  6. Scroll to the right of the table and verify that each raster footprint contains data in the Standard Time, Dimensions, and Variable fields. Standard Time values should increment by a year for each row, starting at 1/1/1998, and ending at 1/1/2016.

    Time and variable columns in the Footprint table

    Note:

    These three fields were created using the Build Multidimensional Information tool. Many of the multidimensional tools in ArcGIS Pro require these specific fields, so this tool creates them based on existing fields (Var and Date) with the correct field names and types.

  7. Close the attribute table.
  8. In the Contents pane, click the Image layer to select it. On the ribbon, the Image Layer contextual ribbon appears.
  9. On the ribbon, click the Appearance tab. Click and expand Stretch Type button and choose Percent Clip. If you are prompted to calculate statistics, click Yes.

    Stretch Type options for an image layer

    The stretch type changes how the raster data is symbolized. Percent Clip cuts off the highest and lowest values to reduce the effect of outliers on the visualization. You can now see areas of the world with higher (white) and lower (black) levels of PM 2.5 pollution.

    PM25 mosaic layer displayed with Percent Clip stretch

Create a multidimensional raster layer

Next, you will use the mosaic dataset to make a multidimensional raster layer (in the CRF format). This is a time-enabled layer that has similar properties to the multidimensional mosaic it is created from, but the data is contained all in a single raster layer. A multidimensional raster layer can be used as input to many geoprocessing tools and charts in ArcGIS Pro, so creating this layer provides more analytical opportunities. As an example, you will explore the time slices of the multidimensional raster layer using the time slider.

  1. On the ribbon, click the Analysis tab. In the Geoprocessing group, click Tools.

    Tools button on the Analysis tab

  2. In the Geoprocessing pane, in the search bar, type copy raster. In the search results, click the Copy Raster tool to open it.

    Copy Raster tool in the Geoprocessing pane

  3. For Input Raster, choose PM25.
  4. For Output Raster Dataset, click the Browse button.

    Browse button for Output Raster Dataset in the Copy Raster tool

  5. Browse to a folder where you want to save the new raster (for example, C:). For Name, type PM25.crf and click Save.

    Output Raster Dataset window

  6. For NoData Value, type 0.
  7. Check the Process as Multidimensional check box.

    Process as Multidimensional check box

  8. Click Run.

    The new PM25.crf layer is added to the map. It looks similar to the PM25 mosaic.

  9. In the Contents pane, turn off the PM25 mosaic layer.

    You now have 19 years of data contained inside a single time-enabled raster layer instead of a mosaic of 19 raster layers. You will use the time slider to animate the PM25.crf layer.

  10. If necessary, on the Contents pane, click PM25.crf to select it.
  11. At the top of the map, point to the time slider to reveal its controls. Click the Play button in the center of the slider. The data animates from 1998 to 2016, showing one year at a time.

    Play button on the time slider

    Later in the lesson, you will also use the PM25.crf layer to create a temporal profile chart. But first, you create a third representation for your data: a space-time cube.

Create a space-time cube

Next, you will create a space-time cube from the multidimensional raster layer. A space-time cube is another way to structure space-time data. It is stored as a netCDF file and can be used with the Space Time Pattern Mining toolbox.

First, you will make sure that no time filtering is present on the multidimensional raster.

  1. On the ribbon, click the Time tab. In the Current Time group, click the Start Time Enabled and End Time Enabled buttons.

    Start Time Enabled and End Time Enabled buttons

    The Start, Span, and End fields are cleared.

    Start Time Disabled and End Time Disabled buttons

  2. In the Geoprocessing pane, click the Back button.

    Back button

    Tip:

    If you cannot find the Geoprocessing pane, click the Analysis tab on the ribbon and click the Tools button.

  3. Search for and open the Create Space Time Cube From Multidimensional Raster Layer tool.
  4. For Input Multidimensional Raster Layer, choose PM25.crf.
  5. For Output Space Time Cube, click the browse button.
  6. Browse to a folder where you want to save the space-time cube (for example, C:). For Name, type PM25 and click Save.
  7. For Fill Empty Bins Method, accept the default of Zeros.

    Create Space Time Cube From Multidimensional Raster Layer parameters

  8. Click Run.

    The tool creates a NetCDF space-time cube. The file has an .nc extension. This space-time cube is not added to the map, but it is the foundational data structure used as input for the tools in the Space Time Pattern Mining toolbox. You will see some examples of these tools later in the lesson.

  9. When the tool finishes running, click the View Details link at the bottom of the pane.

    View Details link in the Results Summary box in the Geoprocessing pane

    A window appears. This is a useful place to review the parameters that were used in the tool and read any messages, errors, or warnings.

  10. In the details window, scroll through the Messages section.

    In this section, you can review information about the space-time cube, including the number of time slices, the time interval, and the number and size of locations for each time slice. There may also be warnings about the data or the action being performed. As long as the action completes you can continue.

  11. Close the details window.
  12. On the Quick Access Toolbar at the top, click the Save button.

    Save button on the Quick Access Toolbar

So far in this lesson, you have learned about three kinds of data structures: multidimensional mosaics, multidimensional raster layers, and space-time cubes. You created and animated a multidimensional raster layer, and generated a space-time cube. Next, you will use the multidimensional raster layer to create a temporal chart, which is another method to visualize multidimensional data.


Chart and visualize pollution through time

You have created a multidimensional raster layer and used the time slider to visualize this data. However, it is difficult to make any solid conclusions using the time slider alone. Next, you will create a temporal profile chart to explore pollution time series in different countries. Condensing temporal data into charts is an important technique to more easily understand and communicate about your data.

Add country boundaries from Living Atlas

For clarity, you will use a new map, adding to it a copy of the PM25.crf layer. You will also add a world country layer using ArcGIS Living Atlas of the World.

  1. If necessary, reopen the InvestigateGlobalPollutionPatterns project.
  2. In the map named Part 1, in the Contents pane, right-click the PM25.crf layer and choose Copy.
  3. Click the Part 2 map tab to activate it. In the Contents pane, right-click Part 2 and choose Paste.

    Paste option in the context menu of the Part 2 map

    The PM25.crf layer appears in the Contents pane and on the map.

  4. Make sure that the map named Part 2 is active and open the Catalog pane.
    Tip:

    If you can’t find the Catalog pane, click the View tab on the ribbon and click the Catalog Pane button.

    Catalog pane button on the View ribbon tab

  5. In the Catalog pane, click Portal and click ArcGIS Living Atlas.

    Catalog pane open to the Portal and Living Atlas tabs

  6. In the ArcGIS Living Atlas search box, search for World Country.
  7. In the search results, right-click World Countries (Generalized) and choose Add To Current Map.

    Add To Current Map option in the context menu of the World Countries (Generalized) layer

  8. In the Contents pane, right-click the symbol for the new layer and click No color.

    No color option in the color picker for the World_Countries_(Generalized) layer

    This makes the fill of the layer transparent, so you can view the raster layer underneath.

  9. On the ribbon, click the Map tab. In the Navigate group, click Bookmarks and choose the Europe bookmark.

    Europe bookmark in the Bookmarks gallery on the Map tab

    The map zooms to Europe.

Create a temporal profile chart

Next, you will create a chart showing pollution time series for three European countries.

  1. In the Contents pane, right-click PM25.crf, click Create Chart, and choose Temporal Profile.

    Create Chart and Temporal Profile options in the layer's context menu

    A blank chart view and the Chart Properties pane both appear.

    The temporal profile chart allows you to define areas of interest from the multidimensional raster layer and chart the time series for them. These areas can be drawn on the map or selected from another layer. You will select countries from the World Countries layer for your areas of interest.

  2. In the Chart Properties pane, under Time series ensure that Multiple locations with one variable is selected. Under Define an area of interest, click the Feature Selector button.

    Feature Selector button in the Chart Properties pane

  3. On the map, click the United Kingdom to select it. The country polygon highlights in light blue.

    United Kingdom selected on the map

    A line chart appears in the chart view, also in blue. In the Chart Properties pane, in the Define an area of interest section, a new entry appears in the table.

    First entry in the table

    You will give each area of interest a different color to make the chart easier to read.

    Note:

    If the chart does not appear, click Delete the selected row under Define an area of interest. Select the UK again.

    Delete selected row button

  4. Click the symbol and choose Mars Red (second column, third row).
  5. Change the symbol size to 10 and the label to UK.

    Area of interest with new color, size, and label

  6. On the map, click Norway. In the Chart Properties pane, change its color to Medium Apple (seventh column, third row).
  7. On the map, click Italy. Change its color to Lapis Lazuli (tenth column, fourth row).
  8. For both new entries, change the symbol size to 10. Change the labels to Norway and Italy.

    Three areas of interest defined in the Chart Properties pane and symbolized on the map

  9. At the top of the Chart Properties pane, click the Axes tab.
  10. Under the X- axis and Time Format sections, you will update the Date Format.

    For Date Format, choose the (yyyy) option (preceded by the current year). For Time Format, choose <none>.

    Date and Time Format

  11. Under Y-axis, for Bounds, change the Minimum value to 0.

    Minimum y-axis bound set to 0

  12. Click the General tab and change the Chart title to Mean Annual PM 2.5, 1998–2016.

    Updated Chart title on the General tab

  13. For X axis title, type Year. For Y axis title, type PM 2.5 (μg/m3).

    Fine particulate matter pollution (PM 2.5) is measured in micrograms per cubic meter.

    Updated text for x- and y-axes titles

    Tip:

    You can copy the μ symbol from this page into ArcGIS Pro.

    The World Health Organization (WHO) recommends that PM 2.5 levels should not be higher that 10 μg/m3. Respecting this guideline could lower air pollution related deaths by 15 percent. To complete the chart, you will add a guide to represent this maximum recommended level.

  14. Click the Guides tab and click Add guide.

    Add guide button on the Guides tab

  15. Next to Guide 1, click the Edit guide name button and edit the text to WHO PM 2.5 Guideline.
  16. For Value, type 10 in the first box and 20 in the second.

    Updated guide name and values

  17. Review your chart and map.

    Map and chart showing the countries selected in the temporal profile chart

    The chart displays a noticeable difference in PM 2.5 pollution levels for countries that are not very far apart, with Italy displaying the highest levels, UK lower levels, and Norway the lowest levels. The chart shows us that this difference is fairly consistent throughout the years.

    In 2003, the UK saw a spike in mean PM 2.5 that did not affect Norway and Italy. In 2014, the opposite was true—Italy and Norway both had spikes in mean PM 2.5, but the UK did not. Overall, from 1998 to 2016, there was not an upward or downward trend in mean PM 2.5 in Norway. However, the UK and Italy show a mild gradual decline in mean PM 2.5 during these 19 years.

    The gray shaded area on the chart identifies values that are higher than the WHO guideline of 10 μg/m3. Norway and the UK have been under the recommended maximum level for the entire time series. However, Italy has been higher than the recommended level for more than half of the 19 years. Italy contains one of the worst areas in Europe for pollution: the Po Valley in the north, where the city of Milan is located. There, the pollution is high mainly due to transportation. The Alps mountain range lies to the north, constraining the pollution to the valley, and making it difficult to remain within the WHO guideline levels.

  18. Close the chart and the Chart Properties pane.

    Close button on the chart tab

    Once you close the chart, the colored regions for UK, Italy, and Norway disappear from the map. The chart is still available and can be reopened from the Contents pane.

  19. Save the project.

In this part of the lesson, you charted PM 2.5 time series data for three countries in Europe. You leveraged ArcGIS Living Atlas to find additional data. You used a temporal profile chart to aggregate and interpret the pollution data. You found that Italy has the highest mean PM 2.5 levels of the three countries charted, and that this was consistent throughout the time series. Next, you will apply statistical tests to the PM 2.5 data, to find areas of high and low pollution on a global scale.


Perform space-time pattern mining

You’ve explored your data with a multidimensional raster layer and temporal chart. But any conclusions that you draw about patterns in the data are stronger if they are backed by objective statistical tests. Next, you’ll use the space-time cube to find statistically significant hot and cold spots of pollution in space and time. You’ll explore how patterns vary across the globe and through time. Applying statistical tests to the space and time data is important, because the human eye may see patterns that do not exist, or miss others that do.

Find hot and cold spots in a space-time cube

You’ll use the space-time cube to find hot and cold spots of pollution with the Emerging Hot Spot Analysis tool from the Space Time Pattern Mining toolbox. Then, you’ll interpret the results. You will do this analysis in the Part 3.1 map.

  1. If necessary, reopen the InvestigateGlobalPollutionPatterns project.
  2. Click the Part 3.1 map tab.

    Map of the world using the Equal Earth projection

  3. In the Geoprocessing pane, if necessary, click the Back button. Search for and open the Emerging Hot Spot Analysis tool.
  4. For Input Space Time Cube, click the Browse button.
  5. Browse to the space-time cube you saved previously. Click the file named PM25.nc and click OK.

    PM25.nc selected from the C:\Learn folder on the browse dialog box

  6. For Analysis Variable, choose PM 2.5_NONE_ZEROS. This field contains the levels of fine particulate matter, with missing values filled with zeros.
  7. For Conceptualization of Spatial Relationships, choose K nearest neighbors.
    Note:

    Many tools from the Spatial Statistics and Space Time Pattern Mining toolboxes give several choices for the conceptualization of spatial relationships in your analysis—it is important to consider what phenomenon you are trying to measure when making your choice.

  8. For Define Global Window, choose Individual time step.

    Emerging Hot Spot Analysis tool with parameters filled in

  9. Click Run.

    The Emerging Hot Spot Analysis tool finds statistically significant trends in space-time data. First, the tool uses the Getis-Ord Gi* statistic to find hot and cold spots for each year. A hot spot is not just a location with high values; it is a location with high values that is also surrounded by other high values. Similarly, a cold spot is a location with low values that is also surrounded by other low values. Once hot and cold spots have been found for each year, the Mann Kendall trend test is applied to the results to find how hot and cold spots have evolved over time. It finds eight different types of hot and cold spots, including Sporadic, Consecutive, and New. Locations without a statistically significant pattern are given the classification No Pattern Detected.

    The Emerging Hot Spot Analysis results are added to the map. There are 17 types of hot and cold spots.

    Legend showing the possible classifications from the Emerging Hot Spot Analysis tool

  10. In the Contents pane, drag the World Terrain Reference layer above the PM25_EmergingHotSpotAnalysis layer.

    That layer provides the names of world countries and other geographic locations.

  11. On the ribbon, on the Map tab, click Bookmarks and zoom to the SE Asia bookmark.

    SE Asia in the Bookmarks gallery

  12. Look at the hot spot analysis results in Southeast Asia.

    Emerging Hot Spot Analysis results for Southeast Asia on the map

    Large areas of Pakistan, India, China, and other countries are hot spots of PM 2.5. The most common trends are Persistent Hot Spot and Intensifying Hot Spot.

    • A Persistent hot spot is a location that has been a statistically significant hot spot for 90 percent of the time-step intervals with no discernible trend indicating an increase or decrease in the intensity of clustering over time. In the context of this lesson, this means a location that was a hot spot for at least 17 years, and PM 2.5 levels were not trending significantly upward or downward within this time.
    • An Intensifying hot spot is a location that has been a statistically significant hot spot for 90 percent of the time-step intervals, including the final time step. In addition, the intensity of clustering of high counts in each time step is increasing overall, and that increase is statistically significant. In the context of this lesson, this means a location that was a hot spot for at least 17 years, including 2016, and there was a statistically significant increase in the intensity of PM 2.5 at the location throughout the years.

    At the southern border of Mongolia, there is an area of cold spots, which are mainly Persistent and Consecutive. There is also a large area of Diminishing and Historical cold spots north of the Himalayan mountain range, on the Tibetan Plateau. This suggests that this area was historically low in pollution, but pollution levels have been rising. Remember that Diminishing and the other descriptive terms refer to the hot and cold spots, not the data values; the cold spot is diminishing, not the pollution.

    Note:

    You can view the full description of all the hot and cold spot types in the How Emerging Hot Spot Analysis works.

  13. On the ribbon, on the Map tab, zoom to the Ethiopia bookmark.

    Around the Ethiopian capital of Addis Ababa, there is a large Persistent hot spot, surrounded by areas of Diminishing, Sporadic, New, and Consecutive hot spots. The pollution pattern in this location is complex and indicates growth in the hot spot's size over time. You'll explore this area next.

    Hot spots in western Ethiopia

Create a 3D representation of space-time data

Next, you’ll transform the space-time cube into a 3D layer. Visualizing the data in 3D can help you better understand the pollution patterns in Ethiopia. You will do this part of the analysis in the Part 3.2 3D local scene.

  1. In the current Part 3.1 map, in the Contents pane, right-click the PM25_EmergingHotSpotAnalysis layer and click Copy.
  2. Click the Part 3.2 local scene tab.
  3. In the Contents pane, right-click Part 3.2 and choose Paste.

    The layer is added to the 2D Layers group.

    Hot spot layer copied in the 2D Layers group of the Contents pane in the Part 3.2 scene

  4. If necessary, on the ribbon, on the Map tab, zoom to the Ethiopia 3D bookmark.

    The scene displays in 3D, although for now you can't see any 3D elements.

  5. In the Geoprocessing pane, click the Back button. Search for and open the Visualize Space Time Cube in 3D tool.
  6. Enter the following Visualize Space Time Cube in 3D parameter values:
    • For Input Space Time Cube, click the Browse button. Browse to and select the space-time cube you created in the first lesson, PM25.nc.
    • For Cube Variable, choose PM 2.5_NONE_ZEROS.
    • For Display Theme, choose Value.
    • For Output Features, type PM25_3D.

    Visualize Space Time Cube in 3D tool with parameters filled in

  7. Click Run.

    A new 3D layer appears on the scene.

    PM25_3D layer in the 3D Layers group of the Contents pane

    You may receive a warning that the layer is causing an excessive amount of draw requests. You'll clip the layer to a smaller area, which will both prevent this problem and make your data easier to explore.

  8. In the Contents pane, drag the Ethiopia layer above the PM25_EmergingHotSpotAnalysis layer.

    Ethiopia layer above the PM25_EmergingHotSpotAnalysis layer

    This is the same World Countries (Generalized) layer from ArcGIS Living Atlas that you used previously, filtered to show only Ethiopia. You will clip the 3D layer to this boundary.

  9. In the Geoprocessing pane, click the Back button. Search for and open the Clip tool.
  10. Enter the following Clip parameter values:
    • For Input Features, choose PM25_3D.
    • For Clip Features, choose Ethiopia.
    • For Output Feature Class, type PM25_Ethiopia.

    Clip tool with parameters filled in

  11. Click Run.

    A new 3D layer, clipped to boundaries of Ethiopia, is added to the scene.

  12. In the Contents pane, turn off the PM25_3D layer.

    Clipped 3D layer on the scene

    The vertical columns stretching from the earth into the sky, show how the PM 2.5 values evolved over time. You'll change the symbology of the new layer to better visualize PM 2.5 values.

  13. In the Contents pane, select the PM25_ Ethiopia layer.
  14. On the ribbon, click the Appearance tab. In the Drawing group, click Import.

    The Import Symbology window appears.

  15. For Input Layer, verify that PM25_ Ethiopia is selected.
  16. For Symbology Layer, click the Browse button.
  17. Under Project, click Folders and open the InvestigatePollutionPatterns folder connection.
  18. Inside the commondata and userdata folders, select space_time_cube_symbology.lyrx, and click OK.

    space_time_cube_symbology.lyrx selected from the project's folder connection

    The other tool parameters populate automatically from the .lyrx file.

  19. For Update Symbology Ranges by Data, choose Update ranges.

    This option will update the symbology class breaks based on the range of the input dataset.

    Import Symbology tool with parameters filled in

  20. Click OK.

    The symbology updates.

    Top-down view of 3D data in Ethiopia with a red-yellow-blue color scheme

    The stacked features now show a greater color range. Red blocks are areas and times with high levels of PM 2.5. Blue blocks have low levels.

With the Visualize Space Time Cube in 3D tool, you created vertical columns showing how the PM 2.5 values evolved over time. You clipped these results to the area of interest and applied a more colorful symbology to make the differences in high and low values easier to see.

Explore hot and cold spots in 3D

For the final steps of this lesson, you’ll explore the pollution data in 3D to gain a better understanding of the hot and cold spot results.

  1. If necessary, zoom to the Ethiopia 3D bookmark. Move around the scene to explore the scene.
    Note:

    You can use the mouse wheel button to zoom in and out. You can drag to pan around the scene. You can also use the on-screen navigator.

    Oblique view of the PM25_Ethiopia layer

    These results represent each year of the time-series data, from 1998 closest to the earth, to 2016 at the top. Notice several dark purple areas—these are areas with PM 2.5 of 0. It is possible these were originally null or missing values, and when you created the space-time cube, they were filled with zeros.

  2. Find one of the Persistent hot spots (dark red) and click the block closest to the earth.

    A pop-up window appears. It shows the date window for the data you clicked and the PM 2.5 level for this date (PM 2.5_NONE_ZEROS).

    Example pop-up with a PM 2.5 value of 21.57

  3. For the same hot spot, click the block farthest from the earth.

    Example pop-up with a PM 2.5 value of 23.96

    In Persistent hot spots, the PM 2.5 value does not change significantly from the start to the end of the time series.

  4. Find one of the New hot spots and explore the PM 2.5 values for its first and last years.

    New hot spots

    You will likely see a proportionally larger increase in PM 2.5 for these locations.

  5. Look at the symbology for some Sporadic hot spots. These locations switch between pale and darker colors several times throughout the time series.

    Columns of data for sporadic hot spots

  6. Explore other types of hot and cold spots in the scene.

    Visualizing temporal data in 3D can make it easier to interpret the results of the Emerging Hot Spot Analysis tool.

You've now applied statistical methods to the pollution data in your space-time cube. You found different types of hot and cold spots of PM 2.5 throughout the world. You explored 2D analysis results to understand different types of hot and cold spot trends. You then used 3D visualization to deepen your understanding of the patterns that each hot spot represents.

In this lesson, you learned how to create, visualize, and analyze space-time data. You discovered three methods of storing temporal data: multidimensional mosaics, multidimensional raster layers, and space-time cubes. You used a temporal profile chart to compare the pollution levels of three European countries. Then you applied space-time pattern mining tools to analyze the pollution data and understand the analysis results.

To learn how to create a space-time cube from your own raster data (including .tif and NetCDF files), check out Explore your raster data with Space Time Pattern Mining. If you are interested in taking the 3D visualization of space-time cubes even farther, you might like to try the Space Time Cube Explorer Add-In for ArcGIS Pro.

You can find more lessons in the Learn ArcGIS Lesson Gallery.