Publish a feature service

In this lesson, you’ll create a hosted feature layer in ArcGIS Online to display near real-time reservoir data for Texas. First, you’ll download reservoir data for Texas. Next, you’ll use this information to publish a hosted feature layer and configure a view of it. You'll configure a Python script to update the feature layer and set up a Windows task to run it. The Texas reservoirs data feature layer will be overwritten as new data becomes available, so any web maps based on the feature view will always display the latest information.

Access near real-time reservoir data for Texas

First, you'll access reservoir information from the Water Data for Texas website. Then, you'll download the data in GeoJSON format and store it locally.

  1. In a browser, go to the Water Data for Texas website. Scroll down and under the Recent Averaged Storage Conditions drop-down menu, click GeoJSON.

    Download GeoJSON file for Recent Averaged Storage Conditions

    The website redirects you to https://www.waterdatafortexas.org/reservoirs/statewide/recent-conditions.geojson, which contains the near real-time reservoir data for Texas. This is a GeoJSON file. For each reservoir in Texas, it contains information such as location (coordinates), name, conservation capacity, percent full, and last updated time (time stamp).

    Recent Averaged Storage Conditions in GeoJSON format

    Note:
    The Water Data for Texas website is dynamic. It is frequently updated, as soon as new data is available. The time stamp value for each reservoir displays the time of the last update.

  2. Select all of the text on the recent conditions website (press Ctrl + A) and copy it (press Ctrl + C).

    Copy recent conditions JSON data

  3. Open a text editor such as Notepad or Notepad++ and paste the text (press Ctrl + V). Save the file to your local computer (for example, in C:\Temp\). Type recent-conditions.geojson for the file name and choose All Files in the Save as type drop-down list.

Add an item to ArcGIS Online

You'll use the local GeoJSON file of reservoir data for Texas to create a hosted feature layer in ArcGIS Online.

  1. Sign in to your ArcGIS organizational account.
    Note:

    If you don't have an organizational account, you can sign up for an ArcGIS free trial.

  2. Click Content, and click My Content.

    Click Content, and click My Content.

  3. Click Add Item and click From your computer.

    Add item from your computer.

  4. Click Choose File and browse to the recent-conditions.geojson file that you just saved. Check the Publish this file as a hosted layer box.
  5. In the Title box, type TX_reservoirs_recent_ and add your name or initials.
  6. Add tags such as Texas, Reservoir, Recent Conditions, press Enter, and click Add Item.

    The process will take a few moments. After it completes, the Overview page for the TX_reservoirs_recent_(your name) hosted feature layer opens. In addition to this layer, a copy of the TX_reservoirs_recent GeoJSON file is added to your My Content section.

You accessed near real-time reservoir information for Texas and created a hosted feature service in ArcGIS Online. Next, you'll use the feature service to create a feature view. This feature view can be customized, shared publicly, and added to your apps. Creating the feature view allows the feature service to be updated without disrupting the symbology and pop-ups configured for the view. This simplifies the process of updating the data later.


Create a hosted feature layer view

Now you'll create a hosted feature layer view from the hosted feature layer, share the hosted feature layer view publicly, update its symbology, and configure its pop-up.

Create the hosted feature layer view

The Texas reservoirs hosted feature layer that you created will be overwritten when new information is available. The overwrite process will remove any custom symbology and pop-up settings and restore it to the default settings. To avoid changing how the data is displayed in your web maps and apps, you will create a hosted feature layer view in ArcGIS Online and configure its symbology and pop-up settings. These will remain the same and display the latest data after the hosted feature layer is overwritten.

  1. On the TX_reservoirs_recent_(yourname) feature layer Overview page, click Create View Layer.

    Click Create View Layer.

  2. In the Create View Layer window, type the title Texas Reservoirs Recent Conditions (yourname) and the tags Water, Texas, Reservoirs, Environment, Natural Resources, Live Feeds, livefeeds. Optionally provide a summary and select a folder for the view layer. Click OK.

    A new hosted feature layer view item is created from the hosted feature layer.

  3. On the Overview page, click Share, choose Everyone (public), and click Save.

    The hosted feature layer view is now public and accessible to everyone.

    There are now three items on the My Content page: the GeoJSON file, the hosted feature layer, and the view.

    Three items on My Content page

Note:
It is a good practice to use a hosted feature layer view to share a hosted feature layer, keeping the original hosted feature layer private. You can change the pop-up and the symbology of the hosted feature layer view without modifying the original hosted feature layer. During an update process, the hosted feature layer can be overwritten, and the new values will be reflected in the hosted feature layer view. The symbology of the hosted feature layer view is saved with it and is not affected when the original hosted feature layer is overwritten.

Change symbology

The hosted feature layer view has the reservoir data for Texas. You'll customize its symbology to better represent the information in the layer using the reservoirs' capacity for different feature sizes and using the current percent-full level for different colors.

  1. Click the Texas Reservoirs Recent Conditions_(yourname) feature layer to open the hosted feature layer view item description.
  2. On the Overview page, click Open in Map Viewer.

    Open in Map Viewer

    Map Viewer displays the Texas Reservoirs Recent Conditions Layer.

  3. In the Content pane, click Change Style.

    Click Change Style.

  4. Click the Choose an attribute to show drop-down arrow and click percent_full.

    Choose the percent_full attribute.

  5. Click Add attribute and choose conservation_capacity.

    Two attributes, percent_full and conservation_capacity

  6. In the Select a drawing style section, under Color and Size, click Options.

    Click Options.

  7. In the Counts and Amounts (Color) section, click Options and Counts and Amounts (Color).

    Click Options.

  8. Change the minimum value to 50 and click Symbols.

    Set the minimum value to 50 and click Symbols.

  9. Click the Fill tab, click the blue-gray color ramp, and click OK.

    Click the Fill tab and blue-gray color ramp.

  10. In the Counts and Amounts (Size) section, click Options, and click Symbols.

    Click Options.

    Click Symbols.

  11. Click the Outline tab, and in the Custom Hexadecimal Color box, type #283849.

    Set a custom color.

  12. Click OK and click OK again.
  13. Click Done and click Done again.

    The symbology for the layer is now set. You'll save the layer.

  14. Click More options and click Save Layer.

    Click More options, and click Save Layer.

  15. Click Basemap and click the Light Gray Canvas basemap.

    Choose the Light Gray Canvas basemap.

    The symbology for the layer and basemap are now set.

    The map symbology is now set.

    The final map shows the recent conditions of reservoirs in Texas, the percent full (color), and the conservation capacity (size).

Configure the pop-up

You will improve the pop-up for the layer to display only the relevant information. You'll show the name, capacity, and percent full for each reservoir.

  1. Click More options and click Configure Pop-up.

    Click More Options and click Configure Pop-up.

  2. In the Display drop-down list, choose A custom attribute display and click Configure.

    Choose A custom attribute display.

    Click Configure.

  3. In the Custom Attribute Display window, for the first line, type {full_name}, set the font size to medium, and make the text bold.
  4. For the second line, type {percent_full}% full as of {timestamp}, set the font size to small, remove any bold from the text, and click OK.

    Configure Custom Attribute Display.

    The attribute field names in curly braces will be replaced with the values from those fields when the pop-up is opened by clicking a feature.

  5. Click Configure Attributes.

    Click Configure Attributes.

  6. In the Configure Attributes window, choose the percent_full field, change the alias to Percent Full (%), and in the Format drop-down menu, choose 0 decimal places.

    Set the alias and the decimal places.

  7. Click OK, and click OK again.
  8. Click a reservoir on the map.

    The pop-up displays the name of the reservoir, the percent full, and the date and time last updated.

  9. Click More options, and click Save Layer.

You've created a hosted feature layer view in ArcGIS Online from the reservoirs hosted feature layer. The hosted feature layer view allows you to customize the display of the Texas reservoir layer and retain these customizations even when the hosted feature layer is overwritten. Next, you'll overwrite the Texas reservoirs hosted feature layer using a simple Python script. The updated information will be reflected automatically in the hosted feature layer view.


Update a hosted feature layer

Now you'll configure a short script to overwrite the Texas reservoirs hosted feature layer. First, you'll download the overwrite script from ArcGIS Online. Then, you'll retrieve the hosted feature layer parameters such as item title and item ID. Finally, you'll run the script locally using the Python Command Prompt. The overwrite process allows you to update the hosted feature layer with the latest information.

Download the overwrite script

First, download the script for overwriting a hosted feature layer in ArcGIS Online.

  1. Go to http://esriurl.com/OverwriteFS and click Download. Save the OverwriteFS.zip file to your computer.

    The .zip file contains two files: the Python script (OverwriteFS_v1.4.1.py) and a document (ReadMe.pdf).

  2. Extract the contents of the .zip file to a folder on your computer, for example, C:\Temp\.
  3. Open a text editor such as Notepad or Notepad++ to a new, blank text file.
  4. In ArcGIS Online, under My Content, click the TX_reservoirs_recent_(yourname) hosted feature layer.

    You will need to know the title and the item ID for this hosted feature layer in order to update it. You will save this information in a text document for reference.

  5. On the Overview page, copy the item title and paste it into the text file.

    Copy the item title and item ID.

  6. Copy the item ID at the end of the item's URL, and paste it into your text file. Save the file, but keep it open, you will need this information soon.

    In this example, the item title is TX_reservoirs_recent_(yourname) and the item ID is 4309a6d6bb77477f986acba13b5f9ef6. The item ID is unique for each item in ArcGIS Online. Your item ID will be different.

Store ArcGIS Online credentials locally

ArcGIS Online requires login credentials to overwrite items. You'll store ArcGIS credentials locally in a secure profile. This profile will be used in the Python script to overwrite the reservoirs hosted feature layer.

  1. Open the Windows Start menu, search for Python Command Prompt, and open it. Browse to the folder where you extracted the OverwriteFS_v1.4.1.py script (for example, C:\Temp).

    Note:

    Go to the folder. Type cd C:\Temp and press Enter.

    Python command prompt

  2. Type python and press Enter.

    Python console running in the Python command prompt.

    This starts the Python console in the Python Command Prompt window. The Python version is noted in the console and the input prompt changes to >>>.

  3. In the Python console, type import arcgis and press Enter.

    Import the arcgis module.

    Note:
    The Python command prompt and the arcgis Python module are included with the ArcGIS Pro installation.

    Now you will store your ArcGIS Online credentials locally by adding a profile.

  4. Type the command gis = arcgis.GIS(profile='learn_user', username= '<account> ', password= '<pass>'), replace <account> with your user name account and <pass> with your password. Then, press Enter.

    Type the command to store a user name and password locally in a profile.

    Note:
    Your ArcGIS Online credentials will be securely saved in the learn_user profile. You can learn more about storing ArcGIS Online credentials locally in a profile by visiting the Working with different authentication schemes page.

  5. Type exit() to close the Python console and return to the command prompt.

Overwrite the hosted feature layer

Now use the overwrite Python script, the ArcGIS Online profile, and the hosted feature layer parameters (item ID and title) to update the reservoirs hosted feature layer.

  1. In the Python command prompt, in the C:\Temp\ folder that contains the OverwriteFS_v1.4.1.py script, run the following command: python OverwriteFS_v1.4.1.py learn_user <item id> <item title> <GeoJSON URL>.

    In this code, learn_user is the ArcGIS Online profile saved locally. Replace <item id> and <item title> with the parameters of the hosted feature layer that you saved in your text file (in the example, 3b2b385c923147349dec02025e10aa55 and TX_reservoirs_recent_(yourname), respectively), and <GeoJSON URL> is the URL of the data source with the updated data.

    Note:
    The GeoJSON URL should be the link from the Water Data for Texas website (https://www.waterdatafortexas.org/reservoirs/statewide/recent-conditions.geojson), but because you've already created the hosted feature layer using the most recent data, the script won’t make any changes. Instead, you can temporarily use https://downloads.esri.com/learnarcgis/schedule-automated-near-real-time-data-updates/recent-conditions.geojson, which contains a snapshot of the recent-conditions.geojson file for December 26, 2019.

    If you run it with the URL to the live Texas water site, the command will resemble the following (the item ID will be different):

    python OverwriteFS_v1.4.1.py learn_user 20af483d68e54ff5835753de0e90ce27 TX_reservoirs_recent_(yourname) https://www.waterdatafortexas.org/reservoirs/statewide/recent-conditions.geojson

    The command prompt displays the results. The Performing Overwrite… Success! Elapsed Time message at the end means that the script correctly updated the hosted feature layer.

    Message showing that the overwrite process completed successfully.

  2. Open the hosted feature layer view in Map Viewer. Click any reservoir.

    The symbology is correct and the updated time and percent-full values appear in the pop-up.

You ran a Python script to overwrite a hosted feature layer in ArcGIS Online. Next, you'll create and schedule a task in Windows to automatically run the overwrite script at regular intervals.


Create and schedule a task

You ran a Python script to update a hosted feature layer. Now, you'll automate running the script by scheduling a task in Windows. First, you'll create a .bat file. Then, you'll create a task in Windows Task Scheduler. The task will regularly run the .bat file, which will run the overwrite Python script, which will check for updates and overwrite the hosted feature layer if new data is available.

Create a .bat file

Windows .bat files are simple text files with a set of instructions to be run by the command line interpreter.

  1. Open a text editor such as Notepad or Notepad++.
  2. Select the text below and press Ctrl+C to copy it, and paste it (press Ctrl+V) into the text editor:

    "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe" "C:\Temp\OverwriteFS_v1.4.1.py" learn_user 3b2b385c923147349dec02025e10aa55  TX_reservoirs_recent_(yourname) https://www.waterdatafortexas.org/reservoirs/statewide/recent-conditions.geojson

    You can also type it as a single line.

  3. Change the item ID from 3b2b385c923147349dec02025e10aa55 to the item ID of your hosted feature layer.
  4. Save the file in your C:\Temp\ folder, and type TXreservoirs_overwrite.bat for the file name. In the Save as type drop-down menu, choose All Files.

    TXreservoirs_overwrite.bat file

    Next, you'll create a task to run the .bat file.

Create a basic task

Now that you have a .bat file, you’ll schedule a task to run it at regular intervals.

  1. Open the Windows Start menu. Search for and open the Task Scheduler.
  2. In the Actions panel, click Create Basic Task.

    Create Basic Task

  3. Type TX reservoirs in the Name text box, and in the Description text box, type Task to run the TXreservoirs_overwrite.bat file to update the feature service. Click Next.

    Type the task name and description.

  4. Choose Daily as the task running frequency and click Next.

    Choose Daily.

  5. Choose today’s date as the start date. The start time can be one hour after the current time (4:35 p.m. in the example).

    Task date and interval

  6. Type 1 in the Recur every text box to make the task run every day and click Next.
  7. Choose Start a Program as the task action and click Next.
  8. In the Program/script text box, type C:\Temp\TXreservoirs_overwrite.bat and click Next.

    Start the .bat file.

  9. Select the Open the properties dialog for this task when I click Finish box and click Finish.

    Select Open the properties dialog for this task when I click Finish.

  10. Click the Run whether user is logged on or not option and select Do not store password. Click OK.

    Run when not logged in and do not store passwords.

  11. The task will run at the scheduled time, updating the hosted feature layer with the latest information from the Water Data for Texas website. The hosted feature layer view will show the updated information from the hosted feature layer while retaining the same symbology and pop-up configuration.

In this lesson, you downloaded data from the Water Data for Texas website and created a hosted feature layer. You created, customized, and shared a hosted feature layer view of the hosted feature layer to isolate the public item from the update process. You adapted, downloaded, and used a Python script to obtain the latest data from the website and update the hosted feature layer. You also created a .bat file and scheduled a Windows task to run the Python script. When the task runs, the overwrite Python script will update the hosted feature layer. You configured the task to run daily, so any web maps or apps that display the hosted feature layer view will show the latest data each day.