Publish a feature service

In this lesson, you’ll create a hosted feature layer in ArcGIS Online to display near real-time reservoir data for Texas. First, you’ll download reservoir data for Texas. Next, you’ll use this information to publish a hosted feature layer and configure a view of it. You'll configure a Python script to update the feature layer and set up a Windows task to run it. The Texas reservoirs data feature layer will be overwritten as new data becomes available, so any web maps based on the feature view will always display the latest information.

Access near real-time reservoir data for Texas

First, you'll access reservoir information from the Water Data for Texas website. Then, you'll download the data in GeoJSON format and store it locally.

  1. In a browser, go to the Water Data for Texas website. Scroll down and under the Statewide Conservation Storage chart, click Recent Averaged Storage Conditions and click GeoJSON.

    Download GeoJSON file for Recent Averaged Storage Conditions

    The website redirects you to https://www.waterdatafortexas.org/reservoirs/statewide/recent-conditions.geojson, which contains the near real-time reservoir data for Texas. This is a GeoJSON file. For each reservoir in Texas, it contains information such as location (coordinates), name, conservation capacity, percent full, and last updated time (time stamp).

    Recent Averaged Storage Conditions in GeoJSON format

    Note:
    The Water Data for Texas website is dynamic. It is frequently updated, as soon as new data is available. The time stamp value for each reservoir displays the time of the last update.

  2. Click on the page and press Ctrl + A to select all of the text on the recent conditions page, then right-click and click Copy. You can also press Ctrl + C to copy the text.

    Copy recent conditions JSON data

  3. Open a text editor such as Notepad++ and press Ctrl + V to paste the text into a new file.
  4. Save the file to your local computer. In the Save as window, navigate to a folder on your computer such as C:\Temp\, then type recent-conditions.geojson for the file name. For Save as type choose All Files.
    Note:

    Check the file in your local folder after you save it to verify that the file extension is .geojson and not .geojson.txt.

Add an item to ArcGIS Online

You'll use the local GeoJSON file of reservoir data for Texas to create a hosted feature layer in ArcGIS Online.

  1. Sign in to your ArcGIS organizational account.
    Note:

    If you don't have an organizational account, you can sign up for an ArcGIS free trial.

  2. If necessary, click Content, and click My Content.

    Click Content, and click My Content.

  3. Click New item.

    New item button.

  4. Drag the recent-conditions.geojson file that you just saved onto the upload pane.
  5. Accept the default option to Add GeoJSON and create a hosted feature layer and click Next.
  6. On the next page of the New item pane, enter the following:

    • For Title, type TX_reservoirs_recent_ and add your name or initials.
    • For Tags, type Texas, Reservoir, Recent Conditions and press Enter after each.
    • For Summary, type Near real-time reservoir information for Texas.

  7. Click Save.

    The process will take a few moments. After it completes, the Overview page for the TX_reservoirs_recent_(your name) hosted feature layer appears. In addition to this layer, a copy of the TX_reservoirs_recent GeoJSON file is added to your My Content section.

You accessed near real-time reservoir information for Texas and created a hosted feature service in ArcGIS Online. Next, you'll use the feature service to create a feature view. This feature view can be customized, shared publicly, and added to your apps. Creating the feature view allows the feature service to be updated without disrupting the symbology and pop-ups configured for the view. This simplifies the process of updating the data later.


Create a hosted feature layer view

Now you'll create a hosted feature layer view from the hosted feature layer, share the hosted feature layer view publicly, update its symbology, and configure its pop-up.

Create the hosted feature layer view

The Texas reservoirs hosted feature layer that you created will be overwritten when new information is available. The overwrite process will remove any custom symbology and pop-up settings and restore it to the default settings. To avoid changing how the data is displayed in your web maps and apps, you will create a hosted feature layer view in ArcGIS Online and configure its symbology and pop-up settings. These will remain the same and display the latest data after the hosted feature layer is overwritten.

  1. On the TX_reservoirs_recent_(yourname) feature layer Overview page, click Create View Layer.

    Click Create View Layer.

  2. In the Create View Layer window, enter the following:

    • For Title, type Texas Reservoirs Recent Conditions (yourname)
    • For Tags, type Water, Texas, Reservoirs, Environment, Natural Resources, Live Feeds, livefeeds, pressing Enter after each.
    • For Summary, you can optionally provide additional text about the view layer.
    • For Save in folder, you can optionally specify a folder where you want to save the view layer.

  3. Click OK.

    A new hosted feature layer view item is created from the hosted feature layer and the item details page for the new feature layer view appears.

  4. On the Overview page, click Share, choose Everyone (public), and click Save.

    The hosted feature layer view is now public and accessible to everyone.

  5. Click Content.

    There are now three items in your My Content page: the GeoJSON file, the hosted feature layer, and the view.

    Three items on My Content page

Note:
It is a good practice to use a hosted feature layer view to share a hosted feature layer, keeping the original hosted feature layer private. You can change the pop-up and the symbology of the hosted feature layer view without modifying the original hosted feature layer. During an update process, the hosted feature layer can be overwritten, and the new values will be reflected in the hosted feature layer view. The symbology of the hosted feature layer view is saved with it and is not affected when the original hosted feature layer is overwritten.

Change symbology

The hosted feature layer view has the reservoir data for Texas. You'll customize its symbology to better represent the information in the layer using the reservoirs' capacity for different feature sizes and using the current percent-full level for different colors.

  1. Click the Texas Reservoirs Recent Conditions_(yourname) feature layer to open the hosted feature layer view item page.

    The item page for Texas Reservoirs Recent Conditions_(yourname) appears on the Overview tab.

  2. Click Open in Map Viewer.

    Open in Map Viewer

    Map Viewer displays the Texas Reservoirs Recent Conditions Layer. This lesson uses Map Viewer. Depending on your organization's settings, the older version of map viewer may have opened. If this is the case, click Open in new Map Viewer in the upper-right corner of the map.

  3. If necessary, in the Layers pane, click the Texas Reservoirs Recent Conditions layer to activate it.
  4. On the Settings (light) toolbar, click Styles.

    Click Change Style.

  5. Under Choose attributes click the + Field button.

    Choose the percent_full attribute.

  6. In the Add fields window that appears, choose percent_full and conservation_capacity and click Add.

    Two attributes, percent_full and conservation_capacity

    The attributes are added to the Styles pane.

  7. Drag percent_full above conservation_capacity.

    Drag the percent_full value attribute above the conservation_capacity attribute.

  8. Under Try a drawing style, for Color and Size, click Style options.

    Click Options.

  9. For Counts and Amounts (Color), click Style options.

    Click Options.

  10. In the Style options pane, on the histogram, click the minimum value break, type 50 and press Enter.

    Set the minimum valye break to 50.

  11. Click the color ramp under Symbol style.

    Click the symbol style

  12. In the Symbol style pane that opens, underFill, click the blue-gray color ramp (it is named Blue 10), and click OK.

    Click the Fill tab and blue-gray color ramp.

    To see the name of a color ramp, point to the color ramp.

  13. In the Style options pane, click Done.
  14. For Counts and Amounts (size), click Style options.

    Click Options.

  15. Click the symbol under Symbol style. In the Symbol style pane that appears, under Outline, expand Custom Color and type #283849 and press Enter.

    Set a new custom color.

  16. On the Style options pane, click Done, and click Done three more times.

    The symbology for the layer is now set. Next you'll configure the map and save it.

  17. On the Contents (dark) toolbar and click theSave and open button and click Save.

    Save the map.

  18. On the Contents toolbar, click Basemap and click Light Gray Canvas.

    Choose the light grey canvas basemap.

    The symbology for the layer and basemap are now set.

    Map with symbology set.

    The final map shows the recent conditions of reservoirs in Texas. The color of each point represents the percentage full of each reservoir, and the size of the point represents the total reservoir capacity.

Configure the pop-up

You will improve the pop-up for the layer to display only the relevant information. You'll show the name, capacity, and percent full for each reservoir.

  1. On the Settings toolbar and click Configure Pop-ups.

    Click More Options and click Configure Pop-up.

  2. On the Pop-ups pane, expand Title, delete the existing text and click the Add Field button.
    Add title buttons

    The Add field window appears.

  3. From the drop-down, choose full_name and click OK.

    The pop-up title will now show the contents of the attribute full_name.

  4. On Fields list, click the Options button and choose Delete.

    Delete the fields list.

  5. Click the Add Content button and click Text

    Choose A custom attribute display.

    Click Configure.

    A text box appears.

  6. In the text box, type {percent_full}% full as of {timestamp} and click OK.

    Configure Custom Attribute Display.

    The attribute field names in curly braces will be replaced with the values from those fields when the pop-up is opened by clicking a feature.

    The pop-up will now show the title and text you configured when a feature is clicked.

    Example pop-up.

    You can still improve the format of the fields by configuring how each of the fields is displayed.

  7. On the Settings toolbar, click the Configure Fields tab and click the percent_full field.
  8. Change the Display name to Percent Full (%), and in the Significant Digits drop-down menu, choose 0 Decimal Places.

    Set the alias and the decimal places.

  9. Click Done.
  10. On the Contents toolbar, click Save and open and click Save.
  11. Click a reservoir on the map.

    The pop-up displays the name of the reservoir, the percent full, and the date and time last updated.

You've created a hosted feature layer view in ArcGIS Online from the reservoirs hosted feature layer. The hosted feature layer view allows you to customize the display of the Texas reservoir layer and retain these customizations even when the hosted feature layer is overwritten. Next, you'll overwrite the Texas reservoirs hosted feature layer using a simple Python script. The updated information will be reflected automatically in the hosted feature layer view.


Update a hosted feature layer

Now you'll configure a short script to overwrite the Texas reservoirs hosted feature layer. First, you'll download the overwrite script from ArcGIS Online. Then, you'll retrieve the hosted feature layer parameters such as item title and item ID. Finally, you'll run the script locally using the Python Command Prompt. The overwrite process allows you to update the hosted feature layer with the latest information.

Download the overwrite script

First, download the script for overwriting a hosted feature layer in ArcGIS Online.

  1. Go to http://esriurl.com/OverwriteFS and click Download. Save the OverwriteFS.zip file to your computer.

    The .zip file contains two files: the Python script (OverwriteFS.py) and a document (ReadMe.pdf).

  2. Extract the contents of the .zip file to a folder on your computer, for example, C:\Temp\.
  3. Open a text editor such as Notepad or Notepad++ to a new, blank text file.
  4. In ArcGIS Online, under My Content, click the TX_reservoirs_recent_(yourname) hosted feature layer.

    You will need to know the title and the item ID for this hosted feature layer in order to update it. You will save this information in a text document for reference.

  5. On the TX_reservoirs_recent item Overview page, copy the item title and paste it into the text file.

    Copy the item title and item ID.

  6. Copy the item ID at the end of the item's URL and paste it into your text file. Save the file, but keep it open, you will need this information soon.

    In this example, the item title is TX_reservoirs_recent_(yourname) and the item ID is 4309a6d6bb77477f986acba13b5f9ef6. The item ID is unique for each item in ArcGIS Online. Your item ID will be different.

Store ArcGIS Online credentials locally

ArcGIS Online requires login credentials to overwrite items. You'll store ArcGIS credentials locally in a secure profile. This profile will be used in the Python script to overwrite the reservoirs hosted feature layer.

  1. Open the Windows Start menu, search for Python Command Prompt, and open it.
  2. Type cd and type the path to the folder where you extracted the OverwriteFS.py script and press Enter.

    For example, if you extracted the script to C:\Temp, you would type cd C:\Temp.

    Python command prompt

  3. Type python and press Enter.

    Python console running in the Python command prompt.

    This starts the Python console in the Python Command Prompt window. The Python version is noted in the console and the input prompt changes to >>>.

    Next you will import the arcgis Python module.

  4. In the Python console, type import arcgis and press Enter.

    Import the arcgis module.

    Note:
    The Python command prompt and the arcgis Python module are included with the ArcGIS Pro installation.

    Now you will store your ArcGIS Online credentials locally by adding a profile.

  5. Type the command gis = arcgis.GIS(profile='learn_user', username= '<account>', password= '<pass>'), replace <account> with your user name account and <pass> with your password. Press Enter.

    Type the command to store a user name and password locally in a profile.

    Note:
    Your ArcGIS Online credentials will be securely saved in the learn_user profile. You can learn more about storing ArcGIS Online credentials locally in a profile by visiting the Working with different authentication schemes page.

  6. Type exit() to close the Python console and return to the command prompt.

Overwrite the hosted feature layer

Now use the overwrite Python script, the ArcGIS Online profile, and the hosted feature layer parameters (item ID and title) to update the reservoirs hosted feature layer.

  1. In the Python command prompt, in the C:\Temp\ folder that contains the OverwriteFS.py script, run the following command: python OverwriteFS.py learn_user <item id> <item title> <GeoJSON URL>, replacing <item id> and <item title> with the parameters of the hosted feature layer you saved in your text file.

    In this code, learn_user is the ArcGIS Online profile saved locally. Replace <item id> and <item title> with the parameters of the hosted feature layer that you saved in your text file. In the example, you would replace <item id> with 3b2b385c923147349dec02025e10aa55 and <item title> with TX_reservoirs_recent_(yourname). You would replace <GeoJSON URL> is the URL of the data source with the updated data, https://www.waterdatafortexas.org/reservoirs/statewide/recent-conditions.geojson.

    Note:
    Because you've already created the hosted feature layer using the most recent data, the script won’t make any changes. Instead, you can temporarily use https://downloads.esri.com/learnarcgis/schedule-automated-near-real-time-data-updates/recent-conditions.geojson, which contains a snapshot of the recent-conditions.geojson file for December 26, 2019.

    If you run it with the URL to the live Texas water site, the command will resemble the following (the item ID will be different):

    python OverwriteFS.py learn_user 20af483d68e54ff5835753de0e90ce27 TX_reservoirs_recent_(yourname) https://www.waterdatafortexas.org/reservoirs/statewide/recent-conditions.geojson

    Press Enter to run the command.

    The command prompt displays the results. The Performing Overwrite… Success! Elapsed Time message at the end means that the script correctly updated the hosted feature layer.

    Message showing that the overwrite process completed successfully.

  2. In a browser, open the Texas Reservoirs recent conditions hosted feature layer view in Map Viewer. Click any reservoir to view its pop-up.

    The pop-up shows the updated time and percent-full values. The symbology of the hosted feature layer view remains as you set it.

You ran a Python script to overwrite a hosted feature layer in ArcGIS Online. Next, you'll create and schedule a task in Windows to automatically run the overwrite script at regular intervals.


Create and schedule a task

You ran a Python script to update a hosted feature layer. Now, you'll automate running the script by scheduling a task in Windows. First, you'll create a .bat file. Then, you'll create a task in Windows Task Scheduler. The task will regularly run the .bat file, which will run the overwrite Python script, which will check for updates and overwrite the hosted feature layer if new data is available.

Create a .bat file

Windows .bat files are simple text files with a set of instructions to be run by the command line interpreter.

  1. Open a text editor such as Notepad or Notepad++.
  2. Copy and paste the following text into the text editor:

    "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe" "C:\Temp\OverwriteFS.py" learn_user 3b2b385c923147349dec02025e10aa55  TX_reservoirs_recent_(yourname) https://www.waterdatafortexas.org/reservoirs/statewide/recent-conditions.geojson

    You can also type the text it as a single line.

  3. Change the item ID from 3b2b385c923147349dec02025e10aa55 to the item ID of your hosted feature layer.
  4. Change TX_reservoirs_recent_(yourname) to the name of your item.
  5. Save the file in your C:\Temp\ folder, and type TXreservoirs_overwrite.bat for the file name. In the Save as type drop-down menu, choose All Files (or if you are using Notepad++, choose Batch file).

    TXreservoirs_overwrite.bat file

    Next, you'll create a task to run the .bat file.

Create a basic task

Now that you have a .bat file, you’ll schedule a task to run it at regular intervals.

  1. Open the Windows Start menu. Search for and open Task Scheduler.
  2. In the Actions panel, click Create Basic Task.

    Create Basic Task

    The Create Basic Task Wizard window appears and starts on the Create a Basic Task tab.

  3. In the Create Basic Task Wizard window, for Name, type TX reservoirs, and for Description type Task to run the TXreservoirs_overwrite.bat file to update the feature service.

    Type the task name and description.

  4. Click Next.

    The window updates to the Trigger tab.

  5. For When do you want the task to start?, confirm Daily is selected and click Next.

    Choose Daily.

    The wizard updates to the Daily tab.

  6. For Start date, choose today’s date. Set the start time to be one hour after the current time (4:35 p.m. in the example).

    Task date and interval

  7. For Recur every, type 1 to make the task run every day and click Next.
  8. On the Action page, confirm that Start a Program is selected as the task action and click Next.
  9. On the Start a Program page, in the Program/script text box, type C:\Temp\TXreservoirs_overwrite.bat and click Next.

    Start the .bat file.

    If you saved the .bat file in another folder, ensure you enter the correct path to the .bat file. You can also click the browse button and browse to the location of the file.

    The Finish tab appears in the wizard.

  10. Check the box for Open the properties dialog for this task when I click Finish box and click Finish.

    Select Open the properties dialog for this task when I click Finish.

  11. Under Security options, check the box for Run whether user is logged on or not and check Do not store password. Click OK.

    Run when not logged in and do not store passwords.

  12. The task will run at the scheduled time, updating the hosted feature layer with the latest information from the Water Data for Texas website. The hosted feature layer view will show the updated information from the hosted feature layer while retaining the same symbology and pop-up configuration.

In this lesson, you downloaded data from the Water Data for Texas website and created a hosted feature layer. You created, customized, and shared a hosted feature layer view of the hosted feature layer to isolate the public item from the update process. You adapted, downloaded, and used a Python script to obtain the latest data from the website and update the hosted feature layer. You also created a .bat file and scheduled a Windows task to run the Python script. When the task runs, the overwrite Python script will update the hosted feature layer. You configured the task to run daily, so any web maps or apps that display the hosted feature layer view will show the latest data each day.