Prepare data and publish a web layer
In this first module, you will make two database connections, one to the DSNY geodatabase which stores the department legacy data and one database connection to the vendors database. You will then prepare the legacy waste data for publishing and connect to your organization portal from ArcGIS Pro. Finally, you will share the waste data with your ArcGIS Enterprise organization by referencing a registered data store.
Note:
If the components are not yet installed, review the Base ArcGIS Enterprise deployment documentation and the tutorial Get started with ArcGIS Enterprise Builder. You should also check the system requirements for SQL Server, or the database platform that you are using, and the system requirements for ArcGIS Server.
Access data from the enterprise geodatabase
First, you will connect to two databases as the data owner who can load data. Then you will copy the waste data from the local file geodatabase into the enterprise geodatabases. Throughout the tutorial, you will follow this general workflow for publishing data by reference.
If you don’t have access to an existing enterprise geodatabase, you must create two before starting the tutorial. To learn how to create and deploy an enterprise geodatabase, follow the steps from the Deploy an enterprise geodatabase for real estate tutorial.
Note:
This tutorial illustrates how to share data from an enterprise geodatabase using SQL Server as the database management system. You can complete this tutorial by using another supported DBMS such as Oracle, PostgreSQL, IBM Db2, and SAP HANA.
- Download the DSNY project package.
- Browse to where the project downloaded to and double-click the DSNY_Project.ppkx file to open it in ArcGIS Pro.
The project opens with a map displaying the Light Gray Canvas basemap and is centered on New York City. Next, you will connect to an enterprise geodatabase so you can add the waste data to it.
- In the Catalog pane, right-click the Databases folder and click New Database Connection.
The Database Connection dialog box appears. You will enter parameters to connect to an enterprise geodatabase that is available to you.
- On the Database Connection dialog box, enter the following parameters:
- For Database Platform, choose the correct database that your organization uses, such as SQL Server.
- For Instance, enter the location in which the database resides.
- For Authentication Type, choose Database authentication or Operating system authentication.
- For User Name, type the database username you want to connect with.
- For Password, type the password for the above database user.
- For Save User/Password, keep the box checked.
- For Database, type the name of the database you want to connect to, such as DSNY_DB, or choose an available database from the list.
Note:
The tool parameters you enter will be different than what is shown in the example image. Also, you can use either database or operating system authentication.
- Click the Validate button to validate the connection properties and click OK.
The database connection appears in the Catalog pane. This enterprise geodatabase will act as the primary geodatabase of the department which will store the legacy waste data.
Next, you’ll create another database connection. This other database connection will store the vendors data.
- Right-click the Database folder and click New Database Connection.
- For Database Platform, choose the correct database that your organization uses, such as SQL Server.
- For Instance, enter the location in which the database resides. For SQL Server, use the SQL Server instance name, DSNYSQL.
- For Authentication Type, choose Database authentication.
- For User Name, type the database username you want to connect with.
- For Password, type the password for the above database user.
- For Save User/Password, keep the box checked.
- For Database, type the name of the database you want to connect to, such as NYC_Vendors, or choose an available database from the list.
Now you have two connections to enterprise geodatabases, one for the legacy waste data and one for vendor data. Notice the naming convention for the enterprise database connections is DatabasePlatform-Instance-DatabaseName(username).sde. Whenever you make a connection, the name will have this convention. For simplicity sake in the tutorial, you will rename the connections to be shorter and easier to distinguish.
- Right-click the connection to the DSNY database and choose Rename.
The name becomes editable.
- Type DSNY and press Enter.
- In the same way, change the name of the NYC_Vendors connection to NYC Vendors.
The connections are renamed and still have the .sde extension, which distinguishes them as an enterprise geodatabase. Notice there is another geodatabase called wastedata.gdb. This is a file geodatabase and next, you will copy data from it into the enterprise database connections.
- In the Catalog pane, expand the wastedata.gdb geodatabase.
The wastedata.gdb file geodatabase contains five feature classes.
- Commercial_waste_zones—A polygon feature class that represents the boundary of the DSNY operational zones.
- Disposal_vendor—A point feature class that represents the vendors locations used to discard various collected materials (refuse, paper, metal/glass/plastic, organics, construction, and demolition).
- Special_waste_sites—A point feature class that represents the locations of DSNY special waste drop-off sites.
- Storage_garages—A point feature class that represents the locations of DSNY garages used to perform emergency responses.
- Vendor_pickup_frequencies—A polygon feature class that represents the frequency boundaries for collection operation (refuse, recycling, organics, bulk items) performed by the collection vendors.
You will first load the legacy data from the file geodatabase into the DSNY enterprise geodatabase
- In the Catalog pane, press the Ctrl key and click the Commercial_waste_zones, Special_waste_sites, and Storage_garages feature classes to select them.
- Right-click the selected feature classes and choose Copy.
- Right-click the DSNY database connection and click Paste.
Note:
Depending on your connection and bandwidth, the copy operation may take several minutes.
- When the operation finishes, right-click the database connection and choose Refresh.
Notice how the database name and username are included in the feature class names. This makes it easier to indicate the data owner of these feature classes.
Next, you will copy the vendor’s feature classes into the NYC Vendors database.
- In the Catalog pane, select the Disposal_vendor and Vendor_pickup_frequencies feature classes and copy and paste them into the NYC Vendors database connection.
- Refresh the NYC Vendors connection to view the feature classes.
You created two enterprise database connections and copied data from file geodatabases into them. In just a few steps, you have populated the enterprise geodatabases with data that you will use for editing and web mapping purposes.
Connect to the enterprise portal
In this section, you will connect to your Enterprise portal in ArcGIS Pro and sign in to it so you can publish a web layer.
Note:
Confirm the privileges your portal user needs to complete this tutorial by reviewing the requirements for user types, member roles, and privileges granted to roles.
- In the upper right corner of the screen, click the Log In menu and choose Manage Portals.
- On the Portals page, click Add Portal.
Depending on the portals, if any, that you are connected to, you may see other portals listed. If you already see the portal listed, you can skip the step to add the portal and proceed to making it the active portal (if necessary).
- On the Add Portal dialog box, type the URL for your portal and click OK.
Note:
The URL's format is https://webadapter.domain.com/arcgis.
The portal you added appears in the list.
- If necessary, in the portal list, right-click the URL and click Set As Active Portal to make the new portal connection your active portal.
You have added the portal and set it as the active portal. Next, you will sign in to it.
- Right-click again the portal you just added and click Sign in.
Note:
To successfully complete this tutorial, make sure you are connecting to a portal account that has publishing capabilities.
- After you successfully sign in, click the back arrow to return to the project.
In the upper right corner of the project, the name of the portal you are connected to is listed.
- On the Quick Access toolbar, click the Save button.
Next, you will add the legacy data to the map and prepare for publishing.
Symbolize layers using layer files
Now that you have connected to the enterprise databases and copied data into them, you will add the waste data to the map, apply appropriate symbology, and share the data as a web feature layer to Enterprise.
- From the Catalog pane, drag all the feature classes from the DSNY database connection onto the DSNY Map.
The layers appear on the map and are symbolized with default colors and symbols. You will change the layer’s symbology to better represent the features and attributes in the map. First, you will rename all the layers to remove the underscores.
- In the Contents pane, click the Commercial_waste_zones layer to select it. Click it again to make the name editable.
- Remove the underscores so the name is Commercial waste zones and click anywhere off the layer.
- Following the same steps, change the names of the Special_waste_sites and Storage_garages layers to remove the underscores.
Now, you will use layer files created for each layer that are included in the project.
- In the Contents pane, click the Commercial waste zones layer. On the ribbon, click the Feature Layer tab and click Import.
The Import Symbology window appears.
- In the Import Symbology window, for Symbology Layer, click the Browse button.
- Under Project, click Folders, expand DSNY_Project/commondata/userdata, click the Commercial_waste_zones_symbology.lyrx file.
- Click OK. In the Import Symbology window, click OK.
- Apply layer files to the other two layers by importing the corresponding layer file from the userdata folder to each.
Now the layers are symbolized in a more visually pleasing manner and ready to be published.
Share data as a referenced web feature layer
Next, you will share the waste data as web layers to Enterprise. To take advantage of the complete web editing workflow in which the web edits are referenced back to the database, without making a copy of the source data, you will share the waste data by referencing a registered data store.
- On the ribbon, click the Share tab and click Web Layer.
The Share As Web Layer pane appears. Here, you can enter the parameters for the web layer and analyze it for errors before publishing.
- In the Share As Web Layer pane, enter the following information:
- For Name, type DSNY Waste Management.
- For Summary, type This web layer represents the legacy dat of the Department of Sanitation in NYC.
- For Tags, type DSNY, waste management, NYC, recycling, and Esri, pressing the Tab key after each.
Next, you’ll complete the Data and Layer Type information. To share data that references registered data, a map image layer is automatically included. To support feature querying, visualization, and editing, you must also enable the Feature option. This will create a web feature layer and a map image layer in your portal.
- For Data and Layer Type, under Reference registered data, check the box for Feature.
- In the Location section, for Portal Folder, click the drop-down menu and choose Creare new folder. Type DSNY for the folder name.
You will use this folder to store all the waste management layers.
- Under Share with, check the option that represents your enterprise organization.
Note:
When selecting to share content with your organization, the check box will have the name of your enterprise organization.
Checking your enterprise organization ensures all members of your organization will have access to this web layer. Next, you’ll analyze the web layer to check for errors.
- Under Finish Sharing, click Analyze.
Several errors and warnings are returned. You must address all errors before publishing, but you can leave the warnings.
- Expand the error regarding the data source being registered with the server.
The first analyzer indicates the layer data source is not registered with the server. There are three feature layers that you are publishing, so there are three errors. To address these errors, you will register the DSNY enterprise geodatabase with the ArcGIS Server site by creating a data store item.
Note:
When you publish web services to Enterprise and choose to reference registered data, the data source must be registered with ArcGIS Server. This registration allows the server to access your data and use it as the source for web layers. Creating a data store is the key to making your data accessible to the server. A data store can be any location—enterprise databases, folders, cloud stores, or NoSQL databases—that houses the data you want to use. After registering the data with the server, the published web service establishes a direct connection to the data source. This connection ensures that the web services reference the data in the data store without duplicating it.
- Right-click the first error in the group and choose Register Data Source With Server.
The Add data store window appears.
- In the Add Data Store window, provide the connection details for the data store:
- For Title, type DSNYDataStore.
- For Tags, type DSNY, waste management, NYC, recycling, and Esri, pressing the Tab key after each..
- For Portal Folder, choose DSNY.
- Click the Validate button to validate the server database connection.
- For Share with, check the box for your enterprise organization.
- Click Create.
A check mark appears in front of the first sever message, which indicates the layer’s data source is registered with the server.
While it appears that you must register the other two layers, adding the data store one time will correct the issue for all layers with the same error because all the data is stored in the same geodatabase. When you analyze the web layer again, those errors will be resolved.
- Click Analyze.
The errors regarding registering the data with the server are resolved. Finally, you’ll clear the last error, by assigning unique numeric IDs. Assignment of unique IDs is a requirement when sharing data as a web layer and it ensures layer IDs remain static when the web layer or service is overwritten.
- Right-click the Unique numeric IDs are not assigned error message and click Auto-Assign IDs Sequentially.
The error is resolved. You will now publish the web layer.
- Click Publish.
After the publishing processes is completed, at the bottom of the pane, a message appears to confirm the web layers have been successfully shared. The message also contains a link to manage the web layers in your enterprise portal. You will use this link in the next module to access the web layers directly in the enterprise portal.
You have created database connections to store the legacy waste data and the vendors data. You then connected to your portal organization from ArcGIS Pro and shared the waste legacy data as a web feature layer that referenced a registered data store. In the next module, you will test the web editing experience by making edits to the web feature layer in a web map and assess the changes in the enterprise geodatabase.
Edit published data in a web map
Now that you have published the web layer, you will test the editing experience by accessing the DSNY web feature layer in Map Viewer and perform edits. Then, in ArcGIS Pro, you will access the waste legacy data from the database connection and confirm the edits you made in the browser reflect the original data.
Explore published items
First, you will access the portal items created after you published the DSNY Waste Management web layer.
- In the Share as Web Layer pane, click Manage the web layer.
- If necessary, in the upper right corner or the page, sign in with the same portal account to access the published data.
- On the ribbon, click the Content tab.
The item details page for the web layer that you published appears in a browser tab.
- In the My Content page, under Folders, click the DSNY folder to view its contents.
Note:
You may have different folders listed depending on what you have created.
In the DSNY folder, there are three portal items that were created when you published the DSNY Waste Management web layer:
- A data store item ensures the ArcGIS Server site has access to the data that is published.
- A map image layer is available only if you are sharing to an Enterprise portal and it is automatically created when you publish data by referencing a registered data store.
- A feature layer supports vector querying, visualization, and editing.
Next, you will open the DSNY Waste Management feature layer in Map Viewer and create a feature.
Edit the feature layer
Next, you will create a waste site feature in the DSNY Waste Management feature layer and save the web map.
- Click the DSNY Waste Management feature layer.
The item details page appears. The item details page is where you would add metadata and other descriptive information about the layer, such as use constraints and credits.
- In the DSNY Waste Management feature layer item details page, click Open in Map Viewer.
There are no features in this layer yet, so just the basemap appears. Before you create features, you will change the basemap to something more applicable to the data that you are working with.
- In the lower left corner of the screen, on the Contents (dark) toolbar, click the Expand button.
Now you can see all the command names.
- On the toolbar, click Basemap and choose Light Gray Canvas.
Having a basemap that is more general, with fewer labels and less topography, is more applicable to waste management data and will make the map easier for others to interpret.
The layers appear on the web map as they were in ArcGIS Pro when you applied the symbology from the layer files. With the change to the basemap, the emphasis for this web map is on the data, not the basemap. Next, you will create features using editing tools.
- Close the Basemap pane.
- On the Settings (light) toolbar, click the Edit button.
- In the Editor pane, click the Special waste sites layer.
By clicking a layer in the Editor pane, you make that layer the active layer and whatever features you create will be stored in that layer.
- Zoom in to the Queens Southeast zone and click to add a point on the map.
- In the Create features pane, enter the following attributes for the added waste site feature:
- For Type, type Recycling site.
- For Name, type Green Solutions.
- For Address, type 133 Dunkirk Street.
- For City, type Albans.
- For State, type NY.
- For ZIP, type 11412.
- For BORO and BOROCD, type 331
- Click Create.
You have created a waste site feature and added attributes for it. Next, you will save the web map.
- On the Contents toolbar, click Save and open and choose Save as.
Note:
The blue dot indicates that there are changes to the web map that have not been saved.
- In the Save map window, enter the following information:
- For Title, type DSNY Waste Collection Web Map.
- For Folder, click the drop-down menu and choose DSNY.
- Add some tags.
- For Summary, type This map is used for web editing.
- Click Save.
- Near the map’s title, click the options button and choose Content.
On the Content page, the DSNY Waste Collection Web Map appears with the other items in the DSNY folder.
Next, you will open ArcGIS Pro and confirm the waste site feature you added is now showing in the enterprise geodatabase.
View the web edits in the source geodatabase
You have edited a feature layer that you published and shared to your Enterprise portal. Next, you will explore the DSNY waste data from the database connection and ensure that the edit you made in the portal is referenced back to the enterprise geodatabase.
- Restore ArcGIS Pro and the DSNY project.
- Zoom in to the Queens Southeast zone where you added the waste site in the web map.
- With the Explore tool, click the waste site feature in Queens Southeast.
The attributes you added for the waste site feature in the web map appear in the pop-up for the feature in ArcGIS Pro.
- Close the Pop-up window.
To test the full editing experience, next you will perform edits directly in the enterprise geodatabase and assess how these edits reflect in the web feature layer, and implicitly the web map. For this example, you will move the location of one of the special waste sites.
- On the ribbon, click the Map tab. In the Selection group, click the Select tool.
- On the map, zoom in to the Staten Island area.
- In the Contents pane, turn off Storage garages.
- On the map, click the waste site feature to select it.
Since there are two features in the same location, a selection chip appears where you can specify which layer you’d like to select from.
- On the selection chip, click the drop-down arrow and choose DSNY Special Waste Drop-Off Site.
The selection chip disappears and the waste site feature is selected.
- On the ribbon, click the Edit tab, and in the Tools section, click Move.
The Modify Features pane appears.
You only have one selected feature in one layer, so any edits you make will take place on that feature. The Move tool is selected by default.
- On the map, point to the selected feature, click the yellow dot, and move the waste site feature to the northwest area of Staten Island.
Now that you have saved the edit made directly to the enterprise geodatabase, you will view the web map containing the waste data to verify that the edit displays.
- On the floating editing toolbar at the bottom of the map, click Finish Sketch.
- Return to the browser tab that has your enterprise portal and if necessary, click the Content tab.
- Click the DSNY Waste Collection Web Map item to open its item details page. Click Open in Map Viewer.
The waste site feature is now in the northern part of the Staten Island zone and reflects the edit made in ArcGIS Pro.
You successfully tested the full web editing experience by adding the web layer to a web map, creating a waste site feature, connecting to the database in ArcGIS Pro, and verified that the edit you made in the web map was present in a different client application. Finally, you performed an edit to the waste data in ArcGIS Pro and viewed the edit in the web map. Next, you will publish the vendors data directly from Enterprise.
Publish and share multiple layers from ArcGIS Enterprise
You will publish the vendors data as separate web feature layers directly from Enterprise. You will create a data store item and publish multiple web layers in one process. Finally, you will add the vendors web layer to the DSNY Waste Management web map and give access to members in the organization to the web map by sharing it with a portal group.
Create a data store item
The vendors data is stored in a different enterprise geodatabase than the legacy data. To ensure the ArcGIS Server has access to the NYC Vendors database, you will create a data store to register the data from this database with the server.
- View the browser tab that has your portal in it and click the Content tab.
- On the Content page, click the New item button.
The New item window appears. Here, you can choose from many item types, such as feature classes, imagery layers, tile layers, data stores, and more. You will create a data store.
- In the New item window, scroll down and click Data store.
- For Select the type of data store to add, click Database.
You chose Database because you want to connect to the NYC Vendors database.
- Click Next.
- For Select the type of database, click Relational database.
- Click Next.
- For Specify connection or location information to allow the ArcGIS Server sites to access the data, click Select File.
The file you choose will be an .sde file that stores the database connection to the NYC Vendors database. The .sde file was created when you established the database connection to the NYC Vendors database in ArcGIS Pro.
- In the file browser window, browse to C:\Users\UserName\Documents\ArcGIS\Packages and expand the DSNY_Project_b2653d folder. Click the NYC Vendors.sde file and click Open.
Note:
For simplicity, you changed the name from the qualified name of the database connection to NYC Vendors. Whatever the name of the database connection is will show up here as the name of the .sde file.
The connection information appears. Your connection information will be different, based on your server, instance, database, and other properties.
- Click Next.
- Check the box next to your server to select it.
- Click Next.
- For the Data store connection properties, enter the following information:
- For Title, type NYCVendorsDataStore.
- For Folder, click the drop-down menu and choose Create new folder. Name the folder Vendors_NYC and verify that is used as the folder.
- For Tags, type NYC and vendors.
- For Summary, type This data store supports the vendors web feature layers.
- Click Create connection.
The item details page for the data store item appears.
- Click the Content tab and for Folders, click Vendors_NYC to view the data store item.
Next, you will use the data store item to publish multiple web feature layers directly from Enterprise.
Publish multiple web feature layers
Next, you will bulk publish web feature layers from the data store item. This will eliminate the need to manually publish each web layer one by one.
- On the Content page, click NYCVendorsDataStore to open the item details page.
- On the item details page, click the Layers tab.
- Click Create Layers.
- For Time zone of the data, click the drop-down menu and choose (UTC-05:00) Eastern Time (US and Canada).
- Click Next.
- For Choose folder, click the drop-down menu and choose Vendors_NYC.
- Click Next.
- On the final page, verify that your server is selected and click Start publishing.
After the process is completed, four layers appear.
A feature layer and a map image layer for the Vendor_pickup_frequencies dataset and a feature layer and a map image layer for the Disposal_vendor dataset are added.
Next, you will create a portal group for sharing your web map. Sharing groups allow you to control who has access or editing capabilities to your data. In this example, you’ll create a group that is for editors of the waste collection data.
Create a group and share items to it
Before you can share your data and web map to a group, you must create a group for the data editors.
- On the ribbon, click the Groups tab and click Create group.
- In the Create a group window, for Name, type Data Quality.
- Scroll to the Group membership section.
In the Group membership section, you can control who can view the group, for example, all members of the organization or only members of the group. You can also control how others can join the group. For example, if you have several data editors in your organization and only want to provide them access to your data, you can invite those members to the group. For now, you’ll keep all the defaults and create the group so you can share to it.
- Click Save.
You have created a sharing group that can help limit access to your data to only specific users. Next, you will add the Vendor_pickup_frequencies and the Disposal_vendor feature layers to the DSNY Waste Collection web map and share it with the Data Quality group.
- Click the Content tab and click the DSNY folder.
- Click DSNY Waste Collection Web Map and open it in Map Viewer.
- In the Layers pane, click the Add button.
- In the Add layer pane, for the Vendor_pickup_frequencies and Disposal_vendor feature layers, click the Add button.
- Click the back arrow to close the Add layer pane.
The vendors layers appear in the Layers pane and on the map.
When you bulk published the vendors datasets, the web layers are displayed with default symbology. Normally, you would customize the symbology, but for now, you will leave the layers as they are. To learn more about thematic map symbology, see this tutorial.
- On the Contents toolbar, click Save and open and click Save.
Next, you will share the web map with the Data Quality group to ensure editing access to the members of that group.
- On the Contents toolbar, click Share map.
You can share with everyone in your organization or with everyone (public), which would provide open access to your data. In this case, sharing with everyone or the organization may not be optimal, as you don’t want anyone editing your data. You will use the sharing group that you created.
- In the Share window, for Set group sharing, click Edit group sharing.
- In the Group Sharing window, select the Data Quality group you created earlier.
Note:
Your groups will be different than what is shown in the image. If you have other groups already created in your portal, they will be listed here.
- Click OK and click Save.
Before you can save your edits, you are prompted to change the access type to the underlying web feature layers stored in the map. Updating the sharing will synchronize the sharing level of the Disposal_vendor, Vendor_pickup_frequencies, and DSNY Waste Management feature layers to match the DSNY Waste Collection web map.
- In the Review sharing window, click Update sharing.
A message appears at the bottom of the screen to confirm the DSNY Waste Collection web map sharing level was successfully updated.
- Navigate back to the Groups tab and click the Data Quality group.
The content that group members have access to is listed within the group. Now that you have the group and have shared the layers and web map in it, you could invite users who you want to have access to these items.
You have successfully published multiple layers from Enterprise, created a sharing group, and shared specific items with the group members.
In this tutorial, you acted as a publisher for the New York City Department of Sanitation where you help increase the department’s collaboration by implementing a web-based editing workflow in Enterprise. You first published the legacy data from ArcGIS Pro by referencing a registered data store. You then tested the web editing workflow by consuming the shared web feature layers into a web map and making edits to that web map. You bulk published the vendor data directly from Enterprise and added all the web layers to a web map. Finally, you ensured access to the web map by sharing it with a specific portal group.