DEQ Environmental Data Viewer
Generic application for searching, viewing and downloading DEQ GIS data and related tables.
Stage - test.mapserv.utah.gov/deqenviro/
Production - enviro.deq.utah.gov
Query Layer Data Requirements
In order for a dataset to be used as a query layer within the application, it must satisfy all of the following requirements:
- Accessible from our server (usually requires some firewall requests)
- For tables that need to be translated into points:
- Coordinates stored in either
LATITUDE
&LONGITUDE
orEASTING
&NORTHING
fields.
- Coordinates stored in either
- A row in the configuration spreadsheet that defines mappings for the five main fields (
ID
,NAME
,ADDRESS
,CITY
, &TYPE
). If there is no mapping for a specific field a value ofn/a
should be used in the config. - A unique id field. If the field is not automatically recognized by ArcGIS Pro, then the
OID Field
column in the config spreadsheet can be used to define it.
Testing
Unit tests are run via intern.
To run tests:
grunt
npx intern serveOnly
- Go to: http://localhost:9000/__intern/
Nightly Script
Runs nightly on test and prod servers.
Builds DEQEnviro.json
which the web app uses to configure itself. Part of building this json file is getting all of the map service layer indices so it needs to be rerun manually after adding, removing or reordering any of the map service layers.
Make sure that you have a latest version of pip before pip install -r requirements.txt
.
This script requires settings/oauth2key.json
. Check out the oauth2 gspread docs to learn how to generate it. Make sure to grant read permission to the email address in client_email
to the config spreadsheets.
Updates related data in SGID10. Reads sources from the config spreadsheet.
Data Schema Changes
Most updates are taken care of via the config spreadsheet and updating the schema of data.
Adding a new field
- Add the field to the "Identify Attributes" column in the config spreadsheet. This will make it show up in the identify pane in the app.
- Add the field to the data in SGID10 (prod & staging).
- Add the field to the data in
staging/deqquerylayers.gdb
. - Delete the associated dataset in
staging/sgid_stage.gdb
if it's there.
Adding a new query layer
- Add the new row in the config spreadsheet
- Run forklift pallet.
- Add new layer to
maps/MapService.mxd
ormaps/Secure.mxd
and republish. - Manually run
build_json.py
to get the layer number from the map service of the newly added layer.
Deploy Steps
-
Set up and install ArcGisServerPermissionsProxy.
- Import RavenDB and web.config from previous server.
- Use configs/permissionproxy.json to create a new application
- May need to set the
AccessRules.EndDate
to5000000000000
for the initial user so that you can log in successfully the first time.
-
Create a
deqnightly
user in ArcGIS Server and assign it to thedeq_admin
role.- Fill in the credentials in the settings for the pallet.
-
Build and deploy (using web deploy) api/Search.Api/Search.Api.sln to the web server (
<root>/deqenviro/api
).- Register SOE from the same project with ArcGIS Server.
-
Publish
maps/MapService.mxd
andmaps/Secure.mxd
to aDEQEnviro
folder in ArcGIS Server.Secure
should be locked down to just thedeq_admin
anddeq_water
roles.- Add the SOE to each of these services:
- sitename:
NAME
- maxrecords:
25000
- returnFields:
ID,NAME,ADDRESS,CITY,TYPE,OBJECTID,ENVIROAPPLABEL,ENVIROAPPSYMBOL
- facilityust:
FACILITYUST
- programid:
ID
- sitename:
-
Publish ExportWebMap service to the
DEQEnviro
folder usingmaps/PrintTemplates/Portrait.mxd
as the default template.- Make sure that the server can resolve the domain name that the app is hosted on (e.g. test.mapserv.utah.gov). If it can't you will need to edit the hosts file. This is required for the
ExportWebMap
service. - synchronous
- Make sure that the server can resolve the domain name that the app is hosted on (e.g. test.mapserv.utah.gov). If it can't you will need to edit the hosts file. This is required for the
-
Run and publish
scripts/download/DeqEnviro.pyt/download
asToolbox/download
in the sameDEQEnviro
folder.pip install xlsxwriter
on the hosting server fom the python installation that ArcGIS Server uses (x64).pip install xlsxwriter
on the publishing server for the python installation that ArcGIS Desktop uses (x32).- You can use these inputs as a test:
{"BFNONTARGETED":["Pre5","Pre9","Pre8","Pre4","Pre7","Pre10","Pre12","Pre13","Pre14","Pre11","13","14"],"BFTARGETED":["2A","3","5","6","4","8","9","10","11","12","1","2","7"]} shp C:\forklift\data\production\deqquerylayers.gdb
-
Add repo to forklift.
- Copy
scripts/nightly/databases
&scripts/nightly/settings/__init__.py
from old server. - Download and install the latest oracle instant client.
- From within the forklift environment:
pip install -r .\scripts\nightly\requirements.txt
- Copy
-
Build and deploy the application by running
grunt build-prod && grunt deploy-prod
.- You will need to run
scripts/nightly/build_json.py
to generateDEQEnviro.json
before you can load the application for the first time.
- You will need to run