Giters

r-b-g-b / clean-water-tool

Reporting Tool to Support Safe Drinking Water in California’s Disadvantaged Communities

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Apply GIS boundaries to water system data to classify systems by types

ruckeralex opened this issue · comments

commented

The Community Water Center (the non-profit we are working with) has requested that the search tool be able to classify water systems by the following types:

  1. Legislative districts - associate each water system with its legislative district (https://services.gis.ca.gov/arcgis/rest/services/Boundaries/CA_Legislative_Districts/MapServer)

  2. Regional water board - associate each water system with its regional board (https://www.waterboards.ca.gov/waterboards_map.html)

For now, we can define GIS points for water systems via centroid of associated zip code. You can access the GIS map shapefile here: https://www.waterboards.ca.gov/water_issues/programs/hr2w/docs/data/ec_summary_feb2019.zip.

NOTE: There should be around 3,297 systems; you can access a summary of the system name / zip code here: https://www.waterboards.ca.gov/water_issues/programs/hr2w/docs/data/2019-04-03_hr2w_web_data_summary.xls

Please let me know if you have any questions! ruckeralex@gmail.com

Thank you!

commented

Don had a good suggestion last night, use the already associated zip code to lookup the associated legislative info, there are some available ways to do this without doing the GIS point in shape boundary calculation

commented
commented
commented

Sounds like Diana Y is able to perform this GIS task!

The goal is to produce an Excel spreadsheet that provides 1 row per existing water system and 3 related associated columns: for water system, legislative district, and regional water board. NOTE: There should be around 3,297 systems, so there will be approx 3,300 rows on the final Excel sheet.

  1. Access a summary of the system name / zip code here: https://www.waterboards.ca.gov/water_issues/programs/hr2w/docs/data/2019-04-03_hr2w_web_data_summary.xls

  2. Associate each water system with its legislative district (https://services.gis.ca.gov/arcgis/rest/services/Boundaries/CA_Legislative_Districts/MapServer)

  3. Associate each water system with its regional board (https://www.waterboards.ca.gov/waterboards_map.html)

For now, we can define GIS points for water systems via centroid of associated zip code. You can access the GIS map shapefile here: https://www.waterboards.ca.gov/water_issues/programs/hr2w/docs/data/ec_summary_feb2019.zip.

commented

Here is the link to the map: https://arcg.is/SKLzy (one thing that bothers me is how the assembly name displays. I tried to modify the size of those brown labels, but they seem to be locked by the layer). Please let me know if you have any comments or questions. Have a great weekend.

commented

This is fabulous! Thank you @dilufapi!

I see everything we need on the interactive map. Could you help me extract the associations between water systems and legislative and water districts? We will use the association between water system and district to list the systems in tabular views by district the CWC has requested and create sums for stats like District X has Y systems out of compliance affecting Z total people.

commented
commented

I opened the map in ArcGIS Desktop, and it looks like the CA_Legislative_District and CA Regional Water Board Offices layers are locked. It does not allow me to edit, extract, or merge the data. Maybe there is a way to get around it, but I do not know it yet. I will keep researching about how to troubleshoot this. I sent some screenshots to Rucker to explain.

commented
commented

I found a way to retrieve the CA state Senate and Assembly District from the lat,lon: https://openstates.org/find_your_legislator/?lat=37.8056148&lon=-122.2725945

commented

Hi Aaron - That sounds great! Does that mean you have all the data you need? ...If not, I participated in the Water Data Science Symposium and Datathon last week and brought up the GIS data layer question to a couple of people. They recommended reaching out directly to the data manager to request the raw data or the unlocked layers. Therefore I just sent an email to the GIS contact (Franklin.Stieringer@waterboards.ca.gov) I got from the Division of Drinking Water Staff. I will let you know if I hear anything.

commented
commented

I have been able to map all the systems to the CA legislators.

I haven't associated them all with their Water District

commented

Nayana - can you please take a stab at outlining the steps needed to associate a legislative district to each of the 3000+ water systems using this file that Aaron sent:
Link shapefiles with each latitude-longitude to each water system ID: https://raw.githubusercontent.com/aaronhans/community-water-center/master/data/locations.json

@aaronhans

commented

This problem is driving me nuts. Just tested to see if we can associate most of the water systems to their legislative assembly district using zip code only... and the answer is: we can only do this for a third of the systems (1,010 out of 3,297 systems; of those, just over 10% are actually out-of-compliance). Of the rest, 947 are not associated with legislative district and the remaining have between 2 and 7 (!) districts associated with the zip code. Attaching file-- see "system by assembly district" tab, which contains the 1.010 districts we can associate to water systems.
Water systems by CA Assembly district.xlsx

commented

I pushed an update last week which includes a new file for each water district with the associated CA House and Assembly District and current legislators: aaronhans/community-water-center@1c279eb

We'll need to rerun that script after each election when those members change.

I plan to start some analysis this evening to compare districts so we can identify the politicians representing areas with the least and most reported affected populations

The ability to associate a water system id with one of the 24 CA Water Districts is still outstanding.

commented

Hi Aaron-- thanks! Looking forward to seeing you / talking tonight about this... want to make sure I understand what you mean by "CA Water District." (Is that the Regulating Agency?)

commented

Hi! Here are a couple of plots that I made using the data! Thanks!
https://github.com/nayanab565/CA-Water-Project/blob/master/CAWaterProjectGraphs.ipynb

commented

@mnorelli - thank you so much for your help! Could you associate for each of the ~3,300 water systems the Regional Water Board (#s 1 through 9) that is is associated with?

Some resources:

Map with (locked?) layers:
https://arcg.is/SKLzy

Fact sheet of regional boards: https://www.waterboards.ca.gov/publications_forms/publications/factsheets/docs/region_brds.pdf

Data source:
https://www.waterboards.ca.gov/water_issues/programs/hr2w/docs/data/2019-04-03_hr2w_web_data_summary.xls

commented

Hi @nayanab565 and @dilufapi -- it turns out that @aaronhans HAS used code to successfully associate legislative districts with each of the 3,300 water systems in California! So, that problem is now solved. Thank you so much, Aaron.

commented

Complete.
WaterSystemsByRegionalBoard

This Excel table:
WaterSystemsByRegionalBoard.xlsx
shows, for each water system point derived from zip code centroids, the regional water board name it falls in, in three columns:
Name like "North Coast" or "Santa Ana"
OFFICE usually blank, except where same Name occurs more than once, like "Central Valley"
RB_OFF the Region Number, 1-9, and where there is an OFFICE, RB_OFF will be followed with letter(s), which are the abbreviation of the office name, like "6SLT" for Lahontan, South Lake Tahoe office.

Next step: find real water systems polygons and redo this overlay process in GIS. I'll look for that, but forward any leads or results of conversations with Board staff...

commented

More! Here are the all the water system points in Excel, with regional water board designations as above, plus CA Assembly and CA Senate district numbers in the rightmost columns:
WaterSystemsByRegionalBoardAndLegDistricts2.xlsx

commented
commented

Excellent! Attaching updated file with sources, and sent copy (with credit to you and Aaron) to CWC and Water Board. Thank [you!!!](url
CAWaterSystems_ByRegionalBoardandLegDistricts.xlsx
)

commented
commented