Review of the `read_shapefile` and `kl_divergence` notebooks

Question

Review of the `read_shapefile` and `kl_divergence` notebooks

noahbouchier opened this issue 3 years ago · comments

As the first evidence of code that will help to form the output of my Google Summer of Code project - I'd be appreciative of any feedback on the read_shapefile and kl_divergence notebooks.

Some of the explanatory text is quite beefy, so feel free to skim that!

Noah Bouchier · Answer 1 · Fri Jul 16 2021 20:52:03 GMT+0800 (China Standard Time)

Please note - the kl_divergence notebook has just been updated to include use and explanation of the scipy.spatial.distance_matrix function

Noah Bouchier · Answer 2 · Mon Jul 19 2021 22:22:50 GMT+0800 (China Standard Time)

Please note - today's (gitmoji-inspired) update raises an issue within read_shapefile.ipynb.

The issue is within the final code block - In[16] - where the following line is returning an error:
leicester_oa_eth_shp.to_file("../leicester/data/leicester_ethnicity.gpkg", driver = "GPKG")

The crux of the error appears to be the following:
DriverError: sqlite3_open(../leicester/data/leicester_ethnicity.gpkg) failed: unable to open database file

Any help would be greatly appreciated - my searching seems to only identify this problem within other pieces of software, which make the solution hard to translate

Levi John Wolf · Answer 3 · Mon Jul 19 2021 22:24:28 GMT+0800 (China Standard Time)

Odd! Is there a file existing there when you open it in Finder?

If there are issues, it's OK to use a shape file for now.

Jeff Sauer · Answer 4 · Mon Jul 19 2021 22:27:42 GMT+0800 (China Standard Time)

Hey Noah! Could you try specifying an additional argument called 'layer'? For example, try the following: leicester_oa_eth_shp.to_file("../leicester/data/leicester_ethnicity.gpkg", layer='OA', driver = "GPKG") Link to geopandas reading and writing files <https://geopandas.org/docs/user_guide/io.html#writing-spatial-data> I'm guessing it needs you to create a table name before it can write the file

…

On Mon, Jul 19, 2021 at 9:24 AM Levi John Wolf ***@***.***> wrote: Odd! Is there a file existing there when you open it in Finder? If there are issues, it's OK to use a shape file for now. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPYN5RFJHRODO2ANEQZ24TTYQYSNANCNFSM5ALBSCQQ> .

-- *Jeff Sauer* | Pronouns: he/him/his PhD candidate in GIS & RS | University of Maryland College Park Phone: +1 (612) 394 1115 | Email: ***@***.***

Noah Bouchier · Answer 5 · Mon Jul 19 2021 22:38:57 GMT+0800 (China Standard Time)

Hey!

Unfortunately adding the layer argument still returns the same issue.

There isn't a file in that location locally - I supposed that the to.file would create the file, as I've experienced previously with the shape files. But that did cross my mind when reading the error.

Is it worth creating a GeoPackage to then update? If so, how's best to do that?

Jeff Sauer · Answer 6 · Mon Jul 19 2021 22:48:24 GMT+0800 (China Standard Time)

Hmm, drat! Okay, let's try a simple test first - Let's load some of the built-in geopandas data and try to write is as a geopackage. ``` import geopandas as gpd test = gpd.read_file(gpd.datasets.get_path("naturalearth_lowres")) test.to_file("FULL/PATH/TO/DESKTOP/test.gpkg", driver = "GPKG") ``` Do the above three lines work?

…

On Mon, Jul 19, 2021 at 9:39 AM Noah Bouchier ***@***.***> wrote: Hey! Unfortunately adding the layer argument still returns the same issue. There isn't a file in that location locally - I supposed that the to.file would create the file, as I've experienced previously with the shape files. But that did cross my mind when reading the error. Is it worth creating a GeoPackage to then update? If so, how's best to do that? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPYN5QU6I5FNA5HHFNL3XDTYQ2IZANCNFSM5ALBSCQQ> .

-- *Jeff Sauer* | Pronouns: he/him/his PhD candidate in GIS & RS | University of Maryland College Park Phone: +1 (612) 394 1115 | Email: ***@***.***

Noah Bouchier · Answer 7 · Mon Jul 19 2021 23:01:31 GMT+0800 (China Standard Time)

Do the above three lines work?

Yes - they create a 324KB file on my desktop that I've been able to successfully read back in under a different object name

Jeff Sauer · Answer 8 · Tue Jul 20 2021 05:35:12 GMT+0800 (China Standard Time)

@noahbouchier worked through the notebook on my local machine. You need to specify the full path when using .to_file(...).

Remove the .. and use the full path.

I got the same error as you when trying to save with '../'. Once I put in the full path I was able to save the GPKG.

Example:

# Save the new file as a shapefile, titled 'leicester_ethnicity'
leicester_oa_eth_shp.to_file("C:/Users/USER/Desktop/leicester_ethnicity.gpkg", driver = "GPKG")

Noah Bouchier · Answer 9 · Tue Jul 20 2021 17:01:34 GMT+0800 (China Standard Time)

@jeffcsauer thanks so much for your help! Your solution helped me to identify the problem - stemming from my oversight.

The file path to_file was attempting to write to ("../leicester/data/leicester_ethnicity.gpkg") was incorrect, and related to a previous iteration of the workbook.

Within the GitHub repository, the correct file path is:

"../Example Data - Leicester, UK/leicester_ethnicity.gpkg"

When I use this, the GeoPackage file is created exactly as it should be, like a dream.

Thanks for your help, and apologies for the error that created the problem

Levi John Wolf · Answer 10 · Tue Jul 20 2021 17:53:31 GMT+0800 (China Standard Time)

no prob! the error raised when the path is wrong are not the most informative... glad you figured it out!