Problem with Glasgow dataset and related error
leekkww opened this issue · comments
I followed the user interface instructions and downloaded the glasgow_osm.zip dataset. However, on Step 3 downloading panoramas from Google, the following exception is thrown:
javax.imageio.IIOException: Can't get input stream from URL!
at javax.imageio.ImageIO.read(ImageIO.java:1395)
at org.twak.readTrace.Mosaic.tile(Mosaic.java:93)
at org.twak.readTrace.Mosaic.lambda$processLines$2(Mosaic.java:64)
at org.twak.utils.Parallel$1.run(Parallel.java:97)
Caused by: java.io.FileNotFoundException: http://cbk0.google.com/cbk?output=tile&panoid=vV2SxgjEPMo0Vf3F5qEiEA&zoom=5&x=0&y=0
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1836)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at java.net.URL.openStream(URL.java:1045)
at javax.imageio.ImageIO.read(ImageIO.java:1393)
If I go to the link thrown by the error (http://cbk0.google.com/cbk?output=tile&panoid=vV2SxgjEPMo0Vf3F5qEiEA&zoom=5&x=0&y=0) then I see a black screen.
If I go to another link used by the previous file in todo.list (http://cbk0.google.com/cbk?output=tile&panoid=mk9dFbTwGNpQiokh5to04w&zoom=5&x=0&y=0) then there is content.
My questions are:
- Is this a problem with the data set or the code? My intuition is that this is the fault of the data set. If so, how do I go about fixing this?
- This exception occurs on image 74 out of 523, and after the exception, the download of the panorama terminates (so we don't get any more images). Is there any way that we can continue to download the other panorama photos and just ignore this one scene? because it seems silly that the entire download thread would terminate just because of this one image failure.
The interface also looks like it's missing some panorama points:
Hi Joanne, so the "scraping images from google" functionality relies on google keeping the panorama ids the same. It look like they've changed them for Glasgow now.
You could edit the todo.list to remove the entry that causes the error and then try again?
Otherwise if you want to hack the code, the error will be in this file:
https://github.com/twak/chordatlas/blob/db2574a87b3b8484958c693213c4c1cba292e1af/src/org/twak/readTrace/Mosaic.java
I'll leave this ticket open, and when I get a chance I'll edit the code to keep going when we see this error.
I pushed a code change to fix this; do you need a new binary (jar)?
There were a bunch of panoramas which have been removed. There's nothing I can do to fix this :(
Yes, a new binary would be really helpful! 😄
As for the panoramas, do you know how the todo.list is generated? (In fact, how is the dataset compiled in general?) Maybe there is some way of retrieving the panoramas that's not just taking the list of jpg names.
Never mind on the data compilation - I see the instructions for compiling the datasets here.
Edit: the instructions for downloading panorama metadata leads to this repository which seems to be deprecated? but this is probably for a separate issue.
here's a new jar, it hasn't been well tested yet: https://drive.google.com/open?id=1H5aTqFAuImTaRZEBH34lwm5dcX8XMcqe
you will need to reset the pano folder to the version in glasgow_osm.zip, to let the the downloads continue.
yes, over the summer google locked down its panorama API, so my Panoscraper tool no longer works to create these todo.list files. I haven't had time to look at this and see if it something that we can fix.
If you are desperate, Panoscraper might work manually - at the bottom of the page it says
current pano scraped info:
33.37460288750182, -111.9102955503079, 362.6907958984375, 25.33748054504395, 88.97329711914062,2.068044900894165
id: l3r5dw8JducpuDcyYhJggw
If you convert it to the following format in todo.list it might work:
33.37460288750182_-111.9102955503079_362.6907958984375_25.33748054504395_88.97329711914062_2.068044900894165_l3r5dw8JducpuDcyYhJggw
Thank you so much for the timely response, and for (what appears to be, so far) a working solution!