Unzip function for windows

Question

Unzip function for windows

Mightynasty opened this issue 4 years ago · comments

Mightynasty commented 4 years ago

Hi, I'm working on windows and when reading file with readDWD I get the following error:

'unzip' is not recognized as an internal or external command,
operable program or batch file.

I have both utils and ncdf4 libraries installed, and separately the unzip function works.

Berry Boessenkool · Answer 1 · Tue Aug 25 2020 00:44:55 GMT+0800 (China Standard Time)

Could you post a reproducible example?

Mightynasty · Answer 2 · Tue Aug 25 2020 01:32:54 GMT+0800 (China Standard Time)

Hi ! Here it is:
I used the example similar to the one in the docs:

link <- selectDWD(id = "150", res="daily", var="kl", per="recent")
file <- dataDWD(link, read=FALSE)
clim <- readDWD(file, varnames=FALSE)

The console response:

dataDWD -> dirDWD: adding to directory '.../DWDdata'
dataDWD -> newFilename: creating 1 file:
'.../DWDdata/daily_kl_recent_tageswerte_KL_00150_akt.zip'
Reading file with readDWD.data().
'unzip' is not recognized as an internal or external command,
operable program or batch file.
'(unzip -p .../DWDdata/daily_kl_recent_tageswerte_KL_00150_akt.zip produkt_klima_tag_20190221_20200819_00150.txt) > ...\AppData\Local\Temp\RtmpSYVSSB\file43042d986b3c' execution failed with error code 1File '...\AppData\Local\Temp\RtmpSYVSSB\file43042d986b3c' has size 0. Returning a NULL data.frame.File contains no rows: .../DWDdata/daily_kl_recent_tageswerte_KL_00150_akt.zip

Berry Boessenkool · Answer 3 · Tue Aug 25 2020 04:27:42 GMT+0800 (China Standard Time)

Thanks for the example. It works on my computer. I wonder if this is related to #22.
Can you try clim <- readDWD(file, fread=FALSE)?

Mightynasty · Answer 4 · Tue Aug 25 2020 04:41:06 GMT+0800 (China Standard Time)

Perfect, it worked! Now I see it was NA by default.

Berry Boessenkool · Answer 5 · Tue Aug 25 2020 04:55:02 GMT+0800 (China Standard Time)

Could you try a few things so I can maybe find out where this comes from?
Do you get the same error for the last line here?

fp <- unzip(file, list=TRUE) # file produkt*, the actual datafile
fp <- fp$Name[grepl("produkt",fp$Name)]

dat <- data.table::fread(cmd=paste("unzip -p", file, fp), na.strings=berryFunctions::na9(nspace=0),
                                  header=TRUE, sep=";", stringsAsFactors=TRUE, data.table=FALSE)

And for the following?

dat <- data.table::fread(cmd=paste("unzip -p", file, fp), header=TRUE, sep=";")

Mightynasty · Answer 6 · Tue Aug 25 2020 05:05:43 GMT+0800 (China Standard Time)

Same error for both:

unzip' is not recognized as an internal or external command,
operable program or batch file.

Berry Boessenkool · Answer 7 · Tue Aug 25 2020 05:11:13 GMT+0800 (China Standard Time)

Seems like unzip is only available if you have e.g. a git shell (source). I'll see if I can find an alternative online...

Mightynasty · Answer 8 · Tue Aug 25 2020 05:14:19 GMT+0800 (China Standard Time)

Yes, like many commands, only working from the shell. Thanks a lot for you help and work!

Berry Boessenkool · Answer 9 · Tue Aug 25 2020 05:19:23 GMT+0800 (China Standard Time)

What output do you get with the following?

system("unzip", show.output.on.console=FALSE)
Sys.which("unzip") # gives me c:\\Rtools\\bin\\unzip.exe

I'm considering keeping the default for fread as NA but not only checking for data.table but also for the unzip command...

Berry Boessenkool · Answer 10 · Tue Aug 25 2020 06:06:10 GMT+0800 (China Standard Time)

I think I have a solution. Could you run updateRdwd() to have version 1.4.1 (2020-08-24) and then run the following two examples?

readDWD(file) # should run fine now with message "Reading file with readDWD.data() and fread=FALSE"
readDWD(file, fread=T) # should give warning [1] and then the error as before

Lastly, if you'd want to take the time, could you install Rtools (directly at C:/Rtools since compiler paths may not have spaces, as there would be with C:/Program Files/R/Rtools/) and see if both examples work fine? That would settle issue #22 for me.
Don't feel pressured, though, I don't want to be a burden on your time. I appreciate you pointing out this issue regardless :).

[1] system command 'unzip' could not be found. Expect trouble with data.table::fread.

Mightynasty · Answer 11 · Tue Aug 25 2020 16:05:14 GMT+0800 (China Standard Time)

Hi, I tried both, the second one still with the same warning and error.
Rtools are now directly in C: drive and I restarted everything.
But since the readDWD works it's great! Thank you.

Berry Boessenkool · Answer 12 · Tue Aug 25 2020 16:43:31 GMT+0800 (China Standard Time)

Thanks so much for checking!
One last question, for clarity: with Rtools installed and R restarted, does it still not work with fread=T? Does it find Sys.which("unzip")?

Mightynasty · Answer 13 · Tue Aug 25 2020 17:22:49 GMT+0800 (China Standard Time)

The Sys.which("unzip") just gives this output:
unzip ""
As for the fread=T, still the same.

Berry Boessenkool · Answer 14 · Tue Aug 25 2020 17:56:53 GMT+0800 (China Standard Time)

Maybe Rtools was not added to the path. Well, it was worth a try. If you're happy with how it works, so am I.

Thank you so much for trying some things out!

Berry Boessenkool · Answer 15 · Tue Sep 22 2020 23:31:06 GMT+0800 (China Standard Time)

I just had the same problem on my laptop.
It was easy to solve by adding rtools40 to the PATH. I now have instructions for that online:
https://bookdown.org/brry/rdwd/fread.html