Identify key variables in download_daymet_batch
spencer-tassone opened this issue · comments
Is it possible to identify which variables a user is interested in when accessing many sites using download_daymet_batch? For instance, I am only interested in TMAX and TMIN in my analysis and do not need the other variables that daymetr provides. Being able to identify which variables are of interest inside the download_daymet_batch() function could cut down on memory usage and increase processing speed.
I don't think you will gain that much speed. You can try to fix this for yourself to test by altering this line:
Line 123 in f2238b0
(only retaining tmax and tmin) on a clone of the project.
Install the altered version by hitting the install button in the build panel when in the project, and see if you see performance gains.
The reason I don't allow this flexibility is that the overhead is relatively low (IMO) and more time can be gained by parallelizing things or scheduling nightly downloads (batch jobs). Note, there might be a rate limiter active on the server so I would not push for more than 10 parallel downloads from the same location.
Another alternative is using my appeears package. Not yet on CRAN but it should allow for fast scheduling and it won't block a session as the API call is asynchronously processed in the cloud (similar to my ecmwfr package). Still under review so you will have to use the development install from github.