Chron job scheduler fails authentification for bigQueryR & googleCloudStorageR
giorsfloto opened this issue · comments
Describe the bug
When a r script is executed through cronjob scheduler, it fails to autentify on the GCP project, even if the json credentials file is available in the directory.
To Reproduce
Listening on http:*******
cronR job
id: job_11bdbddfd266a6cda166bc034507d54e
tags:
desc: scoring first
0 6 * * * /usr/local/lib/R/bin/Rscript '/home/AdvansCAM/0.1-Master.R' >> '/home/AdvansCAM/0.1-Master.log' 2>&1
2020-05-08 06:19:44> No environment argument found, looked in BQ_AUTH_FILE
2020-05-08 06:19:44> No environment argument found, looked in GCS_AUTH_FILE
data.table 1.11.4 Latest news: http://r-datatable.com
Expected behavior
I would expect authentification through json file to be succesfull. Actually when the script is launched manually it works smoothly
**Session Info
R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)
Matrix products: default
BLAS: /usr/lib/openblas-base/libblas.so.3
LAPACK: /usr/lib/libopenblasp-r0.2.19.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8
[4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=C
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] lubridate_1.7.4 data.table_1.11.4 googleCloudStorageR_0.4.0
[4] bigQueryR_0.4.0 shiny_1.4.0.2
loaded via a namespace (and not attached):
[1] Rcpp_0.12.18 magrittr_1.5 xtable_1.8-2 R6_2.2.2
[5] rlang_0.4.6 fastmap_1.0.1 stringr_1.3.1 httr_1.3.1
[9] tools_3.5.1 googleAuthR_0.6.3 htmltools_0.4.0 openssl_1.0.2
[13] yaml_2.2.0 assertthat_0.2.0 digest_0.6.16 zip_1.0.0
[17] later_1.0.0 promises_1.1.0 curl_3.2 memoise_1.1.0
[21] mime_0.5 stringi_1.2.4 compiler_3.5.1 jsonlite_1.5
[25] httpuv_1.5.2
Additional context
We had to reboot our GCP VM, since then the chron job scheduler is not working any more.
My guess is the R script is referring to a path that can't be found when executing in cron under its user - make sure no relative paths, perhaps at the top of the script echo out the working directory to check its where you think it is.
But if the VM is only for scheduling R scripts then these days I use googleCloudRunner https://code.markedmondson.me/googleCloudRunner/articles/cloudscheduler.html#schedule-an-r-script-1