Support httr config argument for OGC service clients
maelle opened this issue · comments
👋 here!
We're dealing with a rather old web service that requires special handling on Ubuntu, it only works with a special curl handle as shown below.
Would it be possible to somehow pass a handle/handle parameters to the WFSclient? (We're also in contact with the web service maintainers)
url <- 'https://drive.emodnet-geology.eu'
req1 <- curl::curl_fetch_memory(url)
#> Error in curl::curl_fetch_memory(url): error:1414D172:SSL routines:tls12_check_peer_sigalg:wrong signature type
h <- curl::new_handle(ssl_cipher_list='DEFAULT@SECLEVEL=1')
req2 <- curl::curl_fetch_memory(url, handle = h)Created on 2022-03-11 by the reprex package (v2.0.1)
I've looked at the docs of the curl package, it does not seem there's a way to customize the handle via options / environment variables.
Oh but i should look into httr::handle_pool() 🤔
@maelle I will have a look, not sure an easy solution can be done through ows4R, considering this is plateform dependent. On Windows it works well.
Oh we'd do the platform handling, we via EMODnetWFS would just pass a different handle / a different set of handle options to ows4r::WFSClient$new(). (that ows4r would pass to httr::GET() handle argument)
I'm also trying to understand if we can influence the "handle pool" directly somehow.
Thank you!
I'm not sure to understand what is the exact problem you deal with, can you share the R code that fails for you? i've just tested the WFS Client from an R Studio server deployed on Ubuntu, and it works:
wfs = WFSClient$new(
url = "https://drive.emodnet-geology.eu/geoserver/gtk/wfs",
serviceVersion = "2.0.0",
logger = "DEBUG"
)session info
R version 4.1.2 (2021-11-01)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.4 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
locale:
[1] LC_CTYPE=C.UTF-8 LC_NUMERIC=C LC_TIME=C.UTF-8 LC_COLLATE=C.UTF-8 LC_MONETARY=C.UTF-8
[6] LC_MESSAGES=C.UTF-8 LC_PAPER=C.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] geosapi_0.5-1 ows4R_0.3 atom4R_0.1-2 geonapi_0.5-3 keyring_1.2.0 geometa_0.6-6
loaded via a namespace (and not attached):
[1] tidyselect_1.1.2 xfun_0.30 purrr_0.3.4 sf_1.0-7 lattice_0.20-45 V8_4.1.0
[7] vctrs_0.3.8 generics_0.1.2 utf8_1.2.2 XML_3.99-0.9 rlang_1.0.2 e1071_1.7-9
[13] pillar_1.7.0 glue_1.6.2 DBI_1.1.2 sp_1.4-6 lifecycle_1.0.1 stringr_1.4.0
[19] rdflib_0.2.5 zip_2.2.0 knitr_1.37 tzdb_0.2.0 curl_4.3.2 parallel_4.1.2
[25] class_7.3-20 fansi_1.0.2 Rcpp_1.0.8 KernSmooth_2.23-20 readr_2.1.2 openssl_2.0.0
[31] classInt_0.4-3 redland_1.0.17-16 jsonlite_1.8.0 hms_1.1.1 askpass_1.1 stringi_1.7.6
[37] jsonld_2.2 dplyr_1.0.8 grid_4.1.2 rgdal_1.5-28 cli_3.2.0 tools_4.1.2
[43] magrittr_2.0.2 proxy_0.4-26 tibble_3.1.6 crayon_1.5.0 tidyr_1.2.0 pkgconfig_2.0.3
[49] ellipsis_0.3.2 xml2_1.3.3 assertthat_0.2.1 roxygen2_7.1.2 httr_1.4.2 R6_2.5.1
[55] units_0.8-0 compiler_4.1.2
So, after digging a bit, I don't think one can modify the handle pool externally, so we'd really need ows4r to let us pass the handle to httr::GET() and friends. (I understand if it's not prioritary 🙂 )
Noting that httr2 has httr2::req_options() which would be perfect here (but maybe switching to httr2 would be even more work here 😅 )
However there are not options we could set externally, we'd still need a way to pass them. 🤔
I had missed your reprex, running it now.
wfs = ows4R::WFSClient$new(
url = "https://drive.emodnet-geology.eu/geoserver/gtk/wfs",
serviceVersion = "2.0.0",
logger = "DEBUG"
)
#> Loading ISO 19139 XML schemas...
#> Loading ISO 19115 codelists...
#> Loading IANA mime types...
#> No encoding supplied: defaulting to UTF-8.
#> [ows4R][INFO] OWSGetCapabilities - Fetching https://drive.emodnet-geology.eu/geoserver/gtk/wfs?service=WFS&version=2.0.0&request=GetCapabilities
#> Error in curl::curl_fetch_memory(url, handle = handle): error:1414D172:SSL routines:tls12_check_peer_sigalg:wrong signature typeCreated on 2022-03-11 by the reprex package (v2.0.1)
And with system info
wfs = ows4R::WFSClient$new(
url = "https://drive.emodnet-geology.eu/geoserver/gtk/wfs",
serviceVersion = "2.0.0",
logger = "DEBUG"
)
#> Loading ISO 19139 XML schemas...
#> Loading ISO 19115 codelists...
#> Loading IANA mime types...
#> No encoding supplied: defaulting to UTF-8.
#> [ows4R][INFO] OWSGetCapabilities - Fetching https://drive.emodnet-geology.eu/geoserver/gtk/wfs?service=WFS&version=2.0.0&request=GetCapabilities
#> Error in curl::curl_fetch_memory(url, handle = handle): error:1414D172:SSL routines:tls12_check_peer_sigalg:wrong signature typeCreated on 2022-03-11 by the reprex package (v2.0.1)
Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.1.0 (2021-05-18)
#> os Ubuntu 20.04.1 LTS
#> system x86_64, linux-gnu
#> ui X11
#> language (EN)
#> collate fr_FR.UTF-8
#> ctype fr_FR.UTF-8
#> tz Europe/Paris
#> date 2022-03-11
#> pandoc 2.16.2 @ /usr/lib/rstudio/bin/quarto/bin/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> askpass 1.1 2019-01-13 [1] CRAN (R 4.1.0)
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.1.0)
#> backports 1.4.1 2021-12-13 [1] CRAN (R 4.1.0)
#> class 7.3-19 2021-05-03 [4] CRAN (R 4.0.5)
#> classInt 0.4-3 2020-04-07 [1] CRAN (R 4.1.0)
#> cli 3.2.0 2022-02-14 [1] CRAN (R 4.1.0)
#> codetools 0.2-18 2020-11-04 [4] CRAN (R 4.0.3)
#> crayon 1.5.0 2022-02-14 [1] CRAN (R 4.1.0)
#> curl 4.3.2 2021-06-23 [1] CRAN (R 4.1.0)
#> DBI 1.1.2 2021-12-20 [1] CRAN (R 4.1.0)
#> digest 0.6.29 2021-12-01 [1] CRAN (R 4.1.0)
#> dplyr 1.0.8 2022-02-08 [1] CRAN (R 4.1.0)
#> e1071 1.7-9 2021-09-16 [1] CRAN (R 4.1.0)
#> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.1.0)
#> evaluate 0.15 2022-02-18 [1] CRAN (R 4.1.0)
#> fansi 1.0.2 2022-01-14 [1] CRAN (R 4.1.0)
#> fastmap 1.1.0 2021-01-25 [1] CRAN (R 4.1.0)
#> fs 1.5.2 2021-12-08 [1] CRAN (R 4.1.0)
#> generics 0.1.2 2022-01-31 [1] CRAN (R 4.1.0)
#> geometa 0.6-6 2022-01-26 [1] CRAN (R 4.1.0)
#> glue 1.6.2 2022-02-24 [1] CRAN (R 4.1.0)
#> highr 0.9 2021-04-16 [1] CRAN (R 4.1.0)
#> htmltools 0.5.2.9000 2022-01-03 [1] Github (rstudio/htmltools@9490b62)
#> httr 1.4.2 2020-07-20 [1] CRAN (R 4.1.0)
#> jsonlite 1.8.0 2022-02-22 [1] CRAN (R 4.1.0)
#> KernSmooth 2.23-20 2021-05-03 [4] CRAN (R 4.0.5)
#> keyring 1.3.0 2021-11-29 [1] CRAN (R 4.1.0)
#> knitr 1.37 2021-12-16 [1] CRAN (R 4.1.0)
#> lattice 0.20-44 2021-05-02 [4] CRAN (R 4.1.0)
#> lifecycle 1.0.1 2021-09-24 [1] CRAN (R 4.1.0)
#> magrittr 2.0.2 2022-01-26 [1] CRAN (R 4.1.0)
#> openssl 2.0.0 2022-03-02 [1] CRAN (R 4.1.0)
#> ows4R 0.3 2022-03-11 [1] Github (eblondel/ows4r@5824587)
#> pillar 1.7.0 2022-02-01 [1] CRAN (R 4.1.0)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.1.0)
#> proxy 0.4-26 2021-06-07 [1] CRAN (R 4.1.0)
#> purrr 0.3.4 2020-04-17 [1] CRAN (R 4.1.0)
#> R.cache 0.15.0 2021-04-30 [1] CRAN (R 4.1.0)
#> R.methodsS3 1.8.1 2020-08-26 [1] CRAN (R 4.1.0)
#> R.oo 1.24.0 2020-08-26 [1] CRAN (R 4.1.0)
#> R.utils 2.11.0 2021-09-26 [1] CRAN (R 4.1.0)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 4.1.0)
#> raster 3.5-15 2022-01-22 [1] CRAN (R 4.1.0)
#> Rcpp 1.0.8 2022-01-13 [1] CRAN (R 4.1.0)
#> reprex 2.0.1 2021-08-05 [1] CRAN (R 4.1.0)
#> rgdal 1.5-28 2021-12-15 [1] CRAN (R 4.1.0)
#> rlang 1.0.2.9000 2022-03-07 [1] Github (r-lib/rlang@6ef0256)
#> rmarkdown 2.12.1 2022-03-07 [1] Github (rstudio/rmarkdown@b5a8002)
#> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.1.0)
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.1.0)
#> sf 1.0-7 2022-03-07 [1] CRAN (R 4.1.0)
#> sp 1.4-6 2021-11-14 [1] CRAN (R 4.1.0)
#> stringi 1.7.6 2021-11-29 [1] CRAN (R 4.1.0)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.1.0)
#> styler 1.6.2 2021-09-23 [1] CRAN (R 4.1.0)
#> terra 1.5-21 2022-02-17 [1] CRAN (R 4.1.0)
#> tibble 3.1.6 2021-11-07 [1] CRAN (R 4.1.0)
#> tidyselect 1.1.2 2022-02-21 [1] CRAN (R 4.1.0)
#> units 0.8-0 2022-02-05 [1] CRAN (R 4.1.0)
#> utf8 1.2.2 2021-07-24 [1] CRAN (R 4.1.0)
#> vctrs 0.3.8 2021-04-29 [1] CRAN (R 4.1.0)
#> withr 2.5.0 2022-03-03 [1] CRAN (R 4.1.0)
#> xfun 0.30 2022-03-02 [1] CRAN (R 4.1.0)
#> XML 3.99-0.9 2022-02-24 [1] CRAN (R 4.1.0)
#> yaml 2.3.5 2022-02-21 [1] CRAN (R 4.1.0)
#>
#> [1] /home/maelle/R/x86_64-pc-linux-gnu-library/4.1
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library
#>
#> ──────────────────────────────────────────────────────────────────────────────We were also seeing this error on GitHub Actions with more recent R versions.
What could also work I think is a way to pass an httr config httr::config() https://msmith.de/2020/10/02/httr-curl-ubuntu-20-04.html
I will dig a bit in the issue, and let you know when i have time to work on that. I will run the ows4R over the most recent R, and will probably include some tests over Emodnet WFS services.
Thank you! As it's quite critical for us (for the services with this problem, Linux users would need to tweak settings which is cumbersome) I'd be happy to help / test-drive this.
@maelle I've implemented a solution which extend present ows4R clients with a 'config' argument, to pass a httr::config() where you cn specify curl options
I've setup a test over Emodnet geology WFS and all seems ok now: See how it is configured:
Thanks so much @eblondel!! 🚀
you're welcome!