add_polygon (and similar) crash using Rstudio when fill_colour is not blank on Windows
deanm0000 opened this issue · comments
Edit by dcooley:
Until a fix is found, the current working solution to this appears to be downgrading to at the latests R 4.1.3
Describe the bug
Using the fill_colour parameter in shapes crashes Rstudio in windows.
If I remove the fill_colour
parameter then it produces the map just fine (although without colors). I put the source code in the editor and the specific place that it crashes is on the line
shape <- mapdeck:::rcpp_polygon_geojson(data, l, geometry_column,
digits)
Oddly enough, if I run the same example in RGui then it doesn't crash but it does hang for at least 5 minutes before it returns results whereas Rstudio announces the fatal error session crash within 5 seconds.
To Reproduce
If I just use the example from the Colours page of the guide, ie.
pp <- spatialwidget::widget_melbourne
mapdeck() %>%
add_polygon(
data = pp , fill_colour = "SA2_NAME"
)
that will produce a crash (or if in Rgui will take 5+ minutes to complete)
Expected behaviour
Not crashing ;)
Screenshots
If applicable, add screenshots to help explain your problem.
Versions
sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.utf8 LC_CTYPE=English_United States.utf8 LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C LC_TIME=English_United States.utf8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] sf_1.0-8 mapdeck_0.3.40004 RPostgres_1.4.4
loaded via a namespace (and not attached):
[1] tidyselect_1.1.2 purrr_0.3.4 jsonify_1.2.1 vctrs_0.4.1 generics_0.1.3 htmltools_0.5.3
[7] utf8_1.2.2 blob_1.2.3 rlang_1.0.4 e1071_1.7-11 pillar_1.8.0 glue_1.6.2
[13] DBI_1.1.3 bit64_4.0.5 lifecycle_1.0.1 htmlwidgets_1.5.4 fastmap_1.1.0 class_7.3-20
[19] fansi_1.0.3 Rcpp_1.0.9 KernSmooth_2.23-20 classInt_0.4-7 bit_4.0.4 hms_1.1.1
[25] digest_0.6.29 dplyr_1.0.9 grid_4.2.1 cli_3.3.0 tools_4.2.1 magrittr_2.0.3
[31] proxy_0.4-27 tibble_3.1.8 spatialwidget_0.2.4 pkgconfig_2.0.3 ellipsis_0.3.2 rstudioapi_0.13
[37] R6_2.5.1 units_0.8-0 compiler_4.2.1
That's very odd. Do you know anyone else with a Windows machine who can test this as well?
I am on Windows and i can confirm the behaviour.
R version 4.2.0 (2022-04-22 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22000)
Matrix products: default
locale:
[1] LC_COLLATE=German_Austria.utf8 LC_CTYPE=German_Austria.utf8 LC_MONETARY=German_Austria.utf8 LC_NUMERIC=C
[5] LC_TIME=German_Austria.utf8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] mapdeck_0.3.4
loaded via a namespace (and not attached):
[1] stringfish_0.15.7 Rcpp_1.0.9 rstudioapi_0.13.0-9000 magrittr_2.0.3 qs_0.25.3 tidyselect_1.1.2
[7] RApiSerialize_0.1.0 lattice_0.20-45 R6_2.5.1 rlang_1.0.4 fastmap_1.1.0 fansi_1.0.3
[13] dplyr_1.0.9 tools_4.2.0 grid_4.2.0 config_0.3.1 utf8_1.2.2 DBI_1.1.3
[19] cli_3.3.0 htmltools_0.5.3 ellipsis_0.3.2 digest_0.6.29 assertthat_0.2.1 yaml_2.3.5
[25] RcppParallel_5.1.5 tibble_3.1.7 lifecycle_1.0.1 purrr_0.3.4 htmlwidgets_1.5.4 vctrs_0.4.1
[31] glue_1.6.2 sp_1.5-0 compiler_4.2.0 pillar_1.8.0 generics_0.1.2 spatialwidget_0.2.3
[37] pkgconfig_2.0.3
does it crash if you use a hex string literal as the fill colour?
mapdeck() %>%
add_polygon(
data = spatialwidget::widget_melbourne
, fill_colour = "#FF0000"
)
yes it is still crashing
GIven it's working on the CRAN checks, and this is not consistently crashing for other windows users, I really don't know how to solve this one...
Are either of you using any obscure C++ compilers?
I tried to dig into your package, but I have some problems installing it.
First it is not working because of the LinkingTo spatialwidget (>= 0.2.1005)
I have spatialwidget 0.2.3 installed, which is not recognized as a newer package.
When I remove the (>= 0.2.1005) dependency, I get this error:
fatal error: geometries/utils/utils.hpp: No such file or directory
6 | #include "geometries/utils/utils.hpp"
I get the same error when I try remotes::install_version("spatialwidget", version = "0.2.2")
and I have sfheaders ‘0.4.0’ installed.
I dont think I have a special C++ compiler. Is there a way to check that?
Could I ask you to update to all the latest dependencies as well, so in a brand new session:
remotes::install_github("dcooley/geometries")
remotes::install_github("dcooley/sfheaders")
remotes::install_github("SymbolixAU/geojsonsf")
remotes::install_github("SymbolixAU/colourvalues")
remotes::install_github("SymbolixAU/spatialwidget")
remotes::install_github("SymbolixAU/mapdeck")
and then try again?
I installed geojsonsf
from SymbolixAU
, because the dcooley
fork is still at version 1.3.2.
Anyway, Rstudio is still crashing..
And I also can't build mapdeck due to this error:
fatal error: geometries/geometries.hpp: No such file or directory
13 | #include "geometries/geometries.hpp"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [C:/PROGRA~1/R/R-42~1.0/etc/x64/Makeconf:259: arc.o] Error 1
ERROR: compilation failed for package 'mapdeck'
if {geometries}
installed correctly you should be able to run this code:
# remotes::install_github("dcooley/geometries")
library(Rcpp)
cppFunction(
includes = '
#include "geometries/geometries.hpp"
',
code = '
SEXP a_point( SEXP x ) {
return geometries::matrix::to_geometry_matrix( x );
}
'
, depends = "geometries"
)
a_point(1:3)
Yeah i can run that.
> a_point(1:3)
[,1] [,2] [,3]
[1,] 1 2 3
and does this line run?
sfheaders:::rcpp_sfg_point(1:2, NULL, "XY")
Yes, also runs.
sfheaders:::rcpp_sfg_point(1:2, NULL, "XY")
[,1] [,2]
[1,] 1 2
attr(,"class")
[1] "XY" "POINT" "sfg"
and this one
js <- spatialwidget::widget_point( data = spatialwidget::widget_capitals, fill_colour = "country", legend = TRUE)
substr( js$data, 1, 200 )
and then this one
l <- widget_polygon( widget_melbourne[1:2, ], fill_colour = "AREASQKM16", legend = F)
substr( l$data, 1, 200 )
no, this line crashes RStudio:
js <- spatialwidget::widget_point( data = spatialwidget::widget_capitals, fill_colour = "country", legend = TRUE)
ok I've just pushed an update to spatialwidget to force the correct down-stream versions align. Could you restart you ression, then
remotes::install_github("dcooley/geometries")
remotes::install_github("dcooley/sfheaders")
remotes::install_github("SymbolixAU/spatialwidget")
js <- spatialwidget::widget_point( data = spatialwidget::widget_capitals, fill_colour = "country", legend = TRUE)
It still crashes.
Not sure if it helps, but in the Rstudio session log I am getting this Error:
ERROR CLIENT EXCEPTION (rsession-user): (TypeError) : Cannot read property 'O' of null;|||org/rstudio/studio/client/workbench/views/source/editors/text/AceEditor.java#4509::setScrollSpeed|||org/rstudio/studio/client/workbench/views/source/editors/text/AceEditorMonitor.java#46::monitor|||org/rstudio/studio/client/workbench/views/source/editors/text/AceEditorMonitor.java#70::execute|||com/google/gwt/core/client/impl/SchedulerImpl.java#140::execute|||com/google/gwt/core/client/impl/Impl.java#306::apply|||com/google/gwt/core/client/impl/Impl.java#345::entry0|||rstudio-0.js#-1::eval|||com/google/gwt/cell/client/AbstractEditableCell.java#41::viewDataMap|||Client-ID: 33e600bb-c1b1-46bf-b562-ab5cba070b0e|||User-Agent: Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) QtWebEngine/5.12.8 Chrome/69.0.3497.128 Safari/537.36
this?
n <- 5
params <- list(fill_colour = "col1")
df <- data.frame(col1 = 1:n)
lst_params <- spatialwidget:::rcpp_construct_params(df, params)
data <- df
total_colours <- nrow( df )
repeats <- 1L
lst_defaults <- list(fill_colour = rep(1,5))
colour_name <- "fill_colour"
opacity_name <- "fill_colour"
lst_legend <- list("stroke_colour" = TRUE)
include_legend <- FALSE
colour_format = "hex"
res <- spatialwidget:::rcpp_resolve_colour(
lst_params,
params,
data,
lst_defaults,
colour_name,
opacity_name,
lst_legend,
include_legend,
repeats,
total_colours,
colour_format
)
This runs and returns:
res
$defaults
$defaults$fill_colour
[1] "#44015400" "#3B528B3F" "#21908C7F" "#5DC963BF" "#FDE725FF"
$legend
$legend$stroke_colour
[1] TRUE
ok this is starting to narrow it down. I'll see if I can carry on debugging tomorrow.
Thanks for your help.
actually one last one
df <- spatialwidget::widget_capitals
l <- list(fill_colour = "country")
res <- spatialwidget:::rcpp_construct_params(df, l)
should get
$parameter
[1] "fill_colour"
$parameter_type
[1] 16
$data_column_index
[1] 0
yes that works
res
$parameter
[1] "fill_colour"
$parameter_type
[1] 16
$data_column_index
[1] 0
sorry for not keeping current. I just went through the latest troubleshooting steps and got the same results as trafficonese
I'm having the same problem. Both stroke_colour and fill_colour hang up for me but it only happens when I assign a hex value. If I assign a column, at least fill_colour works as expected.
There's been a few updates to the imported libraries' C++ code in the last few weeks. Could you try updating those pacakges, and try this again?
## restart your R session
install.packages(c("geometries","sfheaders","colourvalues"))
install.packages("mapdeck")
I've recently tried to use fill_colour() and regardless of what function of mapdeck I try to use it in it crashes my session. I've tried to run this on two computers both running windows and its the same issue.
Reverting to R 4.1.2 seems to have worked
Can someone try these code chunks and let me know if either crash:
df <- spatialwidget::widget_capitals
l <- list(fill_colour = "country", stroke_colour = "capital", legend = T)
data_rows <- nrow( df )
lst_defaults <- list(
fill_colour = rep(1.0, data_rows)
, stroke_colour = rep(1.0, data_rows)
)
layer_legend <- c("fill_colour","stroke_colour")
parameter_exclusions <- c("legend","legend_options","palette","na_colour")
res <- spatialwidget:::rcpp_params_to_data(
df, l, lst_defaults, layer_legend, data_rows, parameter_exclusions,
factors_as_string = TRUE
)
n <- 5
lst_params <- list(parameter = c("stroke_colour", "stroke_width"), parameter_type = c(), data_column_index = c() )
params <- list(stroke_colour = "col1", stroke_width = 3)
param_names <- lst_params[[ "parameter" ]]
df <- data.frame(col1 = 1:n)
data_names <- names( df )
data <- df
data_rows <- nrow( df )
lst_defaults <- list(stroke_opacity = rep(100, n))
spatialwidget:::rcpp_construct_data( param_names, params, data_names, lst_defaults, data, data_rows )
# stroke_opacity stroke_colour stroke_width
# 1 100 1 3
# 2 100 2 3
# 3 100 3 3
# 4 100 4 3
# 5 100 5 3
df <- spatialwidget::widget_capitals
l <- list(fill_colour = "country", stroke_colour = "capital", legend = T)
p <- spatialwidget:::rcpp_construct_params( data = df, params = l)
pn <- names( l )
ll <- c("fill_colour","stroke_colour")
spatialwidget:::rcpp_construct_legend_list(
lst_params = p
, params = l
, param_names = pn
, legend_types = ll
)
# $fill_colour
# [1] TRUE
#
# $stroke_colour
# [1] TRUE
Hi, the first one is crashing (the last expression, on R 4.3.1, not on R 4.1.3 where it is working fine)
Second and third are OK and not crashing.
ok one more:
lst <- list(let = letters, vals = 1:5)
new_lst <- spatialwidget:::rcpp_remove_list_elements(lst, "vals")
That one works perfectly
$let
[1] "a" "b" "c" "d" "e" "f" "g" "h" "i" "j" "k" "l" "m" "n" "o" "p" "q" "r" "s" "t" "u" "v" "w" "x" "y" "z"
ok we have to go old-school because I have no idea what's causing this, and I can't reproduce it:
could you / someone
- clone the spatialwidget library
git clone https://github.com/symbolixau/spatialwidget
- checkout the
stopping
branch -git checkout stopping
- build the pacakge and run this code
df <- spatialwidget::widget_capitals
l <- list(fill_colour = "country", stroke_colour = "capital", legend = T)
data_rows <- nrow( df )
lst_defaults <- list(
fill_colour = rep(1.0, data_rows)
, stroke_colour = rep(1.0, data_rows)
)
layer_legend <- c("fill_colour","stroke_colour")
parameter_exclusions <- c("legend","legend_options","palette","na_colour")
res <- spatialwidget:::rcpp_params_to_data(
df, l, lst_defaults, layer_legend, data_rows, parameter_exclusions,
factors_as_string = TRUE
)
- hopefully you should get the error
Error: 1
-
Then go to the
inst/include/spatialwidget/parameters/parameters.hpp
file and comment out line 87: https://github.com/SymbolixAU/spatialwidget/blob/stopping/inst/include/spatialwidget/parameters/parameters.hpp#L87 -
Run teh code again. You should see
Error: 2
-
comment out the next
Rcpp::stop()
line and run again. Repeat this until you get the crash, and let me know whichRcp::stop()
occurs before and which occurs after the crash?
1 to 4 : ok
Running the code generate Error: 1
5 ok and 6 Error: 2
Crashed instead of Error: 5 (so I saw Error: 4, commented out line 120 and not line 148
Any new updates on this issue?
does it crash if you include the fill_opacity
argument?
pp <- spatialwidget::widget_melbourne
mapdeck() %>%
add_polygon(
data = pp,
fill_colour = "SA2_NAME",
fill_opacity = 1.0
)
Yeah, the above code crashed for me. However, it doesn't crash if I leave out fill_colour.
mapdeck() %>%
add_polygon(
data = pp,
#fill_colour = "SA2_NAME",
fill_opacity = 0.2
)
could your run each of the colour tests and palette tests?
Colours
library(testthat)
library(spatialwidget)
n <- 5
params <- list(fill_colour = "col1")
df <- data.frame(col1 = 1:n)
lst_params <- spatialwidget:::rcpp_construct_params(df, params)
data <- df
total_colours <- nrow( df )
repeats <- 1L
lst_defaults <- list(fill_colour = rep(1,5))
colour_name <- "fill_colour"
opacity_name <- "fill_colour"
lst_legend <- list("stroke_colour" = TRUE)
include_legend <- FALSE
colour_format = "hex"
res <- spatialwidget:::rcpp_resolve_colour(
lst_params,
params,
data,
lst_defaults,
colour_name,
opacity_name,
lst_legend,
include_legend,
repeats,
total_colours,
colour_format
)
expect_equal(
res$defaults$fill_colour, colourvalues::colour_values(1:5, alpha = 1:5)
)
n <- 5
params <- list(fill_colour = "col1")
df <- data.frame(col1 = 1:n)
lst_params <- spatialwidget:::rcpp_construct_params(df, params)
data <- df
total_colours <- nrow( df )
repeats <- 1L
lst_defaults <- list(fill_colour = rep(1,5))
colour_name <- "fill_colour"
opacity_name <- "not_a_column_will_use_default"
lst_legend <- list("stroke_colour" = TRUE)
include_legend <- FALSE
colour_format = "hex"
res <- spatialwidget:::rcpp_resolve_colour(
lst_params,
params,
data,
lst_defaults,
colour_name,
opacity_name,
lst_legend,
include_legend,
repeats,
total_colours,
colour_format
)
expect_equal(
res$defaults$fill_colour, colourvalues::colour_values(1:5)
)
n <- 5
params <- list(fill_colour = "col1")
df <- data.frame(col1 = 1:n)
lst_params <- spatialwidget:::rcpp_construct_params(df, params)
data <- df
total_colours <- nrow( df )
repeats <- 1L
lst_defaults <- list(fill_colour = rep(1,5))
colour_name <- "fill_colour"
opacity_name <- "fill_colour"
lst_legend <- list("stroke_colour" = TRUE)
include_legend <- FALSE
colour_format = "rgb"
res <- spatialwidget:::rcpp_resolve_colour(
lst_params,
params,
data,
lst_defaults,
colour_name,
opacity_name,
lst_legend,
include_legend,
repeats,
total_colours,
colour_format
)
expect_equal(
res$defaults$fill_colour, colourvalues::colour_values_rgb(1:5, alpha = 1:5)
)
Palette
lst_params <- list(parameter = c("stroke_colour", "palette", "stroke_width"), parameter_type = c(), data_column_index = c() )
params <- list(stroke_colour = "col1", palette = "mypalette", stroke_width = 3)
expect_equal( spatialwidget:::rcpp_resolve_palette( lst_params, params ), "mypalette" )
lst_params <- list(parameter = c("stroke_colour", "stroke_width"), parameter_type = c(), data_column_index = c() )
params <- list(stroke_colour = "col1", stroke_width = 3)
spatialwidget:::rcpp_resolve_palette( lst_params, params )
expect_equal( spatialwidget:::rcpp_resolve_palette( lst_params, params ), "viridis" )
res <- spatialwidget:::rcpp_colour_str_with_palette("viridis", letters, 255.0, "#808080FF", TRUE, "fill_colour" )
col <- colourvalues::color_values(letters, summary = T)
expect_equal( res$colours, col$colours )
expect_equal( res$summary_colours, col$summary_colours )
expect_equal( res$summary_values, col$summary_values )
res <- spatialwidget:::rcpp_colour_num_with_palette("viridis", 1:26, 255.0, "#808080FF", TRUE, "fill_colour" )
col <- colourvalues::color_values(1:26, n_summaries = 5, format = T)
expect_equal( res$colours, col$colours )
expect_equal( res$summary_colours, col$summary_colours )
expect_equal( res$summary_values, col$summary_values )
res <- spatialwidget:::rcpp_colour_num_with_palette("viridis", 1:26, 255.0, "#808080FF", TRUE, "fill_colour", 5)
col <- colourvalues::color_values(1:26, n_summaries = 5, format = T, digits = 5)
expect_equal( res$colours, col$colours )
expect_equal( res$summary_colours, col$summary_colours )
expect_equal( res$summary_values, col$summary_values )
Hi, I've had the same behavior since last year.
@dcooley I assume you ask to run the tests with the dev version of spatialwidget, otherwise spatialwidget:::rcpp_resolve_colour will throw an error of unused arguments.
I get successful results for all colour and palette tests except the first colour test, which throws:
Error: res$defaults$fill_colour not equal to colourvalues::colour_values(1:5).
4/5 mismatches
x[1]: "#44015400"
y[1]: "#440154FF"
x[2]: "#3B528B3F"
y[2]: "#3B528BFF"
x[3]: "#21908C7F"
y[3]: "#21908CFF"
x[4]: "#5DC963BF"
y[4]: "#5DC963FF"
I just updated all the relevant packages and don't get any errors for the tests, seems like I had an old version of colourvalues. Mapdeck still doesn't work if using fill_colour. It doesn't crash for me, it just hangs forever and I have to restart the R session.
I manually added stops to see where the crash happens and I think it leads to here: https://github.com/SymbolixAU/spatialwidget/blob/stopping/inst/include/spatialwidget/colour/colour.hpp#L60
ok we can test this directly:
does this work?
library(Rcpp)
cppFunction(
depends = c( "spatialwidget")
, includes = c('#include "spatialwidget/utils/colour/colour.hpp"')
, code = 'void testCString() {
Rcpp::StringVector colour_vec = {"#FFFFFF"};
Rcpp::String first_colour = colour_vec[0];
if ( spatialwidget::utils::colour::is_hex( first_colour.get_cstring() ) ) {
Rcpp::Rcout << "true" << std::endl;
} else {
Rcpp::Rcout << "false" << std::endl;
}
}'
)
testCString()
# true
That code hangs forever for me and makes the R session unresponsive, same as when using fill_colour in mapdeck.
Ok, I did some extra digging and found out that the problem comes from the C++ regex library in Windows. More specifically, the code hangs at this line: https://github.com/SymbolixAU/spatialwidget/blob/4679fc3695665323407114151a9e2630677e7231/inst/include/spatialwidget/utils/colour/colour.hpp#L12
I found that others have solved this by avoiding the use of regex entirely.
So I cloned spatialwidget, and edited the is_hex()
function in spatialwidget/inst/include/spatialwidget/utils/colour/colour.hpp
as follows:
inline bool isHexDigit(char c) {
return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F');
}
inline bool is_hex( std::string str ) {
int len = str.length();
if (len != 7 && len != 9 && len != 4 && len != 5) {
return false; // Invalid length for a hex color
}
if (str[0] != '#') {
return false; // Missing "#" at the beginning
}
for (int i = 1; i < len; i++) {
if (!isHexDigit(str[i])) {
return false; // Non-hexadecimal character found
}
}
return true; // Passed all checks, valid hex color
}
Then I installed this modified version of spatialwidget, reinstalled mapdeck from source using this version of spatialwidget, and voilá! Mapdeck seems to be finally working with fill_colour!
Not sure if this is the best implementation for is_hex()
but seems like this is the culprit.
good work - I'll update the is_hex
function with your code, add some tests, then push to CRAN.