Error when installing Bioconductor packages

Question

Error when installing Bioconductor packages

rcannood opened this issue a year ago · comments

Hi Dirk!

Thanks for creating r2u! It has sped up the build times of our images significantly (From >30 mins to <5 mins?) !

Since the last update of eddelbuettel/r2u:22.04 (more specifically, "eddelbuettel/r2u@sha256:a4899fc475905331ff8a66f7bcbcefa261be1bd16447a34937df00c8a64af028"), I get the following error message when installing any Bioconductor package:

$ docker run eddelbuettel/r2u:22.04 Rscript -e 'install.packages("BiocParallel")'
Available system packages as root...
Get:1 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease [18.1 kB]
Hit https://ppa.launchpadcontent.net/edd/misc/ubuntu jammy InRelease            
Get:2 https://ppa.launchpadcontent.net/marutter/rrutter4.0/ubuntu jammy InRelease [17.5 kB]
Get:3 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy/main amd64 Packages [993 kB]
Ign https://r2u.stat.illinois.edu/ubuntu jammy InRelease                        
Get:4 https://r2u.stat.illinois.edu/ubuntu jammy Release [5713 B]               
Get:5 https://ppa.launchpadcontent.net/marutter/rrutter4.0/ubuntu jammy/main amd64 Packages [7623 B]
Get:6 https://r2u.stat.illinois.edu/ubuntu jammy Release.gpg [793 B]            
Get:7 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 Packages [2222 kB]  
Get:8 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]       
Get:9 https://r2u.stat.illinois.edu/ubuntu jammy/main all Packages [6923 kB]    
Get:10 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [798 kB]
Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [754 kB]
Hit http://archive.ubuntu.com/ubuntu jammy InRelease                            
Get:12 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [114 kB]        
Get:13 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [107 kB]      
Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1008 kB]
Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1078 kB]
Get:16 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [22.4 kB]
Get:17 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [49.0 kB]
Fetched 14.2 MB in 6s (24.0 kB/s)                                               
Error in dbs[bins, "Version"] : subscript out of bounds
Calls: install.packages -> .doTrace -> eval.parent -> eval -> eval
Execution halted

This did work with a previous version of r2u:

$ docker run 'eddelbuettel/r2u@sha256:1d3a92aab5abad11787cd6b6c9479960db9f4e56dcc7f837768da2e3f3c4dfe2' Rscript -e 'install.packages("BiocParallel")'
Install system packages as root...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Get:1 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease [18.1 kB]
Get:2 https://ppa.launchpadcontent.net/edd/misc/ubuntu jammy InRelease [18.1 kB]
...
Setting up r-cran-bh (1.81.0-1-1.ca2204.1) ...
Setting up r-cran-snow (0.4-4-1.ca2204.1) ...
Setting up r-cran-cpp11 (0.4.3-1.ca2204.1) ...
Setting up r-cran-futile.options (1.0.1-1.ca2204.1) ...
Setting up r-cran-formatr (1.14-1.ca2204.1) ...
Setting up r-cran-lambda.r (1.2.4-1.ca2204.1) ...
Setting up r-cran-futile.logger (1.4.3-1.ca2204.1) ...
Setting up r-bioc-biocparallel (1.32.4-1.ca2204.1) ...
Reading state information... Done
Get:1 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-formatr all 1.14-1.ca2204.1 [151 kB]
Get:2 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-lambda.r all 1.2.4-1.ca2204.1 [110 kB]
Get:3 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-futile.options all 1.0.1-1.ca2204.1 [19.4 kB]
Get:4 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-futile.logger all 1.4.3-1.ca2204.1 [94.6 kB]
Get:5 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-snow all 0.4-4-1.ca2204.1 [95.7 kB]
Get:6 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-bh all 1.81.0-1-1.ca2204.1 [8778 kB]
Get:7 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-cpp11 all 0.4.3-1.ca2204.1 [225 kB]
Get:8 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-bioc-biocparallel amd64 1.32.4-1.ca2204.1 [1350 kB]
Fetched 10.8 MB in 0s (0 B/s)

Am I doing something wrong or is this a bug?

Robrecht

cc @KaiWaldrant

Dirk Eddelbuettel · Answer 1 · Mon Feb 06 2023 20:14:12 GMT+0800 (China Standard Time)

An error is never good and neither is a regression. Let me take a look.

Dirk Eddelbuettel · Answer 2 · Mon Feb 06 2023 20:20:32 GMT+0800 (China Standard Time)

I see it too, also via install.r. @Enchufa2 is that possibly bspm?

root@011cfb62bd67:/# install.r BiocParallel      # using eddelbuettel/r2u:22.04
Available system packages as root...
Hit http://archive.ubuntu.com/ubuntu jammy InRelease                                       
Hit http://archive.ubuntu.com/ubuntu jammy-updates InRelease                       
Ign https://r2u.stat.illinois.edu/ubuntu jammy InRelease                                      
Hit https://r2u.stat.illinois.edu/ubuntu jammy Release                                          
Hit http://archive.ubuntu.com/ubuntu jammy-backports InRelease                   
Hit http://security.ubuntu.com/ubuntu jammy-security InRelease                      
Hit https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease  
Hit https://ppa.launchpadcontent.net/edd/misc/ubuntu jammy InRelease                           
Hit https://ppa.launchpadcontent.net/marutter/rrutter4.0/ubuntu jammy InRelease        
Fetched 0 B in 0s (0 B/s)                                                     
Error in dbs[bins, "Version"] : subscript out of bounds
root@011cfb62bd67:/#

Dirk Eddelbuettel · Answer 3 · Mon Feb 06 2023 20:23:13 GMT+0800 (China Standard Time)

It worls when you add type="binary" explicitly:

> install.packages("BiocParallel", type="binary")
Install system packages as root...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Reading package lists... Done    
Building dependency tree... Done 
Reading state information... Done
Get:1 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-formatr all 1.14-1.ca2204.1 [151 kB]          
Get:2 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-lambda.r all 1.2.4-1.ca2204.1 [110 kB]       
Get:3 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-futile.options all 1.0.1-1.ca2204.1 [19.4 kB] 
Get:4 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-futile.logger all 1.4.3-1.ca2204.1 [94.6 kB]   
Get:5 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-snow all 0.4-4-1.ca2204.1 [95.7 kB]               
Get:6 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-bh all 1.81.0-1-1.ca2204.1 [8778 kB]             
Get:7 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-cpp11 all 0.4.3-1.ca2204.1 [225 kB]               
Get:8 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-bioc-biocparallel amd64 1.32.4-1.ca2204.1 [1350 kB] 
Fetched 10.8 MB in 0s (0 B/s)
Fetched 10.8 MB in 0s (0 B/s)             
Selecting previously unselected package r-cran-formatr.
(Reading database ... 21709 files and directories currently installed.)
Preparing to unpack .../0-r-cran-formatr_1.14-1.ca2204.1_all.deb ...
Unpacking r-cran-formatr (1.14-1.ca2204.1) ...
Selecting previously unselected package r-cran-lambda.r.
Preparing to unpack .../1-r-cran-lambda.r_1.2.4-1.ca2204.1_all.deb ...
Unpacking r-cran-lambda.r (1.2.4-1.ca2204.1) ...
Selecting previously unselected package r-cran-futile.options.
Preparing to unpack .../2-r-cran-futile.options_1.0.1-1.ca2204.1_all.deb ...
Unpacking r-cran-futile.options (1.0.1-1.ca2204.1) ...
Selecting previously unselected package r-cran-futile.logger.
Preparing to unpack .../3-r-cran-futile.logger_1.4.3-1.ca2204.1_all.deb ...
Unpacking r-cran-futile.logger (1.4.3-1.ca2204.1) ...
Selecting previously unselected package r-cran-snow.
Preparing to unpack .../4-r-cran-snow_0.4-4-1.ca2204.1_all.deb ...
Unpacking r-cran-snow (0.4-4-1.ca2204.1) ...
Selecting previously unselected package r-cran-bh.
Preparing to unpack .../5-r-cran-bh_1.81.0-1-1.ca2204.1_all.deb ...
Unpacking r-cran-bh (1.81.0-1-1.ca2204.1) ...
Selecting previously unselected package r-cran-cpp11.
Preparing to unpack .../6-r-cran-cpp11_0.4.3-1.ca2204.1_all.deb ...
Unpacking r-cran-cpp11 (0.4.3-1.ca2204.1) ...
Selecting previously unselected package r-bioc-biocparallel.
Preparing to unpack .../7-r-bioc-biocparallel_1.32.4-1.ca2204.1_amd64.deb ...
Unpacking r-bioc-biocparallel (1.32.4-1.ca2204.1) ...
Setting up r-cran-bh (1.81.0-1-1.ca2204.1) ...
Setting up r-cran-snow (0.4-4-1.ca2204.1) ...
Setting up r-cran-cpp11 (0.4.3-1.ca2204.1) ...
Setting up r-cran-futile.options (1.0.1-1.ca2204.1) ...
Setting up r-cran-formatr (1.14-1.ca2204.1) ...
Setting up r-cran-lambda.r (1.2.4-1.ca2204.1) ...
Setting up r-cran-futile.logger (1.4.3-1.ca2204.1) ...
Setting up r-bioc-biocparallel (1.32.4-1.ca2204.1) ...
>

We will have to find a way to propagate that argument. I for starters need to update install.r.

Dirk Eddelbuettel · Answer 4 · Mon Feb 06 2023 20:43:48 GMT+0800 (China Standard Time)

And type="binary-source", the best default for bspm, also works. That is actually what I meant to try first :)

Dirk Eddelbuettel · Answer 5 · Mon Feb 06 2023 20:59:45 GMT+0800 (China Standard Time)

This would set a correct default for install2.r from littler:

## type should reflects bspm where available
if (opt$type == "getOption") {
    opt$type <- getOption("pkgType")
    if (requireNamespace("bspm", quietly=TRUE) && Sys.info()[["sysname"]] == "Linux") opt$type <- "binary-source"
}

I think I will also propagate the options() value for the r2u container. I had noticed the other day it was always taking 'the extra second' that binary-source avoids, and it does avoid the regression you found here.

Dirk Eddelbuettel · Answer 6 · Mon Feb 06 2023 21:16:58 GMT+0800 (China Standard Time)

Adding option(pkgType="binary") at the end of /etc/R/Rprofile.site also seems to help. I should probably add that in the r-bspm rocker container and then rebuild r2u.

Iñaki Ucar · Answer 7 · Mon Feb 06 2023 21:28:08 GMT+0800 (China Standard Time)

Admittedly, we need to raise a better error message in this case, but something is missing in the image:

$ docker run --rm -ti eddelbuettel/r2u:22.04 Rscript -e 'bspm::disable(); install.packages("BiocParallel")'
Untracing function "install.packages" in package "utils"
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
Warning message:
package ‘BiocParallel’ is not available for this version of R

A version of this package for your version of R might be available elsewhere,
see the ideas at
https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages

I.e. the Bioc repo.

Dirk Eddelbuettel · Answer 8 · Mon Feb 06 2023 21:35:52 GMT+0800 (China Standard Time)

I do not follow. We do have BiocParallel and install it fine if / when the correct type is set. Do

  docker run --rm -ti eddelbuettel/r2u:jammy apt-cache show r-bioc-biocparallel

Even without a BioConductor entry in repos :)

The Rocker containers never set BioC as a repo, in that sense it is @rcannood 's error to not set it, but then I also supported installing the binary before so in that sense it is my regression.

But is the best fix in your view? Do we risk other misbehavior by setting options(pkgType="binary") ?

Iñaki Ucar · Answer 9 · Mon Feb 06 2023 22:08:22 GMT+0800 (China Standard Time)

Yes, it's in binary form, but it's not found when running available.packages(), so when the hook tries to see if there is a more recent version from source, it fails, because it's not available from source. I'll check what the Windows/macOS path does in this case, because the error we get right now is not ok.

When you set the type to binary, it works because it doesn't look at the available.packages().

But is the best fix in your view? Do we risk other misbehavior by setting options(pkgType="binary") ?

I think it's better to set options(pkgType="binary-source") for now, because "binary" just fails if there is no binary, it doesn't try from source. Also, as you noted, this needs to be set after bspm::enable().

Dirk Eddelbuettel · Answer 10 · Mon Feb 06 2023 22:54:44 GMT+0800 (China Standard Time)

I tried options(pkgType="binary-source") -- it got me an error from R as that is a non-standard value. Apparently 'both' is allowed. Maybe you can switch bspm to use 'both' instead?

I will try to set if after bspm::enable().

What happens when the value is set in options() and use bspm::disable() ?

Iñaki Ucar · Answer 11 · Mon Feb 06 2023 23:10:05 GMT+0800 (China Standard Time)

bspm::enable() sets "both".
bspm::disable() sets "source" back, because that's the only mode that install.packages understands under Linux.
"both", "binary" and "binary-source" only work after bspm::enable(), because bspm knows how to deal with them.

Iñaki Ucar · Answer 12 · Mon Feb 06 2023 23:19:19 GMT+0800 (China Standard Time)

I discovered a bug in the "binary-source" path.

Iñaki Ucar · Answer 13 · Mon Feb 06 2023 23:34:13 GMT+0800 (China Standard Time)

So the best workaround right now would be to add the Bioc URL to the repos.

Iñaki Ucar · Answer 14 · Tue Feb 07 2023 00:14:49 GMT+0800 (China Standard Time)

Version 0.4.1.3 fixes the issue, now on master.

Dirk Eddelbuettel · Answer 15 · Tue Feb 07 2023 00:26:24 GMT+0800 (China Standard Time)

Nice.

Would it be possible to revert gracefully from binary-source as it is non-standard and use both which can be set in options(pkgType="both") whether or not bspm is loaded or even installed? Or do you need binary-source as a distinct value?

Iñaki Ucar · Answer 16 · Tue Feb 07 2023 00:35:23 GMT+0800 (China Standard Time)

We need binary-source as a distinct value, because we already use both as the default mode (meaning the same as on Windows / macOS, i.e. ask if there are newer versions from source).

Dirk Eddelbuettel · Answer 17 · Tue Feb 07 2023 09:05:11 GMT+0800 (China Standard Time)

Ok -- I rebuilt the rocker/r-bspm images for focal and jammy adding options(pkgType="binary-source") to the end of Rprofile.site, and pushed those. I then rebuilt the r2u images and pushed those. We should be back in business, so I am closing this. If you run into anything else that is unexpected please don't hesitate to contact us.