Using within RStudio

Question

Using within RStudio

robjhyndman opened this issue 2 years ago · comments

While this works using R running in a terminal, I'm getting an error when installing from within RStudio:

> install.packages("hts")
Error in install.packages : cannot connect to the system package manager

On KDE Neon 5.25 (based on Ubuntu 20.04). Using R 4.2.1 and RStudio 2022.07.1 Build 554.

Dirk Eddelbuettel · Answer 1 · Thu Jul 28 2022 08:03:41 GMT+0800 (China Standard Time)

That's known and documented (somewhere) and really unfortunate: bspm hooks around the base R functions, but RStudio (and also renv) all do their own thing so no bspm integration.

Sad as that may be for users of those interfaces, apt still works.

Dirk Eddelbuettel · Answer 2 · Thu Jul 28 2022 08:06:22 GMT+0800 (China Standard Time)

Actually, re-reading. If install.packages does not work maybe the sudo part does not work. Paging @Enchufa2...

Dirk Eddelbuettel · Answer 3 · Thu Jul 28 2022 08:48:32 GMT+0800 (China Standard Time)

Counter-example from my Ubuntu 22.04 laptop with a standard bspm installation (but with sudo=TRUE) and a standard RStudio nightly from my PPA (shown in the screenshot) fetching them two times a week.

(Sorry for posting an image, copying text form the console here does not seem to work well.)

So we would need more details from you about your setup that fails given my existence proof

Rob J Hyndman · Answer 4 · Thu Jul 28 2022 13:29:24 GMT+0800 (China Standard Time)

How to do a standard bspm installation with sudo=TRUE? I tried
sudo apt install r-cran-bpsm
and
sudo R CMD INSTALL bspm_0.3.9.tar.gz
but same result.

Iñaki Ucar · Answer 5 · Thu Jul 28 2022 16:05:00 GMT+0800 (China Standard Time)

This usually happens when bspm is installed in the user library or, as a system package, it is not properly configured (and I think that r-cran-bspm in c2d4u is not; I pinged Michael about this, but got no response), see e.g. cran4linux/bspm/issues/40. So the solution is:

Most importantly, uninstall any trace of bspm copies in your system, especially anything in the user dir, which is loaded with higher priority.
Install bspm using sudo:

sudo Rscript --vanilla -e 'install.packages("bspm", repos="https://cran.r-project.org")'

Iñaki Ucar · Answer 6 · Thu Jul 28 2022 16:13:38 GMT+0800 (China Standard Time)

BTW, we implemented a warning suggesting this fix (v0.3.9.1) but never rolled an update. :( Now CRAN submissions are closed, so maybe in August. I'll add a troubleshooting section to the README in the meanwhile.

Dirk Eddelbuettel · Answer 7 · Thu Jul 28 2022 19:26:21 GMT+0800 (China Standard Time)

Also, one may need

$ tail -3 /etc/R/Rprofile.site 
suppressMessages(bspm::enable())
options(bspm.sudo=TRUE)

$

which the sudo install may add -- it has been a while for me.

Iñaki Ucar · Answer 8 · Thu Jul 28 2022 20:04:37 GMT+0800 (China Standard Time)

Nope, the bspm.sudo option is mainly for non-root containers. I suspect we are talking about a desktop installation here, and this option would interfere.

Dirk Eddelbuettel · Answer 9 · Thu Jul 28 2022 23:29:10 GMT+0800 (China Standard Time)

Are you sure it interferes? Replying (on last-in-Ontario day) from laptop which a) has r2u via bspm for all CRAN content and b) that option and c) no issues whatsoever.

(Also just ran grep -c and I appear to have serverd over 10k deb files since the weekly log rotation (early Sunday). So getting to over 2k/day which is nice.)

Dirk Eddelbuettel · Answer 10 · Thu Jul 28 2022 23:45:04 GMT+0800 (China Standard Time)

Actually maybe because I use Graham's wajig as a wrapper I already have root anyway and it may just be a no-op. Then again it also did not get in the way in the example I posted above.

Iñaki Ucar · Answer 11 · Fri Jul 29 2022 01:47:12 GMT+0800 (China Standard Time)

Yes, I'm sure. This option was added with the container use case in mind, and it is documented as follows:

Note that, if you want to fall back to sudo in a non-interactive session, you need to set options(bspm.sudo=TRUE).

I.e., for non-interactive sessions with password-less sudo. It does not get in the way when you have root access anyway. But it does for the D-Bus mechanism.

Iñaki Ucar · Answer 12 · Fri Jul 29 2022 01:51:52 GMT+0800 (China Standard Time)

For more details, here's where it gets in the way:

https://github.com/Enchufa2/bspm/blob/718eaa73ddec51d863beaf9f437663e2d1eab7d5/R/bridge.R#L43-L51

If sudo is preferred (bspm.sudo=TRUE), then the D-Bus call never happens. It you have root access in the first place, then the direct route is taken.

Rob J Hyndman · Answer 13 · Fri Jul 29 2022 07:37:12 GMT+0800 (China Standard Time)

The problem was fixed when I re-installed bspm using

sudo Rscript --vanilla -e 'install.packages("bspm", repos="https://cran.r-project.org")'

and then removed options(bspm.sudo=TRUE) from Rprofile.site. Now install.package() uses apt install in both RStudio and R in a terminal.

Iñaki Ucar · Answer 14 · Fri Jul 29 2022 07:51:29 GMT+0800 (China Standard Time)

Thanks for confirming! Here's a link for future reference: https://enchufa2.github.io/bspm/#cannot-connect-to-the-system-package-manager

Dirk Eddelbuettel · Answer 15 · Fri Jul 29 2022 13:27:09 GMT+0800 (China Standard Time)

Thanks for confirming. If you have suggestions about how/where we should state this more prominently on the r2u side please do let us know.

Rob J Hyndman · Answer 16 · Fri Jul 29 2022 14:12:39 GMT+0800 (China Standard Time)

Maybe add something after the last para of the "Usage" section in the Readme to explain

bspm must be installed via sudo Rscript ...
Add bspm::enable() should be added to Rprofile.site.

Although I don't know how widely this applies. At least it worked for Ubuntu 20.04.

Dirk Eddelbuettel · Answer 17 · Fri Jul 29 2022 14:18:04 GMT+0800 (China Standard Time)

Thanks -- I'll try to add this this weekend.

Iñaki Ucar · Answer 18 · Fri Jul 29 2022 16:16:48 GMT+0800 (China Standard Time)

I've also updated the installation instructions for Ubuntu in the bspm's README to recommend the installation from CRAN with sudo, see cran4linux/bspm@9d940eb.

gueyenono · Answer 19 · Wed Nov 09 2022 01:01:07 GMT+0800 (China Standard Time)

This usually happens when bspm is installed in the user library or, as a system package, it is not properly configured (and I think that r-cran-bspm in c2d4u is not; I pinged Michael about this, but got no response), see e.g. Enchufa2/bspm/issues/40. So the solution is:

Most importantly, uninstall any trace of bspm copies in your system, especially anything in the user dir, which is loaded with higher priority.

Install bspm using sudo:
sudo Rscript --vanilla -e 'install.packages("bspm", repos="https://cran.r-project.org")'

Do you have to have tips to effectively complete step 1? Thank you.

Dirk Eddelbuettel · Answer 20 · Wed Nov 09 2022 01:16:15 GMT+0800 (China Standard Time)

You mean uninstalling bspm? You can always rely on installed.packages():

> ip <- installed.packages()
> ip[ ip[,"Package"] == "bspm", 1:3]
     Package LibPath                         Version 
bspm "bspm"  "/usr/local/lib/R/site-library" "0.3.10"
bspm "bspm"  "/usr/lib/R/site-library"       "0.3.5" 
>

Does that help? On this machine it tells me two locations of bspm installations.

gueyenono · Answer 21 · Wed Nov 09 2022 01:29:55 GMT+0800 (China Standard Time)

@eddelbuettel Thank you very much. It definitely does! I'll now follow the proposed solution to the problem of the system not being able to connect to the system package manager.

Dirk Eddelbuettel · Answer 22 · Wed Nov 09 2022 01:35:50 GMT+0800 (China Standard Time)

Ok, keep us posted. I also heard from @Enchufa2 that there may be a fix in RStudio because it had to do with whether Rprofile.site is or is not read consistently. I mostly work outside of RStudio when I update my packages so I would not know -- but I do run the nightlies so I could test if we need a fresh "yes / no" on this.

gueyenono · Answer 23 · Wed Nov 09 2022 08:56:59 GMT+0800 (China Standard Time)

@eddelbuettel It is unfortunately still not working for me.

I located the bspm package in my system using the method you shared with me.
Then I deleted the folder using sudo rm -rf bspm
Then, I reinstalled bspm with the code provided above:

sudo Rscript --vanilla -e 'install.packages("bspm", repos="https://cran.r-project.org")'

Finally, I added suppressMessages(bspm::enable()) to the bottom of /etc/R/Rprofile.site

Is there anything else I could try or is there any piece of information you need?

My distribution is Pop OS, which is currently based on Ubuntu 22.04.

Dirk Eddelbuettel · Answer 24 · Wed Nov 09 2022 10:55:33 GMT+0800 (China Standard Time)

Which version of RStudio are you running? (Should be under Help -> About)

gueyenono · Answer 25 · Wed Nov 09 2022 13:37:45 GMT+0800 (China Standard Time)

I actually run the daily builds, but I haven't got the newest one. My current version is:

2022.12.0 Build 310

Iñaki Ucar · Answer 26 · Wed Nov 09 2022 17:43:04 GMT+0800 (China Standard Time)

Did you install the required dependencies? I.e.,

$ sudo apt-get update && sudo apt-get install python3-{dbus,gi,apt}

@eddelbuettel BTW, these are not mentioned in the README of this repo.

Dirk Eddelbuettel · Answer 27 · Wed Nov 09 2022 21:26:40 GMT+0800 (China Standard Time)

@gueyenono On the laptop on which I use r2u as the default, I just removed a package (on the command-line via sudo dpkg --purge r-cran-rcppspdlog) and then (in RStudio) called install.packages("RcppSpdlog") on its R name and got the r2u binary back thanks to bspm. Worked like a charm inside RStudio. I am running 2022.12.0 Build 295 (and update on this machine every few days as I have a 'local PPA' with a twice-a-week cron job). (I have to host RStudio locally as the binaries are too large for GitHub). I will double check the website to update the RStudio language. (I also just checked that it works via the 'Packages' pane -- that also seems to work though fewer messages seems to make it to the console. dpkg -l still shows a reinstall.) So I think this all works.

@Enchufa2 Yes, I should update that too. In the beginning I always relied upon a r-cran-bspm binary I provided that had the dependencies. Now we show the (simpler for me ;-) ) sudo Rscript ... installation so we have add this hint. Will do.

gueyenono · Answer 28 · Wed Nov 09 2022 21:26:51 GMT+0800 (China Standard Time)

@Enchufa2 Yes, I installed all the required dependencies. I also would like to note that when I don't add options(bspm.sudo = TRUE) to /etc/R/Rprofile.site, I get an error saying:

Error: dbus: Call failed: Could not activate remote peer: unit failed.

Also, if I want to restart the entire setup process from scratch, is wiping off the bspm package from my system the only thing to do? Thank you.

Dirk Eddelbuettel · Answer 29 · Wed Nov 09 2022 21:32:25 GMT+0800 (China Standard Time)

@gueyenono For the record I do NOT have that (any more). I used to, but @Enchufa2 patiently conviced me that we can avoid the need for bspm.sudo = TRUE. As I understand it (and it has been months on this laptop) the key is sudo Rscript -e 'install.packages("bspm")'. That gives me bspm in /usr/local/lib/R/site-library which for me (and unless changed, all Debian/Ubuntu machines with the r-base-core binary) is first (and hence "winning" entry) in .libPaths()

gueyenono · Answer 30 · Wed Nov 09 2022 21:49:38 GMT+0800 (China Standard Time)

Is it possible at all that these may cause my issues?

When I add the first repository with sudo add-apt-repository ppa:marutter/rrutter4.0, I get the following warnings and errors. My lack of solidified experience with Linux is the reason for me not being able to troubleshoot the prerequisite steps myself. I was wondering if this had to do with me not being able to use r2u?

Err:25 https://ppa.launchpadcontent.net/mattrose/terminator/ubuntu jammy Release 404 Not Found [IP: 185.125.190.52 443]
Err:26 https://ppa.launchpadcontent.net/tj/bugfixes/ubuntu jammy Release 404 Not Found [IP: 185.125.190.52 443]
W: https://qgis.org/ubuntu/dists/jammy/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'https://qgis.org/ubuntu jammy InRelease' doesn't support architecture 'i386'
E: The repository 'https://ppa.launchpadcontent.net/mattrose/terminator/ubuntu jammy Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'https://ppa.launchpadcontent.net/tj/bugfixes/ubuntu jammy Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

Dirk Eddelbuettel · Answer 31 · Wed Nov 09 2022 21:50:55 GMT+0800 (China Standard Time)

Yes, very likely. You should try to take care of it. I think my script (and the Dockerfile) do. It's a bit of a pain that it changed between focal and jammy.

FWIW, on my laptop (with the 'personal PPA' for RStudio, and docker and brave and whatnot) -- no warnings whatsoever. This matters!

edd@ahmad:~$ sudo apt update
Ign:1 https://dirk.eddelbuettel.com/ppa-rstudio ./ InRelease
Hit:2 https://dirk.eddelbuettel.com/ppa-rstudio ./ Release   
Hit:3 http://us.archive.ubuntu.com/ubuntu jammy InRelease
Hit:5 http://us.archive.ubuntu.com/ubuntu jammy-updates InRelease
Hit:6 http://us.archive.ubuntu.com/ubuntu jammy-backports InRelease
Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease       
Hit:8 https://download.docker.com/linux/ubuntu jammy InRelease       
Hit:9 https://brave-browser-apt-release.s3.brave.com stable InRelease 
Ign:10 https://r2u.stat.illinois.edu/ubuntu jammy InRelease                    
Hit:11 https://dl.google.com/linux/chrome/deb stable InRelease             
Hit:12 https://r2u.stat.illinois.edu/ubuntu jammy Release                   
Hit:13 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease 
Hit:15 https://ppa.launchpadcontent.net/edd/misc/ubuntu jammy InRelease
Hit:16 https://apt.syncthing.net syncthing InRelease      
Hit:14 https://packagecloud.io/slacktechnologies/slack/debian jessie InRelease 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
1 package can be upgraded. Run 'apt list --upgradable' to see it.
edd@ahmad:~$

@Enchufa2 README (and docs/) and scripts update.

gueyenono · Answer 32 · Wed Nov 09 2022 21:55:49 GMT+0800 (China Standard Time)

@eddelbuettel Well noted then. Thank you very much for all the hard work and help. Many thanks to @Enchufa2 as well.