apt pinning seems to have no effect for r-cran-jquerylib

Question

apt pinning seems to have no effect for r-cran-jquerylib

florisvdh opened this issue 8 months ago · comments

Floris Vanderhaeghe commented 8 months ago

r-cran-jquerylib from R2U should be installed, but the one from the standard Ubuntu repo is installed despite the r2u apt repo has priority 700:

$ cat /etc/apt/preferences.d/99cranapt
Package: *
Pin: release o=CRAN-Apt Project
Pin: release l=CRAN-Apt Packages
Pin-Priority: 700

$ apt policy r-cran-jquerylib 
r-cran-jquerylib:
  Installed: 0.1.4+dfsg-3
  Candidate: 0.1.4+dfsg-3
  Version table:
 *** 0.1.4+dfsg-3 500
        500 http://ftp.belnet.be/ubuntu jammy/universe amd64 Packages
        500 http://ftp.belnet.be/ubuntu jammy/universe i386 Packages
        100 /var/lib/dpkg/status
     0.1.4-1.ca2204.1 700
        700 https://dirk.eddelbuettel.com/cranapt jammy/main all Packages
     0.1.4-1cran1.2204.0 500
        500 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu jammy/main amd64 Packages
        500 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu jammy/main i386 Packages
$ 
$ apt install r-cran-jquerylib 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
r-cran-jquerylib is already the newest version (0.1.4+dfsg-3).
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
$ 
$ inxi -Sxxx
System:
  Host: _ Kernel: 5.15.0-89-generic x86_64 bits: 64 compiler: gcc
    v: 11.4.0 Desktop: Cinnamon 5.8.4 tk: GTK 3.24.33 wm: muffin vt: 7
    dm: LightDM 1.30.0 Distro: Linux Mint 21.2 Victoria
    base: Ubuntu 22.04 jammy

I discovered it while inspecting a broken symlink such as jquerylib/lib/3.6.0/jquery-3.6.0.js in this package after it has been recorded (i.e. hardlinked) by renv into its cache, because these symlinks are defined relative. I suspect the relative symlinks (e.g. jquerylib/lib/3.6.0/jquery-3.6.0.js linking to ../../../../../../share/javascript/jquery/jquery.js) will be specific to the Ubuntu-packaged r-cran-jquerylib.

Dirk Eddelbuettel · Answer 1 · Wed Nov 29 2023 22:27:16 GMT+0800 (China Standard Time)

Hi @florisvdh -- thanks for bug report and excellent analysis. And I concur: the appended '+dfsg' from Debian sorts higher. You could experiment with a higher pin setting: different things happen at higher values especially above 900 and more. (The documentation is very dense on this, I would appreciate a second set of eyes.)

In general this should not matter. A recent Ubuntu build should be equivalent unless the package is "special" in some way. (Examples are littler where the pre-made build r2u leans on by default gets the wrong R_HOME, or bspm which drops its 'needs to be installed sudo' part.) This seems to be another one. I will look into tickling a custom build though this may require code surgery or manual intervention to alter the version number to sort higher.

(BTW you see to tell apt to look at r2y and c2d4u. As the former is a superset, may as well skip c2d4u.)

Dirk Eddelbuettel · Answer 2 · Wed Nov 29 2023 22:33:37 GMT+0800 (China Standard Time)

Can you try a manual (non-apt) download of https://dirk.eddelbuettel.com/tmp/r-cran-jquerylib_0.1.4+dfsg-4.ca2204.1_all.deb and see if it behaves relative to the required softlinks? It is built the standard way, and should sort higher.

Floris Vanderhaeghe · Answer 3 · Wed Nov 29 2023 23:23:23 GMT+0800 (China Standard Time)

Thanks for your advices @eddelbuettel.

I manually installed the debian package you link above. I confirm it does not have the symlinks, hence same result as with install.packages("jquerylib") from CRAN. Indeed, this version is seen as 'upgrading' relative to Ubuntu's version.

Floris Vanderhaeghe · Answer 4 · Thu Nov 30 2023 00:25:26 GMT+0800 (China Standard Time)

(BTW you see to tell apt to look at r2y and c2d4u. As the former is a superset, may as well skip c2d4u.)

Thanks. The use of c2d4u is still advertised at https://cran.r-project.org/bin/linux/ubuntu/; I guess that's why I still have it too. Maybe it could be stated on the CRAN page (or in the r2u docs) that only r2u is needed; this relationship between both is not documented AFAIK.

Dirk Eddelbuettel · Answer 5 · Thu Nov 30 2023 00:29:46 GMT+0800 (China Standard Time)

That is mostly because I did not want to step on the toes of @marutter but yes at some point we should mention r2u there too.

Note that the [r2u documentation[(https://eddelbuettel.github.io/r2u) (and scripts and Dockerfiles etc) do not set c2du.

The updated r-cran-jquerylib will go out on the next pulse. Thanks for the heads-up!

Floris Vanderhaeghe · Answer 6 · Thu Nov 30 2023 01:26:43 GMT+0800 (China Standard Time)

Great!

As you suggested, I have experimented with the pin setting. At least for this case, the value needs to be 1000 or more. So I think I'll leave it at 1000 now.

$ cat /etc/apt/preferences.d/99cranapt
Package: *
Pin: release o=CRAN-Apt Project
Pin: release l=CRAN-Apt Packages
Pin-Priority: 999

$ apt install r-cran-jquerylib 
[sudo] password for floris: 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
r-cran-jquerylib is already the newest version (0.1.4+dfsg-3).
0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.
$ 
$ sudo nano /etc/apt/preferences.d/99cranapt
$ 
$ cat /etc/apt/preferences.d/99cranapt
Package: *
Pin: release o=CRAN-Apt Project
Pin: release l=CRAN-Apt Packages
Pin-Priority: 1000

$ apt install r-cran-jquerylib 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be DOWNGRADED:
  r-cran-jquerylib
0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 5 not upgraded.
Need to get 298 kB of archives.
After this operation, 1535 kB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 https://dirk.eddelbuettel.com/cranapt jammy/main all r-cran-jquerylib all 0.1.4-1.ca2204.1 [298 kB]
Fetched 298 kB in 1s (204 kB/s)                         
dpkg: warning: downgrading r-cran-jquerylib from 0.1.4+dfsg-3 to 0.1.4-1.ca2204.1
(Reading database ... 568196 files and directories currently installed.)
Preparing to unpack .../r-cran-jquerylib_0.1.4-1.ca2204.1_all.deb ...
Unpacking r-cran-jquerylib (0.1.4-1.ca2204.1) over (0.1.4+dfsg-3) ...
Setting up r-cran-jquerylib (0.1.4-1.ca2204.1) ...
$

Note the downgrading warning, which of course is specific to this version of the r2u package.

Dirk Eddelbuettel · Answer 7 · Thu Nov 30 2023 01:32:35 GMT+0800 (China Standard Time)

I would love for you to test with 1000 for a few days and see if "see something odd happening". I have been thinking that a global override, while sounding scary, may actually be the right thing. I may turn it on for my laptop too. And could try it in CI as well.

BTW just querying apt-cache policy r-cran-jquerylib may be cheaper/faster. It should different sort order.

Floris Vanderhaeghe · Answer 8 · Thu Nov 30 2023 02:27:10 GMT+0800 (China Standard Time)

I would love for you to test with 1000 for a few days and see if "see something odd happening". I have been thinking that a global override, while sounding scary, may actually be the right thing. I may turn it on for my laptop too. And could try it in CI as well.

With pleasure. I'd be surprised to see consequences, but maybe I'm missing background. Will keep you informed. No news = good news.

BTW just querying apt-cache policy r-cran-jquerylib may be cheaper/faster. It should different sort order.

I had tried it first, but it doesn't order differently which is why above I did the installs. I think it just sorts according to version (the docs of apt-cache policy don't mention any sorting). While selecting r2u with value 1000, the package installation is still regarded as downgrading.

$ apt policy r-cran-jquerylib 
r-cran-jquerylib:
  Installed: 0.1.4-1.ca2204.1
  Candidate: 0.1.4-1.ca2204.1
  Version table:
     0.1.4+dfsg-3 500
        500 http://ftp.belnet.be/ubuntu jammy/universe amd64 Packages
        500 http://ftp.belnet.be/ubuntu jammy/universe i386 Packages
 *** 0.1.4-1.ca2204.1 1000
       1000 https://dirk.eddelbuettel.com/cranapt jammy/main all Packages
        100 /var/lib/dpkg/status
     0.1.4-1cran1.2204.0 500
        500 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu jammy/main amd64 Packages
        500 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu jammy/main i386 Packages

Dirk Eddelbuettel · Answer 9 · Thu Nov 30 2023 02:47:59 GMT+0800 (China Standard Time)

Yeah I quickly tried it too on my laptop and it didn't work. Score a -1 for apt-cache there. Sad.

But keep me posted on trying the higher pin.

Maxim Nazarov · Answer 10 · Fri Dec 01 2023 17:33:40 GMT+0800 (China Standard Time)

The updated r-cran-jquerylib will go out on the next pulse. Thanks for the heads-up!

I started getting errors today in my (docker-based) builds:
Error: Failed to fetch https://r2u.stat.illinois.edu/ubuntu/pool/dists/jammy/main/r-cran-jquerylib_0.1.4%2bdfsg-4.ca2204.1_all.deb 404 Not Found [IP: 192.17.190.167 443]
In the folder above I can only see https://r2u.stat.illinois.edu/ubuntu/pool/dists/jammy/main/r-cran-jquerylib_0.1.4-1.ca2204.1_all.deb so not sure why it tries to install the other one.

To reproduce it is sufficient to run docker build on:

FROM eddelbuettel/r2u:22.04
RUN install.r rmarkdown

Dirk Eddelbuettel · Answer 11 · Fri Dec 01 2023 19:55:56 GMT+0800 (China Standard Time)

Thank you, noted. Can you try now? A quick try in an empty container installed it, along with its six dependency packages.

[...]
Get:1 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-base64enc amd64 0.1-3-1.ca2204.1 [26.9 kB]                                                                                                     
Get:2 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-digest amd64 0.6.33-1.ca2204.1 [192 kB]                                                                                                        
Get:3 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-rlang amd64 1.1.2-1.ca2204.1 [1500 kB]                                                                                                         
Get:4 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-ellipsis amd64 0.3.2-1.ca2204.1 [34.5 kB]                                                                                                      
Get:5 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-fastmap amd64 1.1.1-1.ca2204.1 [61.7 kB]                                                                                                       
Get:6 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-htmltools amd64 0.5.7-1.ca2204.1 [347 kB]                                                                                                      
Get:7 https://r2u.stat.illinois.edu/ubuntu jammy/main all r-cran-jquerylib all 0.1.4+dfsg-4.ca2204.1 [298 kB]                                                                                                     
Fetched 2460 kB in 0s (0 B/s) 
[...]
Setting up r-cran-rlang (1.1.2-1.ca2204.1) ...
Setting up r-cran-base64enc (0.1-3-1.ca2204.1) ...
Setting up r-cran-digest (0.6.33-1.ca2204.1) ...
Setting up r-cran-fastmap (1.1.1-1.ca2204.1) ...
Setting up r-cran-ellipsis (0.3.2-1.ca2204.1) ...
Setting up r-cran-htmltools (0.5.7-1.ca2204.1) ...
Setting up r-cran-jquerylib (0.1.4+dfsg-4.ca2204.1) ...
root@56eb384b5146:/#

Dirk Eddelbuettel · Answer 12 · Fri Dec 01 2023 20:05:03 GMT+0800 (China Standard Time)

What may have happened with the unusual sorting between the two is that the newer one got indexed but then in the delete step got removed. I just took care of that manually leading to the improved situation. Thanks again for the heads-up!

Maxim Nazarov · Answer 13 · Fri Dec 01 2023 21:07:48 GMT+0800 (China Standard Time)

yes, worked also for me, thanks!

Dirk Eddelbuettel · Answer 14 · Sat Dec 02 2023 06:07:01 GMT+0800 (China Standard Time)

A colleague just noticed that the issue came back -- today's normal run produced a rebuild of 1.4.1 but that then sorted lower.

Very weird. So I now made a 1.4.1.0.1 -- the version number is fake but it'll sort higher for now. I will look into the sorting issue and see where/how I need to make an adjustment for version.

Floris Vanderhaeghe · Answer 15 · Mon Dec 04 2023 17:48:06 GMT+0800 (China Standard Time)

Confirmed that 0.1.4.0.1-1.ca2204.1 comes through, its version sorts highest (apt policy ranking order) and it will be picked (apt list --upgradable). Pin is at 1000.

$ apt policy r-cran-jquerylib 
r-cran-jquerylib:
  Installed: 0.1.4+dfsg-4.ca2204.1
  Candidate: 0.1.4.0.1-1.ca2204.1
  Version table:
     0.1.4.0.1-1.ca2204.1 1000
       1000 https://dirk.eddelbuettel.com/cranapt jammy/main all Packages
 *** 0.1.4+dfsg-4.ca2204.1 100
        100 /var/lib/dpkg/status
     0.1.4+dfsg-3 500
        500 http://ftp.belnet.be/ubuntu jammy/universe amd64 Packages
        500 http://ftp.belnet.be/ubuntu jammy/universe i386 Packages
     0.1.4-1cran1.2204.0 500
        500 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu jammy/main amd64 Packages
        500 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu jammy/main i386 Packages
$ 
$ apt list --upgradable | grep jquery

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

r-cran-jquerylib/jammy 0.1.4.0.1-1.ca2204.1 all [upgradable from: 0.1.4+dfsg-4.ca2204.1]