Unable to install packages which are already cached

Question

Unable to install packages which are already cached

szhem opened this issue 6 years ago · comments

Thanks for creating an issue! Please fill out this form so we can be
sure to have all the information we need, and to minimize back and forth.

What are you trying to do?

Install the packages which have already been cached by proxy repository, but no longer available in remote one.

For example, here is a library libgcrypt20_1.6.5-2ubuntu0.4_amd64.deb in the repository

Trying to install exact version of this package leads to the following error

apt-cache policy shows that there is newer version available and it is not the same as the cached one

So it seems that plugin just downloads new metadata from the remote repository and then it's hardly possible to install the previously cached packages.

What feature or behavior is this required for?

To be able to install any package that has already been cached from the remote repository, because these ones just replace old package versions with the newer ones.

How could we solve this issue? (Not knowing is okay!)

Merging the new metadata of the remove repository with the already cached one.

Mike Poindexter · Answer 1 · Wed Jun 27 2018 02:37:24 GMT+0800 (China Standard Time)

I think the snapshot feature is what you want. This is working as intended.

szhem · Answer 2 · Wed Jun 27 2018 02:47:16 GMT+0800 (China Standard Time)

@mpoindexter is the snapshot feature available via UI?

Mike Poindexter · Answer 3 · Wed Jun 27 2018 02:52:32 GMT+0800 (China Standard Time)

No, it's not available in the UI. You can use like this:
https://github.com/sonatype-nexus-community/nexus-repository-apt#create-a-snapshot-of-the-current-package-lists-for-the-repo-that-can-be-pulled-from

szhem · Answer 4 · Wed Jun 27 2018 02:55:28 GMT+0800 (China Standard Time)

@mpoindexter, I've already created it, but cannot find out how to use the created snapshot.

Mike Poindexter · Answer 5 · Wed Jun 27 2018 03:01:53 GMT+0800 (China Standard Time)

You would just change the url in your sources.list to point to the snapshot instead of the root of the repo. So for example http://my-nexus-repo/repository/ubuntu/snapshots/my-snapshot

szhem · Answer 6 · Wed Jun 27 2018 05:16:27 GMT+0800 (China Standard Time)

@mpoindexter, seems I have difficulties creating snapshots, e.g. there is a following exception in the nexus.log

2018-06-26 21:49:31,252+0300 WARN  [qtp470173042-4549]  admin org.sonatype.nexus.repository.httpbridge.internal.ViewServlet - Failure servicing: MKCOL /repository/ubuntu/snapsho
ts/20180626
java.lang.NullPointerException: null
        at net.staticsnow.nexus.repository.apt.internal.snapshot.AptSnapshotFacetSupport.collectSnapshotItems(AptSnapshotFacetSupport.java:126)
        at net.staticsnow.nexus.repository.apt.internal.snapshot.AptSnapshotFacetSupport.createSnapshot(AptSnapshotFacetSupport.java:65)
        at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56)
        at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:54)
        at net.staticsnow.nexus.repository.apt.internal.snapshot.AptSnapshotHandler.handleSnapshotAdminRequest(AptSnapshotHandler.java:81)
        at net.staticsnow.nexus.repository.apt.internal.snapshot.AptSnapshotHandler.handle(AptSnapshotHandler.java:67)
        at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
        at org.sonatype.nexus.repository.storage.UnitOfWorkHandler.handle(UnitOfWorkHandler.java:39)
        at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
        at org.sonatype.nexus.repository.view.handlers.ContentHeadersHandler.handle(ContentHeadersHandler.java:44)
        at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
        at org.sonatype.nexus.repository.http.PartialFetchHandler.handle(PartialFetchHandler.java:55)
        at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
        at org.sonatype.nexus.repository.view.handlers.ConditionalRequestHandler.handle(ConditionalRequestHandler.java:72)
        at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
        at org.sonatype.nexus.repository.cache.NegativeCacheHandler.handle(NegativeCacheHandler.java:42)
        at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
        at org.sonatype.nexus.repository.view.handlers.ExceptionHandler.handle(ExceptionHandler.java:44)
        at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
        at org.sonatype.nexus.repository.security.SecurityHandler.handle(SecurityHandler.java:52)
        at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
        at org.sonatype.nexus.repository.view.handlers.TimingHandler.handle(TimingHandler.java:46)
        at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
        at org.sonatype.nexus.repository.view.Context.start(Context.java:114)
        at org.sonatype.nexus.repository.view.Router.dispatch(Router.java:64)
        at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:52)
        at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:43)
        at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.dispatchAndSend(ViewServlet.java:210)
        at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.doService(ViewServlet.java:172)
        at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.service(ViewServlet.java:126)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)

Mike Poindexter · Answer 7 · Wed Jun 27 2018 05:31:30 GMT+0800 (China Standard Time)

This is probably a configuration problem. See this thread: #46 for someone who had similar

szhem · Answer 8 · Wed Jun 27 2018 17:14:18 GMT+0800 (China Standard Time)

@mpoindexter thanks for pointing me to #46, I have had a configuration mistake.

Now I have the following errors while updating the packages cache from the snapshot

$ apt-get update
Get:1 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial InRelease [247 kB]
Ign:2 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/main Translation-en
Ign:3 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/restricted Translation-en
Ign:4 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/universe Translation-en
Ign:5 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/multiverse Translation-en
Ign:2 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/main Translation-en
Ign:3 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/restricted Translation-en
Ign:4 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/universe Translation-en
Ign:5 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/multiverse Translation-en
Ign:2 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/main Translation-en
Ign:3 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/restricted Translation-en
Ign:4 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/universe Translation-en
Ign:5 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/multiverse Translation-en
Err:2 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/main Translation-en
  404  Not Found
Ign:3 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/restricted Translation-en
Ign:4 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/universe Translation-en
Ign:5 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/multiverse Translation-en
Fetched 247 kB in 0s (873 kB/s)
Reading package lists... Done
E: Failed to fetch https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1/dists/xenial/main/i18n/Translation-en  404  Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.

In case the cache is updated from the main repository - everything is ok

$ apt-get update
Get:1 https://nexus/repository/ubuntu-16-cache xenial InRelease [247 kB]
Get:2 https://nexus/repository/ubuntu-16-cache xenial/main amd64 Packages [1,201 kB]
Get:3 https://nexus/repository/ubuntu-16-cache xenial/main i386 Packages [1,196 kB]
Get:4 https://nexus/repository/ubuntu-16-cache xenial/main Translation-en [568 kB]
Get:5 https://nexus/repository/ubuntu-16-cache xenial/restricted amd64 Packages [8,344 B]
Get:6 https://nexus/repository/ubuntu-16-cache xenial/restricted i386 Packages [8,684 B]
Get:7 https://nexus/repository/ubuntu-16-cache xenial/restricted Translation-en [2,908 B]
Get:8 https://nexus/repository/ubuntu-16-cache xenial/universe amd64 Packages [7,532 kB]
Get:9 https://nexus/repository/ubuntu-16-cache xenial/universe i386 Packages [7,512 kB]
Get:10 https://nexus/repository/ubuntu-16-cache xenial/universe Translation-en [4,354 kB]
Get:11 https://nexus/repository/ubuntu-16-cache xenial/multiverse amd64 Packages [144 kB]
Get:12 https://nexus/repository/ubuntu-16-cache xenial/multiverse i386 Packages [140 kB]
Get:13 https://nexus/repository/ubuntu-16-cache xenial/multiverse Translation-en [106 kB]
Fetched 23.0 MB in 3s (6,940 kB/s)                               
Reading package lists... Done

I understand that I can fix an issue with Acquire::Languages "none" option

$ apt-get update
Get:1 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial InRelease [247 kB]
Ign:2 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/main amd64 Packages
Ign:3 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/main i386 Packages
Ign:4 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/restricted amd64 Packages
Ign:5 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/restricted i386 Packages
Ign:6 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/universe amd64 Packages
Ign:7 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/universe i386 Packages
Ign:8 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/multiverse amd64 Packages
Ign:9 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/multiverse i386 Packages
Get:2 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/main amd64 Packages [1,201 kB]
Get:3 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/main i386 Packages [1,196 kB]
Get:4 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/restricted amd64 Packages [8,344 B]
Get:5 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/restricted i386 Packages [8,684 B]
Get:6 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/universe amd64 Packages [7,532 kB]
Get:7 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/universe i386 Packages [7,512 kB]
Get:8 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/multiverse amd64 Packages [144 kB]
Get:9 https://nexus/repository/ubuntu-16-cache/snapshots/20180627_1 xenial/multiverse i386 Packages [140 kB]
Fetched 18.0 MB in 2s (7,852 kB/s)                        
Reading package lists... Done

So, I'm just wondering whether this is an expected behaviour or I should file an issue so that snapshots will be able to snapshot Translation indexes too?