500 Error when Uploading Ubuntu Kernel Packages
derjohn opened this issue · comments
Hi,
I try to upload an AMD64 kernel from Ubuntu's Mainline Repo.
I worked until 4.16.5 kernel and stopped with 4.16.6 kernel.
(To reproduce look here: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.16.6/ )
I did set the logger to
- org.eclipse.jetty.server.Server logger to DEBUG
- org.sonatype.nexus.internal.web.ErrorPageServlet logger to DEBUG
And noticed that I looks like having to do with the xz compression format, which apache commons can't handle. I bet ubuntu changed that internal compression format to something more hipsterish inside the deb, but didn't really dig deeper.
I am on 3.9.0/1.0.5.
Here is the log output:
#2018-05-01 10:00:31,778+0000 DEBUG [qtp1790585161-43] *SYSTEM org.sonatype.nexus.internal.web.ErrorPageServlet - Attaching cause
javax.servlet.ServletException: Filtered request failed.
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:384)
at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at org.sonatype.nexus.repository.httpbridge.internal.ExhaustRequestFilter.doFilter(ExhaustRequestFilter.java:71)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:102)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:530)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: org/tukaani/xz/XZInputStream
at net.staticsnow.nexus.repository.apt.internal.hosted.AptHostedFacet.ingestAsset(AptHostedFacet.java:144)
at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56)
at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:54)
at net.staticsnow.nexus.repository.apt.internal.hosted.AptHostedHandler.handle(AptHostedHandler.java:59)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningHandler.handle(AptSigningHandler.java:43)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at net.staticsnow.nexus.repository.apt.internal.snapshot.AptSnapshotHandler.handle(AptSnapshotHandler.java:60)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.storage.UnitOfWorkHandler.handle(UnitOfWorkHandler.java:39)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.ContentHeadersHandler.handle(ContentHeadersHandler.java:44)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.http.PartialFetchHandler.handle(PartialFetchHandler.java:55)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.ConditionalRequestHandler.handle(ConditionalRequestHandler.java:72)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.ExceptionHandler.handle(ExceptionHandler.java:44)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.security.SecurityHandler.handle(SecurityHandler.java:52)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.TimingHandler.handle(TimingHandler.java:46)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.Context.start(Context.java:114)
at org.sonatype.nexus.repository.view.Router.dispatch(Router.java:63)
at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:52)
at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:43)
at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.dispatchAndSend(ViewServlet.java:211)
at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.doService(ViewServlet.java:173)
at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.service(ViewServlet.java:126)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85)
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.sonatype.nexus.repository.httpbridge.internal.ExhaustRequestFilter.doFilter(ExhaustRequestFilter.java:71)
at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108)
at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68)
at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:102)
at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
... 35 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.tukaani.xz.XZInputStream not found by org.apache.commons.compress [191]
at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1550)
at org.apache.felix.framework.BundleWiringImpl.access$200(BundleWiringImpl.java:79)
at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1958)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 99 common frames omitted
2018-05-01 10:00:31,788+0000 DEBUG [qtp179
Hm, it looks like I have xz V 1.5 : ./sonatype/nexus/system/org/tukaani/xz/1.5/xz-1.5.jar
According to Stackoverflow https://stackoverflow.com/questions/28229095/apache-commons-compress-using-7zip we need at least 1.6 ...
It looks like, the 1.5 comes with the nexus 3.10.0 release, so changing the pom.xml here in the plugin repo doesn't fetch it. (It's inside nexus-3.10.0-04-unix.tar.gz .... an outdated .m2 cache form the release engineer ?)
Fwiw, as a workaround, you can:
download the file.deb, and then manually perform:
ar xf file.deb
mkdir c d
(cd c; tar xf ../control.xz; tar czf ../control.gz *)
(cd d; tar xf ../data.xz; tar czf ../data.gz *)
ar -r file.deb debian-binary control.tar.gz data.tar.gzAnd then publish the file.
I've done that, because I needed a repo w/ the file and haven't had the energy to fight the java side (although I will probably look at some point).
@jsoref Thx for that workaround! I am quite sure, the issue will be resolved with nexus 3.12, as the XZ / Tukani ans commons compress will be updated. With 3.11 we still have the old s**t inside. I was able to build a nexus-3.12 for testing that, but I have no clue, how the nexus-base-template.*foo is converted into nexus-3.12xxxx.unix.tar.gz. Both look similar inside , but in detail certain things are missing (e.g. the initscripts) and I was not able to find that "transformation step" in any open repo on sonatype.
@DarthHater Can you help? Is there maybe a download URL for the 3.12-SNAPSHOT-unix.tar.gz ?
The pom file does specify that xz is a dependency, and Nexus 3.11 seems to ship with version 1.8. Maybe try adding
<bundle>mvn:org.tukaani/xz/1.8</bundle>
to this bit in the features file:
+ <feature name="nexus-repository-apt" description="net.staticsnow:nexus-repository-apt" version="1.0.6">
+ <details>net.staticsnow:nexus-repository-apt</details>
+ <bundle>mvn:net.staticsnow/nexus-repository-apt/1.0.6</bundle>
+ <bundle>mvn:org.apache.commons/commons-compress/1.16.1</bundle>
+ </feature>
where it looks like
+ <feature name="nexus-repository-apt" description="net.staticsnow:nexus-repository-apt" version="1.0.6">
+ <details>net.staticsnow:nexus-repository-apt</details>
+ <bundle>mvn:net.staticsnow/nexus-repository-apt/1.0.6</bundle>
+ <bundle>mvn:org.apache.commons/commons-compress/1.16.1</bundle>
+ <bundle>mvn:org.tukaani/xz/1.8</bundle>
+ </feature>
Not sure if it will work, but if it does we can update the readme
@derjohn snapshots I believe are not public, if they were they would be on RSO.
@mpoindexter I tried what you proposed - nexus started, but I was still not able to upload the Debian packages. But I don't see the stracktrace anymore, just "curl: (22) The requested URL returned error: 400 Invalid Debian package supplied". Besides the 1.8 library do we need to allow the packing algo in the plugin code somewhere?
In addition to needing the XZ bundle declared in the feature, there was a stupid bug (I forgot a break statement in a switch) in the parsing code. I've fixed it, it works now. Thanks for the bug report and testing out the feature changes.