Giters

voxpupuli / puppet-archive

Compressed archive file download and extraction with native types/providers for Windows and Unix

Home Page:https://forge.puppet.com/puppet/archive

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Missing gem with ruby 3.1

jas01 opened this issue · comments

commented

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 7.23
  • Ruby: 3.1
  • Distribution: FreeBSD
  • Module version: All

How to reproduce (e.g Puppet code you use)

Just run the puppet agent, if the module puppet-archive are installed the agent would failed

What are you seeing

When the puppet agent run (well start) it failed with this message

Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not autoload puppet/provider/archive/ruby: Unable to find archive module in modulepath /usr/local/etc/puppet/modules:/usr/local/etc/puppet/modules
Error: Could not autoload puppet/provider/archive/curl: Could not autoload puppet/provider/archive/ruby: Unable to find archive module in modulepath /usr/local/etc/puppet/modules:/usr/local/etc/puppet/modules
Error: Could not autoload puppet/type/archive: Could not autoload puppet/provider/archive/curl: Could not autoload puppet/provider/archive/ruby: Unable to find archive module in modulepath /usr/local/etc/puppet/modules:/usr/local/etc/puppet/modules
Error: Failed to apply catalog: Could not autoload puppet/type/archive: Could not autoload puppet/provider/archive/curl: Could not autoload puppet/provider/archive/ruby: Unable to find archive module in modulepath /usr/local/etc/puppet/modules:/usr/local/etc/puppet/modules

The reason was find by @smortex (big thanks to him) is with ruby 3.1 the net-ftp become a gem (check https://www.ruby-lang.org/en/news/2021/12/25/ruby-3-1-0-released/) the module puppet-archive need this gem. So without this gem installed the agent cannot run

What behaviour did you expect instead

The module puppet-archive should installed the gem so don't need to installed manually on each client (and because puppet agent without puppet-archive don't need net-ftp)

commented

@bastelfreak Which ruby is Puppet 8 going to ship with? If ruby 3.1 or higher, maybe the AIO packages should vendor this gem??

commented

Ruby 3.2

commented

https://stdgems.org/ net-ftp is a stdlib gem. The ruby package in freebsd is supposed to vendor this (or at least have a dependency on it, but tha causes other issues. I'm fighting with this on Arch Linux). If Puppet creates the AIO packages properly, the gem will be vendored. I recommend raising a bug on the freebsd site.

commented

👋 @bastelfreak good catch! I saw there was a net-ftp gem on Rubygem and though this was removed from the gems shipped with Ruby. I was not aware of stdgems which shows this should still be bundled into the default ruby.

So this is a FreeBSD specific issue I think due to the way we package Ruby:
https://cgit.freebsd.org/ports/tree/lang/ruby31/pkg-message
There is a chicken and egg problem (using this specific net/ftp as an example, but the issue is the same for each gem listed in the link above):

  • ftp/rubygem-net-ftp has a dependency on lang/ruby31;
  • lang/ruby31 should have a dependency on ftp/rubygem-net-ftp (to match stdgems).

I wonder if we can try to lazy-load net-ftp only when trying to use it, and only cause a resource application failure if prerequisites are not found, instead of having a catalog compilation failure when net-ftp is not available as we have today? If we can do this, we can even have the archive module ensure net-ftp is installed, and in the worst case scenario, the configuration should be applied successfully after 2 puppet run.

commented

We tried really long on arch linux to ship each stdlib gem as own package. it's painful, it didn't work in the end and the ruby developers recommend vendoring the gems. I think that's the current goal for arch linux.

commented

Hey just checking if there's any movement on this?
Got hit by it on the ci while adding support for Puppet 8 to Java:
https://github.com/puppetlabs/puppetlabs-java/actions/runs/4598161712/jobs/8121839822?pr=548#step:6:33

This is on Ubuntu

commented

@david22swan are you sure that's related? The error message is a different one.

commented

hmmm, maybe not I guess, had zeroed in on the Could not autoload puppet/type/archive: Could not autoload puppet/provider/archive/curl: Could not autoload puppet/provider/archive/ruby so thought they had to be at least related, though you are right, the final error does differ.

commented

Yet, the backtrace ends with:

     # --- Caused by: ---
     # LoadError:
     #   cannot load such file -- net/ftp
     #   ./spec/fixtures/modules/archive/lib/puppet_x/bodeco/util.rb:148:in `<class:FTP>'

🙄

@david22swan just to be sure, what happen when you add net-ftp to the Gemfile?

commented

@smortex Adding the gem got my test's running green

commented

So as I understand it, Ruby 3.2 on GitHub Action does not include net-ftp, so when installing puppet as a gem it does not pull net-ftp and the archive code does not load properly. I'll try to investigate lazy loading net-ftp / not failing hard when the gem is not found to see if it can help and if it make sense to integrate it into the archive module.

commented

Fun: when net-ftp is not installed but you prevent this failure, the archive module use curl if it is available to download the file over FTP. I'll open a PR to not fail hard when the net-ftp gem is not found so that it can rely on another system to download, and only if no download solution is found fail at catalog application time.

commented

#491 should fix the issue