Installing FOSS puppet master broken or unclear

Question

Installing FOSS puppet master broken or unclear

antgel opened this issue 7 years ago · comments

I see that since the days of #146, the codebase has moved on, in particular with PR #177 . However, it's unclear how to install a FOSS master in the new world.

I see that there's a new file, hieradata/roles/puppetserver.yaml, which wasn't there before. In the old world, I created my own file with the following content, and it used to work:

---
profiles:
  - profile::puppet::gems
  - profile::puppet::foss_master
  - profile::puppet::foss_server_metrics

profile::puppet::gems::install_puppetserver_gems: true

In the new world, I've tried to update hieradata/roles/puppetserver.yaml according to the updated docs. I've added the three lines with # FOO.

---
psick::profiles::linux_classes:
  puppetserver: puppet::profile::server
  puppet_gems: psick::puppet::gems # FOO
  puppet_master: psick::puppet::foss_master # FOO
  psick::puppet::gems::install_puppetserver_gems: true # FOO

# Sample settings for a small footprint devel server
# puppet::profile::server::options:
#   java_args: '-Xms512m -Xmx512m -XX:MaxPermSize=256m'

But when I run papply.sh I get the following errors:

ubuntu@ip-10-10-10-249:~/src/git/psick/environments/production$ FACTER_role=puppetserver bash -x bin/papply.sh
+ PATH=/home/ubuntu/bin:/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/puppetlabs/bin:/opt/puppetlabs/server/bin
++ dirname bin/papply.sh
+ repo_dir=bin/..
+ . bin/../bin/functions
++ tty -s
+++ tput setaf 2
++ SETCOLOR_SUCCESS=''
+++ tput setaf 1
++ SETCOLOR_FAILURE=''
+++ tput setaf 3
++ SETCOLOR_WARNING=''
+++ tput setaf 13
++ SETCOLOR_ASK=''
+++ tput sgr0
++ SETCOLOR_NORMAL=''
+++ tput setaf 6
++ SETCOLOR_TITLE=''
+++ tput setaf 14
++ SETCOLOR_SUBTITLE=''
+++ tput setaf 15
++ SETCOLOR_BOLD=''
+ manifest=bin/../manifests/site.pp
+ extra_options=
+ PATH=/home/ubuntu/bin:/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/puppetlabs/bin:/opt/puppetlabs/server/bin:/opt/puppetlabs/puppet/bin
++ puppet --version
+ echo_title 'Running Puppet version 4.10.8 apply on bin/../manifests/site.pp'
+ echo

+ echo '### Running Puppet version 4.10.8 apply on bin/../manifests/site.pp ###'
### Running Puppet version 4.10.8 apply on bin/../manifests/site.pp ###
++ facter -p role
+ echo_subtitle 'Role: puppetserver - puppetserver'
+ echo '# Role: puppetserver - puppetserver'
# Role: puppetserver - puppetserver
+ puppet --version
+ grep '^[4|5]'
+ '[' x0 == x0 ']'
+ manifest_option=
+ which git
+ '[' x0 == x0 ']'
+ '[' -d bin/../.git ']'
+ config_version='/usr/bin/git --git-dir bin/../.git log --pretty=format:"%h - %an, %ad : %s" -1'
+ puppet apply --verbose --report --show_diff --summarize --modulepath bin/../site:bin/../modules:/etc/puppetlabs/code/modules --environmentpath bin/../.. --hiera_config=bin/../hiera3.yaml '--config_version=/usr/bin/git --git-dir bin/../.git log --pretty=format:"%h - %an, %ad : %s" -1' --detailed-exitcodes bin/../manifests/site.pp
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Warning: /home/ubuntu/src/git/psick/environments/production/hiera3.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5
   (in /home/ubuntu/src/git/psick/environments/production/hiera3.yaml)
Warning: Undefined variable '::env';
   (file & line not available)
Warning: Undefined variable '::zone';
   (file & line not available)
Warning: Undefined variable 'env';
   (file & line not available)
Warning: Undefined variable 'zone';
   (file & line not available)
Warning: Undefined variable 'psick::auto_conf'; class psick has not been evaluated
   (file & line not available)
Warning: The function 'hiera_hash' is deprecated in favor of using 'lookup'. See https://docs.puppet.com/puppet/4.10/reference/deprecated_language.html
   (file & line not available)
Warning: Unknown variable: 'dns_alt_name_entries'. at /home/ubuntu/src/git/psick/environments/production/modules/psick/manifests/puppet/foss_master.pp:45:16
Warning: ModuleLoader: module 'puppetdb' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
   (file & line not available)
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Pattern[]. There is further documentation for validate_legacy function in the README. at ["/home/ubuntu/src/git/psick/environments/production/modules/puppetdb/manifests/server.pp", 91]:["/home/ubuntu/src/git/psick/environments/production/modules/psick/manifests/profiles.pp", 68]
   (at /home/ubuntu/src/git/psick/environments/production/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in `deprecation')
Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::String. There is further documentation for validate_legacy function in the README. at ["/home/ubuntu/src/git/psick/environments/production/modules/puppetdb/manifests/server/jetty.pp", 74]:["/home/ubuntu/src/git/psick/environments/production/modules/psick/manifests/profiles.pp", 68]
   (at /home/ubuntu/src/git/psick/environments/production/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in `deprecation')
Error: Evaluation Error: Error while evaluating a Function Call, undefined method `downcase' for nil:NilClass at /home/ubuntu/src/git/psick/environments/production/modules/psick/manifests/profiles.pp:68:11 on node ip-10-10-10-249.ec2.internal
+ result=1
+ '[' x1 == x0 ']'
+ '[' x1 == x2 ']'
+ echo_failure 'There were errors in the Puppet run'
+ echo 'There were errors in the Puppet run'
There were errors in the Puppet run
+ return 1
+ exit 1

So, a couple of questions:

What's the reason for the error? Am I doing something wrong?
What's the purpose of the puppetserver: puppet::profile::server line, when according to the docs, the way to install the server is the following lines:

puppet_gems: psick::puppet::gems
puppet_master: psick::puppet::foss_master
psick::puppet::gems::install_puppetserver_gems: true

Antony Gelberg · Answer 1 · Thu Oct 12 2017 21:07:44 GMT+0800 (China Standard Time)

Oops. 😊 Obviously psick::puppet::gems::install_puppetserver_gems: true was indented and appeared as a linux_class. Sorry for the noise.

Still interested to know the answer to question 2 at the end of my post. In order to get things to run, I had to comment out the puppetserver: puppet::profile::server line and leave in the:

puppet_gems: psick::puppet::gems
puppet_master: psick::puppet::foss_master
psick::puppet::gems::install_puppetserver_gems: true

Martin Alfke · Answer 2 · Thu Oct 12 2017 22:37:58 GMT+0800 (China Standard Time)

The profile::puppet::server is no longer existing (and I suppose it never really has been there).
The documentation you have linked in your first post also does not show anything related to the mentioned profile.
We have moved all profile:: items to psick::

Please let us know whether you are able to spin up a FOSS puppet server and whether we can close this issue.

Antony Gelberg · Answer 3 · Fri Oct 13 2017 01:00:56 GMT+0800 (China Standard Time)

Hi, puppetserver: puppet::profile::server still seems to exist: https://github.com/example42/psick/blob/production/hieradata/role/puppetserver.yaml

Unfortunately I'm still having big problems getting the FOSS master up. Here are the relevant log extracts:

Debug: Executing with uid=postgres gid=postgres: '/usr/bin/psql -d puppetdb -t -c "SELECT COUNT(*) FROM (SELECT t.count FROM (SELECT count(extname) FROM pg_extension WHERE extname = 'pg_trgm') as t WHERE t.count = 1) count"'
Error: /Stage[main]/Puppetdb::Database::Postgresql/Postgresql::Server::Extension[pg_trgm]/Postgresql_psql[Add pg_trgm extension to puppetdb]: Could not evaluate: Error evaluating 'unless' clause, returned pid 9730 exit 1: ''
Debug: Executing: '/usr/sbin/service postgresql status'
Error: /Stage[main]/Postgresql::Server::Reload/Exec[postgresql_reload]: Failed to call refresh: invalid byte sequence in US-ASCII
Error: /Stage[main]/Postgresql::Server::Reload/Exec[postgresql_reload]: invalid byte sequence in US-ASCII

I don't know why the first error happens. I ran the query manually when Puppet had finished, and it worked, although I don't think the pg_trgm extension is loaded.

ubuntu@puppet:~$ sudo -u postgres /usr/bin/psql -d puppetdb -t -c "SELECT COUNT(*) FROM (SELECT t.count FROM (SELECT count(extname) FROM pg_extension WHERE extname = 'pg_trgm') as t WHERE t.count = 1) count"
     0
ubuntu@puppet:~$ sudo -u postgres /usr/bin/psql -d puppetdb -t -c "SELECT * FROM pg_extension"
 plpgsql |       10 |           11 | f              | 1.0        |           |

I think the reason for the second one is related to lack of UTF-8. The output of /usr/sbin/service postgresql status does contain a non-ASCII character. I'm running this stuff from an AWS userdata bash script. When I add locale > /tmp/locale.out to the script, it seems the script is running with a locale of POSIX, but yet, when I log in normally to the machine, the locale is correct, both for the ubuntu user and for root.

ubuntu@puppet:~$ cat /tmp/locale.out
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
ubuntu@puppet:~$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_US.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=en_US.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=en_US.UTF-8
LC_NAME=en_US.UTF-8
LC_ADDRESS=en_US.UTF-8
LC_TELEPHONE=en_US.UTF-8
LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION=en_US.UTF-8
LC_ALL=
ubuntu@puppet:~$ sudo locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_US.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=en_US.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=en_US.UTF-8
LC_NAME=en_US.UTF-8
LC_ADDRESS=en_US.UTF-8
LC_TELEPHONE=en_US.UTF-8
LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION=en_US.UTF-8
LC_ALL=

Puppet logs also contain lines like:

Could not find locales configuration file for puppetlabs-catalog_preview at /home/ubuntu/src/git/psick/environments/production/modules/catalog_preview/locales/config.yaml. Skipping i18n initialization.

for every module that r10k tries to load. But I'm not sure if that's a harmless warning or a bug.

I tried running bin/papply.sh twice in my user data script, but it didn't help. It does work when I run bin/papply.sh from a ssh login (with the correct login). I also tried with postgres-9.6 instead of the default 9.4 that installs on my setup, but it was no different. I'm using Ubuntu 16.04 LTS.

Any help you can give on this is appreciated, just let me know what you need to help debug.

Antony Gelberg · Answer 4 · Fri Oct 13 2017 02:08:49 GMT+0800 (China Standard Time)

By adding export LANG="en_US.UTF-8"to the script, the second error goes away. It's strange, because I didn't have to do that a month ago, but something must have changed, perhaps in the AMI, perhaps in Puppet, perhaps in psick, I don't know.

Anyway, I think once the pg_trgm error is solved, the master will be up. Any clues?

Antony Gelberg · Answer 5 · Fri Oct 13 2017 03:46:32 GMT+0800 (China Standard Time)

Okay, I finally see the forest through the trees. Somehow, puppetlabs-postgresql is actually trying to run that query before postgres is installed. I think it's to do with puppetlabs/puppetlabs-postgresql@50d510c#diff-b0fb13f182fc009f5b0b34ff75b16acfR48, or something else in that commit. I pinned puppetlabs-postgresql to 5.1.0 (the latest 5.2.0 was released a few days ago), and things run fine. How would you like to treat this, both from a psick and an upstream point of view? My puppet-fu isn't good enough to submit an upstream PR.

Martin Alfke · Answer 6 · Fri Oct 13 2017 04:58:03 GMT+0800 (China Standard Time)

Hi @antgel , many thanks for reporting the issue. I suppose that upstrema has now fixed the missing pg_trgm extension. We have this also in our FOSS server profile.
I will cross check after returning from PupperConf.

Antony Gelberg · Answer 7 · Fri Oct 13 2017 05:02:58 GMT+0800 (China Standard Time)

It's not that the extension is missing as such, it's just that the module tries to configure it before Postgres is even installed. I commented to the author of the upstream PR. Hope PuppetConf was good, wish I'd been!

Alessandro Franceschi · Answer 8 · Sun Nov 05 2017 01:37:36 GMT+0800 (China Standard Time)

@antgel @tuxmea Documentation updated (https://github.com/example42/psick/blob/integration/docs/FOSS_puppet_server.md) and some cleanup done. Postgres errors are still there: 2 puppet apply runs are needed.
We can wait to close this one once fix is done on the postgres module.