Issue with Cloudformation Template
KuruvaSomaSekhar opened this issue · comments
Today I tried to deploy flocker and docker swarm using Cloudformation template provided here. I started in N.Virginia region , but AMI provided in the default template doesn't exists in that region. I changed to Ubuntu Server 16.04 (ami-f4cc1de2). Then I started with required input parameters. It's started but after some time it rollbacked. Then I checked in Cloudformation Template-Resources location, it was saying that -WaitCondition timed out. Received 0 conditions when expecting 1. It started only three instances (2 Agent nodes, 1 Control node) instead of four nodes.
I logged in into Control node and observed logs in "/var/log/cloud-init-output.log" ,it's mostly saying like flocker itself not downloaded. Docker also not started before starting flocker installation. I'm not sure it's issue with Image I used or provided Cloudformation Template .
Find below logs from Control node
Cloud-init v. 0.7.8 running 'init-local' at Wed, 15 Mar 2017 08:53:53 +0000. Up 7.15 seconds.
Cloud-init v. 0.7.8 running 'init' at Wed, 15 Mar 2017 08:53:55 +0000. Up 9.11 seconds.
ci-info: +++++++++++++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++++++++++++
ci-info: +--------+------+----------------------------+---------------+-------+-------------------+
ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |
ci-info: +--------+------+----------------------------+---------------+-------+-------------------+
ci-info: | eth0 | True | 172.31.26.206 | 255.255.240.0 | . | 0e:a1:07:24:af:7e |
ci-info: | eth0 | True | fe80::ca1:7ff:fe24:af7e/64 | . | link | 0e:a1:07:24:af:7e |
ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . | . |
ci-info: | lo | True | ::1/128 | . | host | . |
ci-info: +--------+------+----------------------------+---------------+-------+-------------------+
ci-info: +++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++
ci-info: +-------+-------------+-------------+---------------+-----------+-------+
ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags |
ci-info: +-------+-------------+-------------+---------------+-----------+-------+
ci-info: | 0 | 0.0.0.0 | 172.31.16.1 | 0.0.0.0 | eth0 | UG |
ci-info: | 1 | 172.31.16.0 | 0.0.0.0 | 255.255.240.0 | eth0 | U |
ci-info: +-------+-------------+-------------+---------------+-----------+-------+
Generating public/private rsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_rsa_key.
Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub.
The key fingerprint is:
SHA256:38p+R1RnC09upgJKxoX9rtUjGRxImjov6RRGvjK5P6c root@ip-172-31-26-206
The key's randomart image is:
+---[RSA 2048]----+
| +.. |
| .o+ . . .+|
| ..o. o . =oo|
| o .+ . + .* |
| *o S o +.+ |
| o *. . B +. |
| + = . + +.. |
| *... o .. . |
| ..E+ .+. . |
+----[SHA256]-----+
Generating public/private dsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_dsa_key.
Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub.
The key fingerprint is:
SHA256:0u4Q90o3dkwAfv6vUveFaUJS58a9anSlzhvMq+VZ2TM root@ip-172-31-26-206
The key's randomart image is:
+---[DSA 1024]----+
| . |
| . . . . |
| . o. + . |
| . o... + o|
| o S .o.. +o|
| = . +o+*o+|
| . o =.=*BE+|
| + +.o o=+|
| o .+o. |
+----[SHA256]-----+
Generating public/private ecdsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_ecdsa_key.
Your public key has been saved in /etc/ssh/ssh_host_ecdsa_key.pub.
The key fingerprint is:
SHA256:ktnkrGXJjNHeVS+NUZzNyznlu65sMZdYwv9rWdtwSVk root@ip-172-31-26-206
The key's randomart image is:
+---[ECDSA 256]---+
| o++|
| . . *E|
| . o o +.O|
| & o . o X.|
| = S . * =|
| = +.o|
| . +oB|
| .. =o|
| .o+o.|
+----[SHA256]-----+
Generating public/private ed25519 key pair.
Your identification has been saved in /etc/ssh/ssh_host_ed25519_key.
Your public key has been saved in /etc/ssh/ssh_host_ed25519_key.pub.
The key fingerprint is:
SHA256:deEqVfcdCOPAGvBX9OU4eTJOVwmK4u/dNgJafj54yLE root@ip-172-31-26-206
The key's randomart image is:
+--[ED25519 256]--+
| .. ...=+.+oo|
| .. .=o++=o+|
| ooooooO +o|
| ..oo oo * |
| .S . . |
| .= |
| =. |
| ..E.=.o |
| .+o+.. |
+----[SHA256]-----+
Generating locales (this might take a while)...
en_US.UTF-8... done
Generation complete.
Cloud-init v. 0.7.8 running 'modules:config' at Wed, 15 Mar 2017 08:53:57 +0000. Up 10.85 seconds.
- set -ex
- RETRY_COMMAND_SLEEP_INTERVAL=5
- RETRY_COMMAND_RETRY_LIMIT=10
- retry_command apt-get update
- local count=0
- local last_return_code=1
- echo RETRY_COMMAND_SLEEP_INTERVAL=5
RETRY_COMMAND_SLEEP_INTERVAL=5 - echo RETRY_COMMAND_RETRY_LIMIT=10
RETRY_COMMAND_RETRY_LIMIT=10 - true
- count=1
- apt-get update
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:2 http://security.ubuntu.com/ubuntu xenial-security/main Sources [63.6 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security/restricted Sources [2288 B]
Get:4 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [24.4 kB]
Get:5 http://security.ubuntu.com/ubuntu xenial-security/multiverse Sources [1144 B]
Get:6 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [230 kB]
Get:7 http://security.ubuntu.com/ubuntu xenial-security/main Translation-en [96.4 kB]
Get:8 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [93.6 kB]
Get:9 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [50.7 kB]
Get:10 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [2744 B]
Hit:11 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial InRelease
Get:12 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:13 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Get:14 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main Sources [868 kB]
Get:15 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/restricted Sources [4808 B]
Get:16 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/universe Sources [7728 kB]
Get:17 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/multiverse Sources [179 kB]
Get:18 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main Sources [236 kB]
Get:19 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/restricted Sources [2688 B]
Get:20 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe Sources [142 kB]
Get:21 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/multiverse Sources [4552 B]
Get:22 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [492 kB]
Get:23 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [196 kB]
Get:24 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [435 kB]
Get:25 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [166 kB]
Get:26 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [8080 B]
Get:27 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main Sources [3200 B]
Get:28 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/universe Sources [1868 B]
Get:29 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [4672 B]
Get:30 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/main Translation-en [3200 B]
Get:31 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [2512 B]
Get:32 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [1216 B]
Fetched 11.4 MB in 49s (227 kB/s)
Reading package lists... - return 0
- node_number=0
- node_name=ControlNode
- wait_condition_handle='https://cloudformation-waitcondition-us-east-1.s3.amazonaws.com/arn%3Aaws%3Acloudformation%3Aus-east-1%3A579800346274%3Astack/flocker-plugin/ba696500-095c-11e7-8a97-50d5ca63261e/ControlNodeFlockerSwarmReadySignal?AWSAccessKeyId=AKIAIIT3CWAIMJYUTISA&Expires=1489654386&Signature=Jt8Z8woTyVgD0r%2FqPi4klKliWGg%3D'
- set -ex
- cat
- retry_command apt-get --yes install s3cmd
- local count=0
- local last_return_code=1
- echo RETRY_COMMAND_SLEEP_INTERVAL=5
RETRY_COMMAND_SLEEP_INTERVAL=5 - echo RETRY_COMMAND_RETRY_LIMIT=10
RETRY_COMMAND_RETRY_LIMIT=10 - true
- count=1
- apt-get --yes install s3cmd
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
libpython-stdlib libpython2.7-minimal libpython2.7-stdlib python
python-dateutil python-magic python-minimal python-six python2.7
python2.7-minimal
Suggested packages:
python-doc python-tk python2.7-doc binutils binfmt-support
The following NEW packages will be installed:
libpython-stdlib libpython2.7-minimal libpython2.7-stdlib python
python-dateutil python-magic python-minimal python-six python2.7
python2.7-minimal s3cmd
0 upgraded, 11 newly installed, 0 to remove and 25 not upgraded.
Need to get 4067 kB of archives.
After this operation, 17.4 MB of additional disk space will be used.
Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython2.7-minimal amd64 2.7.12-1ubuntu016.04.1 [339 kB]16.04.1 [1295 kB]
Get:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7-minimal amd64 2.7.12-1ubuntu0
Get:3 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 python-minimal amd64 2.7.11-1 [28.2 kB]
Get:4 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython2.7-stdlib amd64 2.7.12-1ubuntu016.04.1 [1884 kB]16.04.1 [224 kB]
Get:5 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7 amd64 2.7.12-1ubuntu0
Get:6 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libpython-stdlib amd64 2.7.11-1 [7656 B]
Get:7 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 python amd64 2.7.11-1 [137 kB]
Get:8 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 python-six all 1.10.0-3 [10.9 kB]
Get:9 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 python-dateutil all 2.4.2-1 [42.5 kB]
Get:10 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/universe amd64 python-magic all 1:5.25-2ubuntu1 [5374 B]
Get:11 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/universe amd64 s3cmd all 1.6.1-1 [92.6 kB]
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin:
Fetched 4067 kB in 0s (29.2 MB/s)
Selecting previously unselected package libpython2.7-minimal:amd64.
(Reading database ... 53844 files and directories currently installed.)
Preparing to unpack .../libpython2.7-minimal_2.7.12-1ubuntu016.04.1_amd64.deb ...16.04.1) ...
Unpacking libpython2.7-minimal:amd64 (2.7.12-1ubuntu0
Selecting previously unselected package python2.7-minimal.
Preparing to unpack .../python2.7-minimal_2.7.12-1ubuntu016.04.1_amd64.deb ...16.04.1) ...
Unpacking python2.7-minimal (2.7.12-1ubuntu0
Selecting previously unselected package python-minimal.
Preparing to unpack .../python-minimal_2.7.11-1_amd64.deb ...
Unpacking python-minimal (2.7.11-1) ...
Selecting previously unselected package libpython2.7-stdlib:amd64.
Preparing to unpack .../libpython2.7-stdlib_2.7.12-1ubuntu016.04.1_amd64.deb ...16.04.1) ...
Unpacking libpython2.7-stdlib:amd64 (2.7.12-1ubuntu0
Selecting previously unselected package python2.7.
Preparing to unpack .../python2.7_2.7.12-1ubuntu016.04.1_amd64.deb ...16.04.1) ...
Unpacking python2.7 (2.7.12-1ubuntu0
Selecting previously unselected package libpython-stdlib:amd64.
Preparing to unpack .../libpython-stdlib_2.7.11-1_amd64.deb ...
Unpacking libpython-stdlib:amd64 (2.7.11-1) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for mime-support (3.59ubuntu1) ...
Setting up libpython2.7-minimal:amd64 (2.7.12-1ubuntu016.04.1) ...16.04.1) ...
Setting up python2.7-minimal (2.7.12-1ubuntu0
Linking and byte-compiling packages for runtime python2.7...
Setting up python-minimal (2.7.11-1) ...
Selecting previously unselected package python.
(Reading database ... 54590 files and directories currently installed.)
Preparing to unpack .../python_2.7.11-1_amd64.deb ...
Unpacking python (2.7.11-1) ...
Selecting previously unselected package python-six.
Preparing to unpack .../python-six_1.10.0-3_all.deb ...
Unpacking python-six (1.10.0-3) ...
Selecting previously unselected package python-dateutil.
Preparing to unpack .../python-dateutil_2.4.2-1_all.deb ...
Unpacking python-dateutil (2.4.2-1) ...
Selecting previously unselected package python-magic.
Preparing to unpack .../python-magic_1%3a5.25-2ubuntu1_all.deb ...
Unpacking python-magic (1:5.25-2ubuntu1) ...
Selecting previously unselected package s3cmd.
Preparing to unpack .../archives/s3cmd_1.6.1-1_all.deb ...
Unpacking s3cmd (1.6.1-1) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up libpython2.7-stdlib:amd64 (2.7.12-1ubuntu016.04.1) ...16.04.1) ...
Setting up python2.7 (2.7.12-1ubuntu0
Setting up libpython-stdlib:amd64 (2.7.11-1) ...
Setting up python (2.7.11-1) ...
Setting up python-six (1.10.0-3) ...
Setting up python-dateutil (2.4.2-1) ...
Setting up python-magic (1:5.25-2ubuntu1) ...
Setting up s3cmd (1.6.1-1) ... - return 0
- flocker_node_type=control
- set -ex
- : us-east-1
- : us-east-1a
- : AKIAJS7E747E4HREHFGA
- : 4dp0deuFi5Rq1Xi4/P1dsuxIw4uol3yS1glWnZhQ
- : 3
- : flocker-plugin-clusterconfig-d2mnt666zdvc
++ /usr/bin/ec2metadata --public-ipv4 - control_service_ip=52.91.177.171
- rm -rf /tmp/flocker-config
- mkdir /tmp/flocker-config
- cat
- pushd /tmp/flocker-config
/tmp/flocker-config / - /opt/flocker/bin/flocker-ca initialize flocker-cluster
/var/lib/cloud/instance/scripts/part-001: line 106: /opt/flocker/bin/flocker-ca: No such file or directory
Cloud-init v. 0.7.8 running 'modules:final' at Wed, 15 Mar 2017 08:53:59 +0000. Up 12.45 seconds.
2017-03-15 08:54:57,493 - util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [1]
2017-03-15 08:54:57,496 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
2017-03-15 08:54:57,497 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed
Cloud-init v. 0.7.8 finished at Wed, 15 Mar 2017 08:54:57 +0000. Datasource DataSourceEc2. Up 70.01 seconds
Any update on this issue ?
@KuruvaSomaSekhar clusterhq shutdown in december 16. So those aws resources are probably destroyed.
You can try to find more help here: https://github.com/ScatterHQ/flocker where the project moved where some of the maintainers are trying to help where they can and people interested in using it.