Creating domain with attached cloudinit iso hangs permanently
NHAS opened this issue · comments
System Information
Archlinux latest using libvirt 9.4.0
Linux distribution
Archlinux
Version information
Terraform v1.5.0
on linux_amd64
+ provider registry.terraform.io/dmacvicar/libvirt v0.7.1
+ provider registry.terraform.io/hashicorp/template v2.2.0
Checklist
- Is it a bug or something that does not work as expected? Please make sure you fill the version information below:
Description of Issue/Question
When trying to do a very simple setup the creation of a single domain hangs forever. With builds taking over 20 minutes.
This only happens when a cloud-init image is attached to the configuration, otherwise it takes less than 2.
Attaching a cloud-init image works fine on terraform 1.4.6
, but hangs on 1.5.0
.
Setup
On archlinux with the most recent version of everything (just after a pacman -Syu
and reboot)
terraform {
required_version = ">= 0.13"
required_providers {
libvirt = {
source = "dmacvicar/libvirt"
version = "0.7.1"
}
}
}
# instance the provider
provider "libvirt" {
uri = "qemu:///system"
}
# We fetch the latest ubuntu release image from their mirrors
resource "libvirt_volume" "rocky9" {
name = "rocky9"
source = "https://download.rockylinux.org/pub/rocky/9/images/x86_64/Rocky-9-GenericCloud.latest.x86_64.qcow2"
format = "qcow2"
}
data "template_file" "user_data" {
template = file("${path.module}/cloud_init.cfg")
}
data "template_file" "network_config" {
template = file("${path.module}/network_config.cfg")
}
# for more info about paramater check this out
# https://github.com/dmacvicar/terraform-provider-libvirt/blob/master/website/docs/r/cloudinit.html.markdown
# Use CloudInit to add our ssh-key to the instance
# you can add also meta_data field
resource "libvirt_cloudinit_disk" "commoninit" {
name = "commoninit.iso"
user_data = data.template_file.user_data.rendered
network_config = data.template_file.network_config.rendered
}
# Create the machine
resource "libvirt_domain" "domain-rocky9" {
name = "rocky9-terraform"
memory = "2048"
vcpu = 2
cloudinit = libvirt_cloudinit_disk.commoninit.id
network_interface {
network_name = "default"
wait_for_lease = true
}
cpu {
mode = "host-passthrough"
}
# IMPORTANT: this is a known bug on cloud images, since they expect a console
# we need to pass it
# https://bugs.launchpad.net/cloud-images/+bug/1573095
console {
type = "pty"
target_port = "0"
target_type = "serial"
}
console {
type = "pty"
target_type = "virtio"
target_port = "1"
}
disk {
volume_id = libvirt_volume.rocky9.id
}
graphics {
type = "vnc"
autoport = true
}
}
cloud_init.cfg:
#cloud-config
# vim: syntax=yaml
ssh_pwauth: False
users:
- name: ansible
homedir: /home/ansible
shell: /bin/bash
sudo: 'ALL=(ALL) NOPASSWD:ALL'
ssh_authorized_keys:
- 'ssh-ed25519 <my_public_key>'
network_config.cfg
version: 2
ethernets:
ens3:
dhcp4: true
Steps to Reproduce Issue
Using the platform and the provided TF file,
terraform apply
Debug log:
2023-06-14T09:46:58.535+1200 [INFO] backend/local: apply calling Apply
2023-06-14T09:46:58.535+1200 [DEBUG] Building and walking apply graph for NormalMode plan
2023-06-14T09:46:58.535+1200 [DEBUG] Resource state not found for node "libvirt_cloudinit_disk.commoninit", instance libvirt_cloudinit_disk.commoninit
2023-06-14T09:46:58.535+1200 [DEBUG] Resource state not found for node "libvirt_volume.rocky9", instance libvirt_volume.rocky9
2023-06-14T09:46:58.535+1200 [DEBUG] Resource state not found for node "libvirt_domain.domain-rocky9", instance libvirt_domain.domain-rocky9
2023-06-14T09:46:58.535+1200 [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/template"], implied first by data.template_file.user_data (expand)
2023-06-14T09:46:58.535+1200 [DEBUG] ProviderTransformer: "data.template_file.user_data (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/template"]
2023-06-14T09:46:58.535+1200 [DEBUG] ProviderTransformer: "data.template_file.network_config (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/template"]
2023-06-14T09:46:58.535+1200 [DEBUG] ProviderTransformer: "libvirt_cloudinit_disk.commoninit" (*terraform.NodeApplyableResourceInstance) needs provider["registry.terraform.io/dmacvicar/libvirt"]
2023-06-14T09:46:58.535+1200 [DEBUG] ProviderTransformer: "libvirt_volume.rocky9" (*terraform.NodeApplyableResourceInstance) needs provider["registry.terraform.io/dmacvicar/libvirt"]
2023-06-14T09:46:58.535+1200 [DEBUG] ProviderTransformer: "libvirt_domain.domain-rocky9" (*terraform.NodeApplyableResourceInstance) needs provider["registry.terraform.io/dmacvicar/libvirt"]
2023-06-14T09:46:58.535+1200 [DEBUG] ProviderTransformer: "libvirt_cloudinit_disk.commoninit (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/dmacvicar/libvirt"]
2023-06-14T09:46:58.535+1200 [DEBUG] ProviderTransformer: "libvirt_domain.domain-rocky9 (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/dmacvicar/libvirt"]
2023-06-14T09:46:58.535+1200 [DEBUG] ProviderTransformer: "libvirt_volume.rocky9 (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/dmacvicar/libvirt"]
2023-06-14T09:46:58.536+1200 [DEBUG] ReferenceTransformer: "libvirt_volume.rocky9" references: []
2023-06-14T09:46:58.536+1200 [DEBUG] ReferenceTransformer: "libvirt_domain.domain-rocky9" references: [libvirt_cloudinit_disk.commoninit libvirt_cloudinit_disk.commoninit libvirt_cloudinit_disk.commoninit (expand) libvirt_volume.rocky9 libvirt_volume.rocky9 libvirt_volume.rocky9 (expand)]
2023-06-14T09:46:58.536+1200 [DEBUG] ReferenceTransformer: "data.template_file.user_data (expand)" references: []
2023-06-14T09:46:58.536+1200 [DEBUG] ReferenceTransformer: "data.template_file.network_config (expand)" references: []
2023-06-14T09:46:58.536+1200 [DEBUG] ReferenceTransformer: "libvirt_cloudinit_disk.commoninit" references: [data.template_file.network_config (expand) data.template_file.user_data (expand)]
2023-06-14T09:46:58.536+1200 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/dmacvicar/libvirt\"]" references: []
2023-06-14T09:46:58.536+1200 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/template\"]" references: []
2023-06-14T09:46:58.536+1200 [DEBUG] ReferenceTransformer: "libvirt_cloudinit_disk.commoninit (expand)" references: []
2023-06-14T09:46:58.536+1200 [DEBUG] ReferenceTransformer: "libvirt_domain.domain-rocky9 (expand)" references: []
2023-06-14T09:46:58.536+1200 [DEBUG] ReferenceTransformer: "libvirt_volume.rocky9 (expand)" references: []
2023-06-14T09:46:58.537+1200 [DEBUG] Starting graph walk: walkApply
2023-06-14T09:46:58.537+1200 [DEBUG] created provider logger: level=debug
2023-06-14T09:46:58.537+1200 [INFO] provider: configuring client automatic mTLS
2023-06-14T09:46:58.553+1200 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/template/2.2.0/linux_amd64/terraform-provider-template_v2.2.0_x4 args=[.terraform/providers/registry.terraform.io/hashicorp/template/2.2.0/linux_amd64/terraform-provider-template_v2.2.0_x4]
2023-06-14T09:46:58.553+1200 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/template/2.2.0/linux_amd64/terraform-provider-template_v2.2.0_x4 pid=5474
2023-06-14T09:46:58.553+1200 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/hashicorp/template/2.2.0/linux_amd64/terraform-provider-template_v2.2.0_x4
2023-06-14T09:46:58.557+1200 [INFO] provider.terraform-provider-template_v2.2.0_x4: configuring server automatic mTLS: timestamp=2023-06-14T09:46:58.557+1200
2023-06-14T09:46:58.569+1200 [DEBUG] provider.terraform-provider-template_v2.2.0_x4: plugin address: address=/tmp/plugin209818447 network=unix timestamp=2023-06-14T09:46:58.569+1200
2023-06-14T09:46:58.569+1200 [DEBUG] provider: using plugin: version=5
2023-06-14T09:46:58.592+1200 [DEBUG] created provider logger: level=debug
2023-06-14T09:46:58.592+1200 [INFO] provider: configuring client automatic mTLS
2023-06-14T09:46:58.593+1200 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
2023-06-14T09:46:58.593+1200 [DEBUG] No provider meta schema returned
2023-06-14T09:46:58.605+1200 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/dmacvicar/libvirt/0.7.1/linux_amd64/terraform-provider-libvirt_v0.7.1 args=[.terraform/providers/registry.terraform.io/dmacvicar/libvirt/0.7.1/linux_amd64/terraform-provider-libvirt_v0.7.1]
2023-06-14T09:46:58.605+1200 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/dmacvicar/libvirt/0.7.1/linux_amd64/terraform-provider-libvirt_v0.7.1 pid=5498
2023-06-14T09:46:58.605+1200 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/dmacvicar/libvirt/0.7.1/linux_amd64/terraform-provider-libvirt_v0.7.1
2023-06-14T09:46:58.608+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: configuring server automatic mTLS: timestamp=2023-06-14T09:46:58.608+1200
2023-06-14T09:46:58.621+1200 [DEBUG] provider: using plugin: version=5
2023-06-14T09:46:58.621+1200 [DEBUG] provider.terraform-provider-libvirt_v0.7.1: plugin address: network=unix address=/tmp/plugin3627148650 timestamp=2023-06-14T09:46:58.621+1200
2023-06-14T09:46:58.638+1200 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/template/2.2.0/linux_amd64/terraform-provider-template_v2.2.0_x4 pid=5474
2023-06-14T09:46:58.638+1200 [DEBUG] provider: plugin exited
2023-06-14T09:46:58.640+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 [DEBUG] Configuring provider for 'qemu:///system': &{map[uri:0xc00037c3c0] <nil> <nil> 0xc0005d2680 map[] <nil> {{<nil>} <nil>} 0xc000504180 0xc0003263a8 0xc0000be8f0 false {1 {0 0}} false false}: timestamp=2023-06-14T09:46:58.640+1200
2023-06-14T09:46:58.640+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 [INFO] Set up libvirt transport: &{{0xc0005d2a80}}: timestamp=2023-06-14T09:46:58.640+1200
2023-06-14T09:46:58.643+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 [INFO] libvirt client libvirt version: 9004000: timestamp=2023-06-14T09:46:58.643+1200
2023-06-14T09:46:58.646+1200 [WARN] Provider "registry.terraform.io/dmacvicar/libvirt" produced an invalid plan for libvirt_cloudinit_disk.commoninit, but we are tolerating it because it is using the legacy plugin SDK.
The following problems may be the cause of any confusing errors from downstream operations:
- .pool: planned value cty.StringVal("default") for a non-computed attribute
2023-06-14T09:46:58.646+1200 [WARN] Provider "registry.terraform.io/dmacvicar/libvirt" produced an invalid plan for libvirt_volume.rocky9, but we are tolerating it because it is using the legacy plugin SDK.
The following problems may be the cause of any confusing errors from downstream operations:
- .pool: planned value cty.StringVal("default") for a non-computed attribute
libvirt_cloudinit_disk.commoninit: Creating...
libvirt_volume.rocky9: Creating...
2023-06-14T09:46:58.646+1200 [INFO] Starting apply for libvirt_cloudinit_disk.commoninit
2023-06-14T09:46:58.646+1200 [INFO] Starting apply for libvirt_volume.rocky9
2023-06-14T09:46:58.646+1200 [DEBUG] libvirt_volume.rocky9: applying the planned Create change
2023-06-14T09:46:58.646+1200 [DEBUG] libvirt_cloudinit_disk.commoninit: applying the planned Create change
2023-06-14T09:46:58.646+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 [DEBUG] creating cloudinit: timestamp=2023-06-14T09:46:58.646+1200
2023-06-14T09:46:58.646+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 [INFO] cloudInit: {Name:commoninit.iso PoolName:default MetaData: UserData:#cloud-config
# vim: syntax=yaml
ssh_pwauth: False
users:
- name: ansible
homedir: /home/ansible
shell: /bin/bash
sudo: 'ALL=(ALL) NOPASSWD:ALL'
ssh_authorized_keys:
- 'ssh-ed25519 <redacted>'
NetworkConfig:version: 2
ethernets:
ens3:
dhcp4: true}: timestamp=2023-06-14T09:46:58.646+1200
2023-06-14T09:46:58.646+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 Creating new ISO: timestamp=2023-06-14T09:46:58.646+1200
2023-06-14T09:46:58.646+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 Creating ISO contents: timestamp=2023-06-14T09:46:58.646+1200
2023-06-14T09:46:58.646+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 ISO contents created: timestamp=2023-06-14T09:46:58.646+1200
2023-06-14T09:46:58.646+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 About to execute cmd: /usr/bin/mkisofs -output /tmp/cloudinit1030668553/commoninit.iso -volid cidata -joliet -rock /tmp/cloudinit1030668553/user-data /tmp/cloudinit1030668553/meta-data /tmp/cloudinit1030668553/network-config: timestamp=2023-06-14T09:46:58.646+1200
2023-06-14T09:46:58.646+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 [DEBUG] Locking "default": timestamp=2023-06-14T09:46:58.646+1200
2023-06-14T09:46:58.646+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 [DEBUG] Locked "default": timestamp=2023-06-14T09:46:58.646+1200
2023-06-14T09:46:58.649+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 ISO created at /tmp/cloudinit1030668553/commoninit.iso: timestamp=2023-06-14T09:46:58.649+1200
2023-06-14T09:46:58.649+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:58 [DEBUG] Locking "default": timestamp=2023-06-14T09:46:58.649+1200
2023-06-14T09:46:59.766+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:59 Image https://download.rockylinux.org/pub/rocky/9/images/x86_64/Rocky-9-GenericCloud.latest.x86_64.qcow2 image is: 989265920 bytes: timestamp=2023-06-14T09:46:59.766+1200
2023-06-14T09:46:59.766+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:59 [DEBUG] Generated XML for libvirt volume:
<volume>
<name>rocky9</name>
<capacity unit="B">989265920</capacity>
<target>
<format type="qcow2"></format>
<permissions>
<mode>644</mode>
</permissions>
</target>
</volume>: timestamp=2023-06-14T09:46:59.766+1200
2023-06-14T09:46:59.767+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:59 [INFO] Volume about to be created was found and left as-is: rocky9: timestamp=2023-06-14T09:46:59.767+1200
2023-06-14T09:46:59.767+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:46:59 [INFO] Volume ID: /var/lib/libvirt/images/rocky9: timestamp=2023-06-14T09:46:59.767+1200
2023-06-14T09:47:00.034+1200 [INFO] provider.terraform-provider-libvirt_v0.7.1: 2023/06/14 09:47:00 [DEBUG]: url resp status code 200 OK (retry #0): timestamp=2023-06-14T09:47:00.033+1200
libvirt_cloudinit_disk.commoninit: Still creating... [10s elapsed]
libvirt_volume.rocky9: Still creating... [10s elapsed]
libvirt_volume.rocky9: Still creating... [1m0s elapsed]
libvirt_cloudinit_disk.commoninit: Still creating... [1m0s elapsed]
libvirt_volume.rocky9: Still creating... [1m10s elapsed]
libvirt_cloudinit_disk.commoninit: Still creating... [1m10s elapsed]
AS this never completes I've truncated it at the 1 min mark
Additional information:
Do you have SELinux or Apparmor/Firewall enabled? Some special configuration?
Nope, just stock standard arch
Have you tried to reproduce the issue without them enabled?
N/A
My bad it was hanging because a volume was created and never destroyed, but for some reason didnt error out.