Packer is not handling line endings correctly when running on Windows

[Jacob Hodges]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Overview of the Issue

Packer is not handling line endings correctly when running on Windows. I have a provisioner that is running .sh files in an Ubuntu VM which errors with null.build_linux: /imagegeneration/helpers/os.sh: line 6: $'\r': command not found

Reproduction Steps

This seems to only be an issue on Windows.

1. Create a super simple bash script
2. Ensure that the script has windows line endings.
3. Use a provision like so:

provisioner "shell" {
    environment_vars = ["IMAGE_VERSION=${var.image_version}", "IMAGE_OS=${var.image_os}", "HELPER_SCRIPTS=${var.helper_script_folder}"]
    execute_command  = "sudo sh -c '{{ .Vars }} {{ .Path }}'"
    scripts          = ["${path.root}/scripts/simple.sh"]
  }

Packer version

1.9.4

Simplified Packer Template

n/a

Operating system and Environment details

Windows 11 22h2 Build 22621.2428

Log Fragments and crash.log files

2023/10/27 18:07:00 ui: ==> null.build_linux: Provisioning with shell script: images/linux/scripts/installers/configure-environment.sh
2023/10/27 18:07:00 packer.exe plugin: Opening images/linux/scripts/installers/configure-environment.sh for reading
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] Opening new ssh session
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] Starting remote scp process:  scp -vt /tmp
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] Started SCP session, beginning transfers...
2023/10/27 18:07:00 packer.exe plugin: [INFO] 2658 bytes written for 'uploadData'
2023/10/27 18:07:00 [INFO] 2658 bytes written for 'uploadData'
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] Copying input data into temporary file so we can read the length
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] scp: Uploading script_3950.sh: perms=C0644 size=2658
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] SCP session complete, closing stdin pipe.
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] Waiting for SSH session to complete.
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] scp stderr (length 71): Sink: C0644 2658 script_3950.sh
2023/10/27 18:07:00 packer.exe plugin: scp: debug1: fd 0 clearing O_NONBLOCK
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] Opening new ssh session
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] starting remote command: chmod 0755 /tmp/script_3950.sh
2023/10/27 18:07:00 packer.exe plugin: [INFO] RPC endpoint: Communicator ended with: 0
2023/10/27 18:07:00 [INFO] RPC client: Communicator ended with: 0
2023/10/27 18:07:00 [INFO] RPC endpoint: Communicator ended with: 0
2023/10/27 18:07:00 packer.exe plugin: [INFO] RPC client: Communicator ended with: 0
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] Opening new ssh session
2023/10/27 18:07:00 packer.exe plugin: [DEBUG] starting remote command: sudo sh -c 'HELPER_SCRIPTS='/imagegeneration/helpers' IMAGE_OS='ubuntu22' IMAGE_VERSION='dev' PACKER_BUILDER_TYPE='null' PACKER_BUILD_NAME='build_linux'  /tmp/script_3950.sh'
2023/10/27 18:07:00 ui error: ==> null.build_linux: /imagegeneration/helpers/os.sh: line 6: $'\r': command not found
2023/10/27 18:07:00 packer.exe plugin: [ERROR] Remote command exited with '127': sudo sh -c 'HELPER_SCRIPTS='/imagegeneration/helpers' IMAGE_OS='ubuntu22' IMAGE_VERSION='dev' PACKER_BUILDER_TYPE='null' PACKER_BUILD_NAME='build_linux'  /tmp/script_3950.sh'
2023/10/27 18:07:00 packer.exe plugin: [INFO] RPC endpoint: Communicator ended with: 127
2023/10/27 18:07:00 [INFO] 65 bytes written for 'stderr'
2023/10/27 18:07:00 [INFO] 0 bytes written for 'stdout'
2023/10/27 18:07:00 [INFO] RPC client: Communicator ended with: 127
2023/10/27 18:07:00 [INFO] RPC endpoint: Communicator ended with: 127
2023/10/27 18:07:00 packer.exe plugin: [INFO] 65 bytes written for 'stderr'
2023/10/27 18:07:00 packer.exe plugin: [INFO] 0 bytes written for 'stdout'
2023/10/27 18:07:00 packer.exe plugin: [INFO] RPC client: Communicator ended with: 127
2023/10/27 18:07:00 [INFO] (telemetry) ending shell

[Lucas Bajolet]

Hey @jachin84,

This does not seem like a Packer bug. It's expected in the UNIX world that scripts have their lines terminated with LF.

In your case, since you're using a script that you provide through the scripts option, it gets copied over to the guest machine, which in this case being a Linux, expects LF-only in its shell scripts.
Packer does not do anything with those besides copying to the guest, so you should convert them to LF (in general I would advise always converting the line-feeds to LF and not CRLF for UNIX scripts).
I suggest using a tool like dos2unix for this purpose.

I'll close this issue for now since this is not a Packer problem on its own, but please feel free to reopen it and continue the conversation if you think I missed something in your description.

Thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.