Giters

actions / upload-artifact

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

upload-artifact does not retain artifact permissions

kcgen opened this issue · comments

commented

The baseline behavior of the zip utilty on Linux and macOS is to retain permissions.

However, when the upload-artifact action zips a directory, it loses permissions, which subsequently breaks the artifacts for users and downstream tools.

Expected behavior: the permissions applied to assets in prior steps should be retained by the upload-artifact zipper, and should be present in the resulting asset zip file.

commented

Adding this issue to the download-artifact action, as that's where the zipping is currently performed in the v1 API. actions/download-artifact#14.
Will leave this open in both repos open until it's solved (however github wants to manage this in their backend for v2).

commented

Can confirm that with unzip -Z on the resulting .zip, there are no executable permissions on files that should have executable perms.

commented

Is anything being done about this? It's been over 6 moths and this is the recommended way by GitHub to handle artifact upload

commented

This is still broken as of today, very annoying.

commented

I am not proud of it, but actions/cache@v2 keeps the permissions. If the goal is to pass some artifacts between jobs the following is working for me so far.

      - name: Artifact (with permissions)
        uses: actions/cache@v2
        with:
          path: /path/to/store
          key: artifacts-${{ github.run_id }}-${{ github.run_number }}
commented

A workaround is documented in the readme

https://github.com/actions/upload-artifact#maintaining-file-permissions-and-case-sensitive-files

  - name: 'Tar files'
    run: tar -cvf my_files.tar /path/to/my/directory

  - name: 'Upload Artifact'
    uses: actions/upload-artifact@v2
    with:
      name: my-artifact
      path: my_files.tar
commented

Wow. This is still an issue?

commented

@konradpabjan @joshmgross any comments from the authors of this action?

When are you going to fix this BUG?
This makes GitHub workflows in whole a useless tool compared to your competitors offering CI solutions.

commented

FYI this has probably been mentioned in passing but this completely breaks uploading of binaries as artifacts on macOS, such as .apps etc.

commented

@lsegal agreed - the workaround in my case was to downlad the artifact - uses: actions/download-artifact@v2, add a step to repair any broken permissions with a chmod e.g.

  - run:  chmod +x myapp-macos/myapp.app/Contents/MacOS/myapp

zip up the .app (directory), then uses: softprops/action-gh-release@v1 to upload the zip as a GitHub release.

commented

Uploading as a release actually sounds like a nice idea, as some of us just want to make artifacts directly downloadable by users, so "just chmod +x on download" is not a good solution, because it will be downloaded by the user and not by another workflow, so they have to remember to do it, or I have to make it a nested zip, which is ugly.

Though it could get spammy if a release is posted for each commit rather than just linking it to the workflows, for a project that doesn't really have versioning.

commented

Applying chmod manually is no solution. It's easy a job of hundreds to thousands of files.

Also from system architecture is a gigantic faul if the receiving system needs to know how the transmitting system has configured access rights. This breaks encapsulation!

commented

Applying chmod manually is no solution. It's easy a job of hundreds to thousands of files.

Also from system architecture is a gigantic faul if the receiving system needs to know how the transmitting system has configured access rights. This breaks encapsulation!

As mentioned before, the actual workaround for preserving permissions (not restoring them afterwards) is to upload tarballs.

commented

@brcrista Any word on this? Would anyone from the team be willing to review and possibly merge a PR fixing this behavior?

commented

@brcrista Any word on this? Would anyone from the team be willing to review and possibly merge a PR fixing this behavior?

#3 (comment)

Based on this comment and the fact that the cache action preserves permissions, I'd assume the permissions are actually preserved on the github server; it's only the zipping during the download that discards it.

If that is correct, the code that needs fixing is in fact the download UI, i.e. nothing that's open source.

commented

I don't think we'll be getting to this anytime soon as we have bigger initiatives in progress. But we have looked at this and here is our understanding of what a fix would take:

  • The permissions issue isn't with the zipping (or if it is, that is easy to fix)
  • https://github.com/actions/runner/ would need to read the file permissions and send them up to the service along with the artifacts. Currently, it only uploads the raw file content over HTTP, which does not include the file metadata.
  • The service would store these permissions along with the artifact contents
  • For artifact download, we would reverse this process: read permissions, send them down, and have the runner re-apply them
  • There are some open questions to answer around whether to preserve who the file owners are, since these are linked with permissions. We would also want to consider how support a cross-platform implementation -- I am thinking mainly of POSIX file permissions here. The way Git handles file permissions would be a good design reference.

So yes, as @MCOfficer pointed out, most of this work is on the service side.

commented

The permissions issue isn't with the zipping (or if it is, that is easy to fix)

It is. .zip files simply don't store any of the meta data that's used on platforms outside of Windows and DOS. That's why on Linux and other Unix-like platforms people never use .zip files, and they use compressed tarballs instead.

commented

It is. .zip files simply don't store any of the meta data that's used on platforms outside of Windows and DOS. That's why on Linux and other Unix-like platforms people never use .zip files, and they use compressed tarballs instead.

This is actually the opposite. zip does maintain executable bit and other permissions on Unix platforms. macOS uses zip for its standard compression just fine. It's actually Windows platforms that don't respect exec bits and such, but fortunately Windows doesn't need it: https://unix.stackexchange.com/questions/313656/preserving-permissions-while-zipping

There's nothing fundamentally wrong with using zip files. It is actually default behavior to store/restore permissions, which makes this bug all the more odd that it's somehow not working here. It's almost as if this action is explicitly doing something to break zip's behavior.

commented

You're right, it does preserve most permissions! Probably shows that I never use .zip files. 😁

That being said, I'd appreciate it if GitHub Actions would just let me upload tarballs. Right now we end up with these zipped tarballs which no doubt result in some head scratching when users try to download builds from the CI.

commented

So is this all on purpose? Or will upload-artifact fix this issue??

commented

Bump. I need this

commented

I admire GitHub’s desire and ambition to reinvent tar.

Reinventing wheels because “we can do it better” is an age-old tradition in software engineering and it would certainly depress me to find out that an $8B company would feel it was beneath them.

I look forward to reporting fresh bugs to the inevitable @github/gh-tar and feeling that wholesome, warm, heady glow of nostalgia when I refer back to the same bugs already fixed 20 years ago by GNU.

refs #38 (comment)

commented

An alternative workaround, if you need to keep the artifact format as a pure ZIP is used here. The approach is to use the getfacl / setfacl CLI tools to backup the permissions into a file that is included in the artifact upload, then restore them from the file on download.

The tools are present in the GHA ubuntu-latest image, so the require setup is minimal.

commented

waiting for fix too.

commented

Same for me, I was building an APK using Unity (floatingIslands is the project/APK name). The building was fine, uploaded artifact fine, but didnt have permission to download it to create a release :/

This is the log of the workflow:

Run actions/download-artifact@v3
with:
name: FloatingIslands-v1.0.0.33
path: build/Android
Starting download for FloatingIslands-v1.0.0.33
Directory structure has been setup for the artifact
Total number of files that will be downloaded: 1
node:events:368
throw er; // Unhandled 'error' event
^

Error: EACCES: permission denied, open '/home/runner/work/FloatingIslands/FloatingIslands/build/Android/Android.apk'
Emitted 'error' event on WriteStream instance at:
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -13,
code: 'EACCES',
syscall: 'open',
path: '/home/runner/work/FloatingIslands/FloatingIslands/build/Android/Android.apk'
}

commented

Is this going to be fixed? This bug its still present and it's been almost a year for something super essential as is keep the artifacts permissions

commented

We are building macOS app and archiving it. The archive is zipped by the upload artifact actions and the executable will lost +x flag. This makes the app non functional when downloaded from GitHub Actions artifact.

commented

any progress on this ?

commented

We who paid for github actions should get a refund. It's obviously not working.

commented

Wow, this is a pretty massive problem imo, and really frustrating to see that nothing has happened since 2019 😮‍💨

Wonder what the accumulated time spent to fix this locally for users of this action has been since then??

Please fix this 🙏

commented

This issue still persists as for 2024. Just lost some time trying to figure out why a downloaded artifact's permissions weren't as I expected them to be.

commented

I don't know whether this is a good way or not, whe i trying upload and download artifact which is a golang binary, i set permissions for the binary file on the job that needs execute it

chmod 777 <binary filename>
commented

Looks like there's a PR that would fix half of this issue: actions/toolkit#1609. (Edit: Nope, see below). That would store file permissions in the .zip files created by upload-artifact; the next step after that would be to fix download-artifact to recreate those permissions, but that can't happen until the permissions actually exist in the .zip file.

But that PR has been sitting there for four months with no review from the GitHub Actions team yet. Hopefully someone from the GHA team will notice that PR and give it some attention soon.

UPDATE: Looks like that PR wouldn't work (the zip.append method wants a mode property, not stats) so I created actions/toolkit#1723 which is basically actions/toolkit#1609 but passing mode into zip.append.