MathieuLoutre / grunt-aws-s3

Grunt plugin to interact with AWS S3 using the AWS SDK

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Differential delete confusion.

tristanoneil opened this issue · comments

I'm trying to use the differential delete to remove files that exist in my S3 bucket that don't exist locally though the feature appears to either be non-functional or I'm just misunderstanding how it's meant to be used. My configuration looks something like this:

aws_s3: {
  options: {
    accessKeyId: '<%= aws().key %>',
    secretAccessKey: '<%= aws().secret %>',
    bucket: 'some-bucket',
    access: 'public-read',
    differential: true,
  },
  css: {
    files: [
      {
        action: 'delete',
        expand: true,
        cwd: 'www/',
        src: ['styles/stylesheets/**/*.css', 'js/vendor/**/*.css'],
        dest: '/'
      }
    ]
  }
}

This configuration works fine for the upload action but when set it to delete first I get an error that says Differential delete needs a "cwd" I was able to get around this by modifying the grunt-aws-s3 task to check for the cwd on filePair.orig.cwd instead but then I ran into issues/confusion with the deleteObjects method. It appears that the argument being passed in is the destination to to each individual file not a directory that contains files to be deleted so no objects are returned from listObjects. I think I have an idea for how to reasonably refactor this to make the behavior act to how I would expect it to but I wasn't sure if I just was configuring things incorrectly or didn't understand the feature. Any clarification would be greatly appreciated.

Hi there!

I must say I haven't used delete in production in a while, but the tests seem to still work. I must admit delete and differential together is a little confusing because it works at the opposite of upload, as well as being a little limited.

Currently:

  • dest is the folder on the bucket that you want to target. At the moment, a globbing patterns shouldn't use src (which would reference local files) but exclude. Exclude takes 1 globbing pattern, and can be "flipped" so that it becomes "delete all that match this pattern" rather than "don't delete all that match this pattern".
  • If you use differential, you need to give a cwd, which will indicate which folder dest is referencing locally. In that case, differential will only delete the files on AWS which don't exist locally (look at this in terms of cleaning up if you have changed some assets names or something).

What I'm not sure about is what you're trying to achieve now. Do you just want to delete these CSS files on AWS? Or do you want to delete the CSS files on AWS which aren't there locally?
Which way would you like it to work? If there's something this task can't do and you'd like to do, I'm all ears.

listObjects takes a string, which should represent a path, and AWS will match automatically all all the files which starts with this path (there's no real concept of folders).

Ok I think the feature was just confusing me, now that you've explained the details a bit more in depth I've gotten it to work as desired.

Do you just want to delete these CSS files on AWS?

This is exactly what I wanted it to do. The configuration is just quite a bit different from the upload configuration so that's what was getting me. I could see maybe pointing to your description of the feature in this issue from the README this clarified a lot for me so could see how it might save someone in the future.

Anyway I'd say this issue is resolved so I'll close for now. Thanks so much for taking the time to help!

Sorry if the docs weren't clear, this task starts to do a lot more than I initially planned haha. I'll add the explanation I gave to the current one, hoping that it'll clarify the dark corners of the settings :)