How to upload only changed files?
dhawalbhanushali opened this issue · comments
Hi,
I have a shell script for the purpose of deployment. In that script after 'git pull' command I need to upload only the modified and new files to s3. How can I pass the list of files from 'git pull' command as an argument?
grunt-aws-s3 will only upload the new and/or different files to your bucket if you enable the differential option. It will work out what's different based on what's already uploaded on the bucket using MD5 hashes. You don't need to pass the list of the files that have been modified, rather just target the directories you want to upload and let grunt-aws-s3 do the rest.
Have a look at the documentation and examples to see how to enable differential in the options: https://github.com/MathieuLoutre/grunt-aws-s3#usage-examples
This still checks for each file whether or not it has been changed. This does not serve the purpose. Idea is not to hit the s3 server unnecessarily.
The call is very cheap. It's a single call for 1000 objects. So wether you have 10 or 500 objects, it's the same cost and is done with a single request. I guess if you have more than 10 000 files it can be interesting to bypass the check. However, we would still need to have a changeset/manifest of some kind that's generated by another task and feed it into this one which isn't practical at all. And even if you had that you could feed it to your grunt config and just target the right files directly so this task is independent from manifest formats.
Thank you for the info. I will try to find a solution and update. :)
Closing for now. Feel free to re-open if you found something :)