Giters

silentbugs / hon-client-scraper

Messy python script to quickly fetch hon client versions

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature Request] Add a flag to download delta patches

mrhappyasthma opened this issue · comments

commented

I have working prototype code that I can pull into a PR.

Essentially instead of downloading the entire version, you can download just the files that are 'new' relative to some previous version.

For example:

Say you already have the full download for 4.9.5. You could download the 'delta patch' files for 4.9.6. In this case, it would only download the new files. Or if you wanted to patch to 4.9.10, you could download just the range of files that are newer than the 4.9.5 version.

The logic is actually fairly trivial since the manifest file specifies the version of each file. You can just skip any file for which the version is <= 4.9.5, in that example.

commented

That's neat, please make a PR so I can check it out.

My thinking is that there could be a -p | --previous flag that specifies the previous version to download the delta from, along with the -sv flag which specifies the current version.

So: python main.py -o wac -a i686 -sv 3.8.3 -p 3.8.2 in order to download the delta between 3.8.3 and 3.8.2?

Ideally we would checksum the previous files to ensure they are correctly downloaded (which could take more time than to actually download them) but as of now I haven't figured out how the provided checksum works (i.e. what algorithm they use).

commented

That's exactly how I have it working. I'll clean it up and get a PR out one of these days. (Been slowly downloading each delta and anytime a new manifest mentions that I skipped a version, I go back and restart the loop from that point onwards. I'm up to the 2.* right now for i686).

I have not looked specifically at the manifest checksums, but I did put together a project to checksum a given install (relative to a pre-calculated expected set) https://github.com/HoN-Revival/HoN-Install-Checksums

One thing I noticed was that I couldn't rely on the checksum of the .zip (i.e. .s2z). Depending on how it was zipped and when it was last touched would cause the metadata to differ. So instead I actually checked the pre-computed checksums in the metadata of the zip itself. And that worked like a charm.

I need to see if the checksum from the manifest is that of the .s2z (zip) or of the underlying file.