[Feature] Better download handling with step-back (503s, etc.)

[Godsfoot]

This extension is an amazing asset and I use it routinely, but as the popularity of Civitai grows, it seems the stability of their website has suffered, likely due to the increase in demand and an increase in complexity of the overall site itself.

As a side-effect it seems the likelihood of requests to their APIs returning 503 errors or other error codes has also increased. I think this extension should implement some configurable settings to allow users to control how resilient to instability they want it to be before giving up.

This should be a fairly small change to the 'downloader.py' and then exposing them as configurable settings in the UI.

Proposed Changes:

• Make MAX_RETRIES configurable with sane limits so as not to indefinitely spam Civitai end-points (0 - fast fail, 3 - default, 30 - max)

• Make INITIAL_RETRY_DELAY configurable with an increasing delay between retries. This reduces spamming Civitai and makes it more likely to succeed as opposed to immediately retrying and giving up. (min: 1s, max: 60s)

• Make MAX_RETRY_DELAY configurable to cap the retry delay to some realistic limit that users can set to cause it to fail faster, or limp along as desired. (min/default: 60s, max: 10m?)

• Make 'request_get' take in a 4th argument 'retry_delay=INITIAL_RETRY_DELAY' and in the 'not-404, should retry' block have it calculate an increasing retry_delay based on INITIAL_RETRY_DELAY and MAX_RETRY_DELAY and the total number of retries that have happened, so that it steps-back on repeat retries (0s, 1s, 3s, 5s, 15s, 30s, 45s, 60s, ...). There are different ways to come up with those step-backs, you can have the same step-back be used multiple times, etc. it should not matter that much as long as it grows over the period of MAX_RETRIES.

I quickly modified 'downloader.py' locally to test my theory and with >1000 models being scanned and multiple periods of 503 outage it continued.

DL_EXT = ".downloading"
MAX_RETRIES = 30
INITIAL_RETRY_DELAY_S = 0

...

def request_get(
    url:str,
    headers:dict | None=None,
    retries=0,
    retry_delay=INITIAL_RETRY_DELAY_S
) -> tuple[Literal[True], requests.Response] | tuple[Literal[False], str]:

...

        if status_code != 404 and retries < MAX_RETRIES:
            util.printD("Retrying")

            # Step-back delay (could look into threading event sleeps)
            time.sleep(retry_delay)

            # Recursively call with an increased delay time (naive +3s each time, could make smarter) 
            return request_get(url, headers, retries + 1, retry_delay + 3)
            
...

[Zixaphir]

Can you send your changes as a pull request to the dev branch?

[Godsfoot]

Not being a proficient extension writer/python dev, I opted to keep it simple but functional, and you can look towards improving it/making it customizable later.

Here is a basic PR for an initial implementation: #72

[Zixaphir]

This was merged awhile back. I apologize for not updating this issue