Giters

cullenmcdermott / terraform-provider-porkbun

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Issues due poor requests limits in Porkbun API

bhean opened this issue · comments

commented

Hi,

I think the Porkbun API has very low limits about the requests/time and paralell requests.
If you use this provider to manage just a few DNS records, everything looks good... but if you are trying to handle bigger configurations, there are a lot of random issues during the Terraform plan/apply.

I disabled the Terraform paralelism (with terraform plan -parallelism=1) and certainly it was better... but after adding a few more domains (nothing huge, just 5 or 6 domains with 6 or 7 DNS records each) the plan/apply is not able to work again.

So I'm pretty sure that there are some very low limits in the Porkbun API about concurrent request and request/min, at least.

There is no documentation at all in Porkbun website. I wrote to their support asking about that, but I have no response yet.

I was checking some other providers, and lot of them implements retries mechanism to avoid these kind of errors, so looks like the best solution to avoid error...

I wish I could send a PR with that implementation, but I'm not familiar with Go at all...

Regards,

commented

Hey! I can try to look into this in the next week or so. Could you run TFLOG=trace terraform apply it'll be a lot of output so I'm really just interested in the errors if you're able to pick them out, but if not it's ok. Please also check for any sensitive info in the logs and redact before sharing with me.

I'll try to take a look at what other providers do to get around rate limits like this.

commented

This is the plan with tracing enabled (I removed sensitive information, and replace domain names with domain1, domain2, etc): terraform.log

BTW, I received the response from the Porkbun developers confirming my thoughts:
"About 60 / minute or 2 / second. There are some bursting capabilities up to 5 / second. The best bet would be to keep things to 1 / second if there are constant commands being issued and to wait / resend if a threshold is reached."

So, for me make sense that the problem is that Terraform is reaching these limits even with parallelism=1 because when I try with a small group of DNS records it works, but as soon as I add a few more, it fails (the log attached is trying to manage 28 records, in case this information helps you)

Please, don't hesitate to ask me any other info or tests from my side,

Thanks