Update terraform modules based on prefix tags

[Bogdan A]

Which Renovate are you using?

Renovate Open Source CLI v23.67.0

Which platform are you using?

GitHub.com

Have you checked the logs? Don't forget to include them if relevant

{
"updates": []
 "depIndex": 0,
 "skipReason": "unsupported-version",
 "lookupName": "githubuser/myrepo",
 "datasource": "github-tags",
 "currentValue": "tfmodule_one-v0.0.9",
 "depNameShort": "githubuser/myrepo",
 "depName": "github.com/githubuser/myrepo",
 "depType": "github"
}

What would you like to do?

I have a couple of Terraform modules located in a repo and multiple tags pointing to each module. So for example I have tags like tfmodule_one-v0.0.9, tfmodule_two-v0.1.4 , etc. In Terraform I would reference them like this:

source = "github.com/githubuser/myrepo//terraform/modules/moduleone?ref=tfmodule_one-v0.0.9"

What I want to achieve is for Renovate to take first part before "-v" (so tfmodule_one) and maybe treat it as compatibility and then look into the target repo for the latest version which has this compatibility field as a prefix. So let's say now in the repo there are multiple tags: tfmodule_one-v0.0.11, tfmodule_two-v0.2.14, it should raise a PR to update to tfmodule_one-v0.0.11 since I am now using tfmodule_one-v0.0.9
Currently I'm getting unsupported-version and here is my current config, but I did try a lot of variations and none of them seems to work:

{
	"prHourlyLimit": 20,
	"terraform": {
		"fileMatch": ["\\.tf$", "\\.tfr$"],
		"versioning": "regex:((?<compatibility>.*)-v|v*)(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)$"
	},
	"packageRules": [{
		"datasources": ["github-tags"],
		"versioning": "regex:(?:.*)v(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)$"
	}]
}

The new solution should also work for the default tags, the ones which have v1.2.3 for example (which is the default behavior anyway, but just wanted to underline this)

[Rhys Arkins]

Can you format all code references above to be more readable?

[Bogdan A]

Should be better now

[Rhys Arkins]

You were definitely on the right path to use regex versioning.

Unfortunately it seems we validate versions too eagerly in the terraform manager instead of deferring such a check until after packageRules can be applied. I created renovatebot/renovate#7642 for tracking that, and I'd hope your approach would work after that (maybe with some tweaking - I haven't checked it for sure)

[Michael Kriese]

@bgdanix You should use a package rule with manager=["terraform"] and some package name / pattern filter. As workaround for now you can use a regexManger to do custom extract.

The terraform regex versioning you posted should work, remove the datasource regex, it would override the previous one.

[Bogdan A]

Thank you @viceice , I did try with regexManager but still no luck. This is what I used:

• input: source = "github.com/githubuser/myrepo//terraform/modules/moduleone?ref=tfmodule_one-v0.0.9"

	"regexManagers": [{
		"fileMatch": ["\\.tfr$ "],
		"matchStrings": ["\\s*source\\s*=\\s*\"(?<depName>.*?)\\/\\/.*\\?ref=(?<currentValue>.*?)\"$"],
		"datasourceTemplate": "github-tags",
		"versioningTemplate": "(?<compatibility>.*)-v|v*)(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)$"
	}]

[Michael Kriese]

fileMatch looks wrong to me, shouldn't it be "\\.tf$"

[Bogdan A]

Yes, but I also have other files which I want them to be considered as such, hence the ".tfr" extension. So just a simple file with .tfr extension, with the content of source = "github.com/githubuser/myrepo//terraform/modules/moduleone?ref=tfmodule_one-v0.0.9" and the previous indicated regexManagers is not being parsed at all, not showing in the log.

[Michael Kriese]

because your manager is wrong anyways: you need to remove the trailing space from fileMatch regex and you need to add regex: to versionigTemplate.

  "regexManagers": [{
      "fileMatch": ["\\.tfr$"],
      "matchStrings": ["\\s*source\\s*=\\s*\"github.com/(?<depName>.*?)\\/\\/.*\\?ref=(?<currentValue>.*?)\"\\s"],
      "datasourceTemplate": "github-tags",
      "versioningTemplate": "regex:(?<compatibility>.*)-v|v*)(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)$"
    }
  ]

updated matchStrings

[Michael Kriese]

can you please provide a simple public github repo, so we can solve your config more easily

[Bogdan A]

I tried but still not working. For the public repo: renovate-tst and tfrepo

[Michael Kriese]

thx, getting closer:

[Michael Kriese]

bgdanix/renovate-tst#1 🎉

[Bogdan A]

Nice! So the only thing missing was the ^ for versioningTemplate regex

[Michael Kriese]

no, was a little more, compare config:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": ["config:base"],
  "regexManagers": [{
      "fileMatch": ["\\.tfr$"],
      "matchStrings": ["\\s*source\\s*=\\s*\"github.com/(?<depName>.*?)\\/\\/.*\\?ref=(?<currentValue>.*?)\"\\s"],
      "datasourceTemplate": "github-tags",
      "versioningTemplate": "regex:^(?<compatibility>.*-v)(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)$"
    }
  ]
}

[Bogdan A]

Got it. Thank you very much for your help @viceice , much appreciated! I can confirm that it also works on my side

[Rhys Arkins]

I know there's a lot of symbols and backslashes in there but that regex manager is still a thing of beauty :)

[Michael Kriese]

yeah, feel like a regex guru 😜

[Bogdan A]

Just as an update for future reference: with the new version 23.69.0 and the below config it also works now, so there is no need for the regexManagers:

{
	"prHourlyLimit": 20,
	"terraform": {
		"fileMatch": ["\\.tf$", "\\.tfr$"],
		"versioning": "regex:^((?<compatibility>.*)-v|v*)(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)$"
	}
}