HIera 5 Scoped Lookup
soudaburger opened this issue · comments
Due to the way that Hiera 5 works, parameters are no longer defaulted to GLOBAL scope. As a result, when attempting to use lookup(somestring) against an HTTP backend, like Consul for example, is either 1) ignored when this module is referenced in module-level hiera or 2) required to be in global scope to actually serve the lookups.
Putting the hiera-http lookups in global scope would cause WAY too many lookups against the HTTP backend, especially in cases where I know I don't need to lookup against the HTTP backend.
I could do lookup(mymodule::somestring) but then my URI ends up being http://example.com/path/to/uri/mymodule::somestring, and when wishing to use this with Consul, just makes it extremely messy.
I have instead modified the hiera_http.rb file to allow the context of lookup(mymodule::somestring)' to remove the scoped variable header and thus will lookup mymodule::somestringassomestring`.
This allows for my modules to determine when/how to look things up without having to confine the keys and then manage them globally. Thoughts?
Lines 29-31 below. I added line 30.
options['uri'].gsub! 'KEY', key
options['uri'].gsub!(/([^\/]*::)/, '')
result = http_get(context, options)
@soudaburger I see your issue here, I don't like the idea of just chopping off .*:: as people may already be relying on this behaviour as it is now.... It seems to me we could expand on the idea of __KEY__ and add other replacements.
Eg: for a lookup key of foo::bar::tango
__KEY__ == foo::bar::tango__CLASS__ == foo::bar__MODULE__ == foo__PARAMETER__ == tango
That would enable you to interpolate __PARAMETER__ in your URL to achieve what you are trying to do.
Thoughts?
@crayfishx , that would actually be AWESOME.
That makes complete sense and still gives us the flexibility to use to meet our needs. That's why I threw this out there. Whenever you have the bandwidth to implement, I would be more than happy to test. Thanks!
Also, if there is a way you can allow for filenames with file extensions, that would save me an incredible amount of time!! I believe lookup() currently strips off file extensions, and for a URI with files, I would LOVE to be able to use this project for that. Thanks for any help!
I somehow managed to edit your comment instead of replying to it in forkhub for Android.... Sorry.... Please let me know (again) what you need :)
@soudaburger I fixed my earlier fubar with your comment.... I'm working on the variable interpolation part of this but I dont understand your later comment regarding filename extensions and how that's even relevant to hiera-http.... could you explain?
Cheers
@crayfishx , so, I would ultimately love to use hiera-http for more than just consul, possibly regular HTTP lookups, so this would include serving up binary files from some locale that don't belong in Git. That being said, the way lookup works is by chomping on ".", so any lookup such as lookup('somefile.txt') would ultimately result in looking up somefile without the file extension. This makes it extremely difficult to deal with files with file extensions. Does that make sense? It may be out of scope for this, but solving that problem would be a huge leap forward in usability from my perspective. Let me know what else you need from me. Thanks!
Ah - so this isn't related to filenames per-se... puppet lookup / hiera 5 allows you to drill down into data lookups using the dot notation... eg facts.os.family so when you do a lookup for somefile.txt the lookup function thinks you want somefile['txt'] - so it will try and lookup somefile and then drill down - at least thats my understanding... It's been long documented that dots are illegal in Hiera lookups for this reason. Its out of scope for this ticket but let me think on it:-)
Totally understand, just thought i would ask. Thanks for the quick reply.
@soudaburger - I believe I just had an epiphany with my morning coffee - you might find URL encoding your key might just work when using hiera-http
lookup('somefile%2Etxt')
It's a long shot, but might be a starting place for your problem