Automate getting instance data
hidalgopl opened this issue · comments
Currently, keeping instance data up-to-date is hard, as they are generated by bunch of home-grown scripts which are run manually (and then commited to repo).
To avoid maintaining those scripts, we should use ready solution to get info and pricing about instances across various providers.
Possible solution can use banzaicloud/cloudinfo. As a first step, we could use cloudinfo to get that data and a custom script to convert it to the format understandable for kip [example]https://github.com/elotl/kip/blob/master/pkg/util/instanceselector/aws_instance_data.go). This could be done as a part of kip's release pipeline, so we can be sure that latest kip release has fresh instance data.
cloudinfo currently supports storing data in redis or cassandra, so we probably need to scrape data, store it in redis, and then fetch it and save to json.