configstore
The configstore library aims to facilitate configuration discovery and management. It mixes configuration items coming from various (abstracted) data sources, called providers.
Items
An item is composed of 3 fields:
- Key: The name of the item. Does not have to be unique. The provider is responsible for giving a sensible initial value.
- Value: The content of the item. This can be either manipulated as a plain scalar string, or as a marshaled (JSON or YAML) blob for complex objects.
- Priority: An abstract integer value to use when priorizing between items sharing the same key. The provider is responsible for giving a sensible initial value.
Configuration format
The item keys are NOT case-sensitive. Also, -
and _
characters are equivalent.
The exact input format of the configuration depends on the provider. Providers can either be loaded manually in your code, or controlled by the env variable CONFIGURATION_FROM
.
Example main.go
func main() {
configstore.InitFromEnvironment()
val, err := configstore.GetItemValue("foo")
if err != nil {
panic(err)
}
fmt.Println(val)
}
Outputs:
bar
Reading from a file
Env:
CONFIGURATION_FROM=file:foo.cfg
Contents of foo.cfg file:
- key: foo
priority: 12
value: bar
Key/value pairs are read from a single file in yaml.
Reading from env
Env:
CONFIGURATION_FROM=env:CONFIG
CONFIG_FOO=bar
Key/value pairs are read from the environment, with an optional prefix. Remember that key names are case-insensitive, and that _
and -
are equivalent in key names.
Reading from a file hierarchy
Env:
CONFIGURATION_FROM=filetree:configdir
Contents of configdir directory:
foo
Contents of configdir/foo file:
bar
Key/value pairs are read by traversing a root directory. Each file in the dir represents an item: the filename is the key, the contents are the value.
To have several items sharing the same key, you can use a single level of sub-directory as such: configdir/foo/bar1
, configdir/foo/bar2
, ... The filenames bar1
/bar2
are not used in the resulting items.
Reading from a custom source
These built-in providers implement common sources of configuration, but configstore can be expanded with other data sources. See Example: multiple providers.
Example 101
file.txt:
- key: foo
value: bar
- key: baz
value: bazz
func main() {
configstore.File("/path/to/file.txt")
v, err := configstore.GetItemValue("foo")
fmt.Println(v, err)
}
This very basic example describes how to get a string out of a configuration file (which can be JSON or YAML). To do more advanced configuration manipulation, see the next examples.
Example: multiple providers
Configuration Providers represent an abstract data source. Their only role is to return a list of items.
Some built-in implementations are available (in-memory, file, env, ...), but the library exposes a way to register a provider factory, to extend it and bridge with any other existing system.
Example mixing several providers
// custom provider with hardcoded values
func MyProviderFunc() (configstore.ItemList, error) {
ret := configstore.ItemList{
Items: []configstore.Item{
// an item has 3 components: key, value, priority
// they are defined by the provider, but can be modified later by the library user
configstore.NewItem("key1", `value1-higher-prio`, 6),
configstore.NewItem("key1", `value1-lower-prio`, 5),
configstore.NewItem("key2", `value2`, 5),
},
}
return ret, nil
}
func main() {
configstore.RegisterProvider("myprovider", MyProviderFunc)
configstore.File("/path/to/file.txt")
configstore.Env("CONFIG_")
// blends items from all sources
items, err := configstore.GetItemList()
if err != nil {
panic(err)
}
for _, i := range items.Items {
val, err := i.Value()
if err != nil {
panic(err)
}
fmt.Println(i.Key(), val, i.Priority())
}
}
Example: advanced filtering
When calling configstore.GetItemList(), the caller gets an ItemList.
This object contains all the configuration items. To manipulate it, you can use a ItemFilter object, which provides convenient helper functions to select and reorder the items.
All objects are safe to use even when the item list is empty.
Example of use:
func main() {
items, err := configstore.GetItemList()
if err != nil {
panic(err)
}
// we start by building a filter to manipulate our configuration items
// we will apply it on our items list later
filter := configstore.Filter()
// get the databases
filter = filter.Slice("database")
// now we have a list of database objects, let's assume the payload resembles this:
// {"name": "foo", "ip": "192.168.0.1", "port": 5432, "type": "RO"}
// {"name": "foo", "ip": "192.168.0.1", "port": 5433, "type": "RW"}
// {"name": "bar", "ip": "192.168.0.1", "port": 5434, "type": "RO"}
//
// the "database" initial key provides too little information to extract the data relating to a specific DB
// we need to drill down into the value
// we need to unmarshal the JSON representation of the whole sublist
// we pass a factory function that instantiates objects of the correct concrete type
filter = filter.Unmarshal(func() interface{} { return &Database{} })
// now we want to actually index and lookup by database name, instead of the generic "database"
// we apply a rekey function that does payload inspection
filter = filter.Rekey(rekeyByName)
// we have duplicate elements: database "foo" is present twice
// we want to favor the RW instance if possible
// we apply a reordering function that does payload inspection
filter = filter.Reorder(prioritizeRW)
// we only need 1 of each, we squash to only keep the single highest priority of each key
filter = filter.Squash()
// now we have only 2 items left:
// {"name": "foo", "ip": "192.168.0.1", "port": 5433, "type": "RW"}
// {"name": "bar", "ip": "192.168.0.1", "port": 5434, "type": "RO"}
// we can finally apply it on our list
items = filter.Apply(items)
// the same thing, more concise:
filter = configstore.Filter().Slice("database").Unmarshal(func() interface{} { return &Database{} }).Rekey(rekeyByName).Reorder(prioritizeRW).Squash()
items, err = filter.GetItemList() // shortcut: applies the filter to the full list from configstore.GetItemList()
if err != nil {
panic(err)
}
// declaring your filter separately like this lets you define it globally and execute it later
// that way, you can use its description (String()) to generate usage information.
}
type Database struct {
Name string `json:"name"`
IP string `json:"ip"`
Port int `json:"port"`
Type string `json:"type"`
}
func rekeyByName(s *configstore.Item) string {
i, err := s.Unmarshaled()
// we see here the error that was produced when we called *ItemList.Unmarshal(...)*
// we ignore it for now, it will be handled when the *main()* retrieves the object.
if err == nil {
return i.(*Database).Name
}
return s.Key()
}
func prioritizeRW(s *configstore.Item) int64 {
i, err := s.Unmarshaled()
if err == nil {
if i.(*Database).Type == "RW" {
return s.Priority() + 1
}
}
return s.Priority()
}