Create file-based persistence store
mhthies opened this issue · comments
The persistence store shall allow to store the current values of connected objects in a file or directory on the harddrive to persist them across restarts of the SHC application and the host computer. Therefore, it shall provide any number of named, readable + writable objects which write every received value update to the persistence file.
Optimally, changed values are stored immediately to persist them even with unexpected crashes/shutdowns. Still, the implementation should be somehow optimized to avoid parsing, updating serializing and rewriting the whole persistence store on each value update.
Current implementation approach: Persist values using their JSON-representation (since we expect value types to be JSON-serializable anyway). The connector names and values are stored as an object in a JSON file, but we insert some additional spaces after each key-value paar and keep a map of byte-offsets to each key-value pair in memory. This way, we can update individual values in-place in the JSON-file, as long as the length of the new value's JSON serialization does not exceed the old lengt (including the buffer-spaces). In this case, we would clear the whole key-value pair with spaces in-place and append the new value to the end of the file. Now and then, we can rewrite the whole file (esp. at startup) to truncate those empty bloat lines. Rewriting, however, should be done in crash-safe way, i.e. rewrite the data to a temporary file and replace the original file afterwards.