Migrating between versions of data
nightly opened this issue · comments
Default values work when first persisting to storage.
However, after updating the store schema, newly created fields are not given their default values. Instead, the old persisted store is loaded with missing fields.
Hi @nightly,
I haven't decided if this package should handle migrations.
It would require versioning the data, and that means adding extra data, so I'd have to think about backwards compatibility.
It could be built as an opt-in feature.
But I think for right now, you can perform it by placing a version number in the key name:
const store = persisted('my-key:v1', initialValue)Then to migrate, check if the key for v1 exists:
let initialValue = ...
let v1 = localStorage.getItem('my-key:v1')
// check if old version exists
if (v1) {
// migrate and update storage
localStorage.setItem('my-key:v2', migrate(v1))
}
// update the key
const store = persisted('my-key:v2', initialValue)This above can be wrapped in a function.
I will close this issue for now, but if many people need this, I'd consider adding it in the future.
Hello, Josh.
This is the case I exposed in one of my issues: A validation function that runs after first retrieval of the stored data. We would then write logic inside the validation function to upgrade values, set new defaults, delete unneeded data, etc.
Issue #191 is the one I wrote requesting the feature for things just like this one.
@webJose what do you think about adding a new option for passing a "transformer" function,
For example, the API could look like this:
persisted(key, initialValue, {
transform(value) {
// migrate from version 1
if (value.version == 1) {
return {...value, newField: defaultValue, version: 2 }
}
// otherwise, just use the current value
return value
}
})It could handle migrations, plus any other conversions that are necessary.
Hi! Yes, regardless of the name: A place for consumers of the data to be able to make sure that the store will return a valid, consistent value. I would just emphasize that the function should only be called if there was a value stored in local/session storage. If there was no value, it should simply fall back to the initial value. The function would only be called once: On first storage read.
I still think my other issue is important: What if the stored value throws an error when being de-serialized? Maybe we can merge the two: If an error occurs trying to de-serialize, then transform() is run with the stored string value as value, and maybe a second parameter with the error that occurred.
@webJose I think parse errors should be considered a seperate thing. That way the API for transforming is kept simple.
Also, I think for now, parse errors can be handled with a custom serializer.
I found there is one problem with migrating schemas when syncing between multiple tabs.
It's possible to have one tab with older code using the old schema, and another tab is reloaded with new code and needs to migrate the schema. That would cause a storage event between tabs and could cause inconsistent data in the tab with the old code.
So, I wonder if migration would be safer using a different key. ie key:v1, key:v2
Hello again. I understand there are ways to get the parsing errors. I think the point is not that, but what needs to be done. I'll just say this much and then shut up about it.
I imagine I would do this to catch parsing errors:
const initialValue = {};
const myParser = {
parse(data) {
let value = undefined;
try {
value = JSON.parse(data);
}
catch (e) {
value = initialValue;
// Do something about e.
}
return value;
}
};
export default persisted(initialValue, { ... });The highlights:
- I think 99% of the time this is what people will need to do: On error, default to
initialValue. - I had to refactor
initialValueto be able to consume it in two places. - I am forced to do this with every persisted store I create in the application.
Yes, of course an experienced developer would encapsulate this into a factory function and not repeat the code. But let's remember that persisted() is the factory. Wouldn't it be much safer to have this built-in in persisted(), pretty much covering all points above? What remains there is: What to do with the caught error object. This is why I asked for an option or feature to tell the factory function what to do with it.
Al right, I won't bring this topic up anymore. 😄 I don't want to come across as annoyingly persistent.
Good catch about the storage event migrating newer data into an older app instance. Your keyed idea is probably viable, although versioning the keys is probably not everyone would go for. I can imagine people might want options. From the top of my head, another option would be to not listen to the storage event at all, or perhaps v1 and v2 have common ground, so maybe what we want is to selectively merge the incoming value with the existing value.
I think this is an entirely new topic. My first inclination would be to have an onStorage handler in the options for consumers of the store to be able to run whatever code they want. On top of that, I would probably code the versioning of the keys as the "stock" option to handle these conflicts. I know, however, that this probably sound badly in your ear because you want to keep options at a minimum. Personally, I don't mind the options to grow as long as they have reasonable defaults. This way I have a lot of functionality at my disposal without having to grow my codebase to cover many things I might not be needing.