Any way to support storing FS.File in mongo db?
aldeed opened this issue · comments
For some reason I thought we could store an FS.File pointer directly in mongodb as an EJSON type, but we can't. Mongo throws a not okForStorage error. The docs do have a note saying that EJSON support is limited to minimongo, but I guess I forgot or didn't read that.
Anyway, I'm wondering if there's some way we can work around this? Possibly a method one can pass a collection to add a transform to it, which would transform FS.File pointers into actual FS.File instances? I'm referring to related collections, not FS.Collections.
This should still work... Cannot see why not, is the ejson package installed?
Maybe we could do something clever in collection2.
EJSON.fromJSONValue and EJSON.toJSONValue
I dont get why meteor.collections on the server dont support ejson,
We are going to ditch the ejson package and force the user to use collection2 or handle references Them selfs. It's a bit misleading, ejson should not be used for any other types than types natively supported in bson vs json.
We are ditching ddp, ejson, http package from meteor, it's basically only the client-side that have to be meteor packages, the rest could have been node packages.
From what I can tell with experimenting, this should be easily fixable in Meteor core. See meteor/meteor#1890
And actually, I think the fix is all in overridable parts of ejson code, so cfs-ejson-file could include patched copies of the EJSON methods pending core changes.
Nope I dont think so - Ive imagined ejson all wrong i think.
Dates are actual dates in the mongo - eg. fromEJSON is stored into the server mongo - what happens when we do this with the FS.File? (We try to save an instance of FS.File)
bson can store binary, dates etc. so its not EJSON in the server mongo and it shouldnt be - why I think that custom ejson types should deprecate totally only the builtin types makes sense?
If we should have custom ejson types they would have to be treaded differently when stored into the server mongo - eg. it should not use fromEJSON when putting custom types into the mongodb - and work differently when extracting.
I'm not betting on changing the Meteor core - I honestly dont think they have the time, and I kinda like raw mongo on the server for performance.
I think we should either deprecate the cfs-file-ejson or we could make a bet: I say they wont implement - not sure what the stakes should be, beer? :)
I don't think I'll take that bet. :)
But there are literally only 4 lines of code to change and they would not lose backwards compatibility. The only confusing thing might be people learning how to query on the data structure that is stored.
Yeah, probably won't get done in core.
Well I really liked the idea of being able to store the file instance directly in a related collection rather than manually storing id or id+collection pointers. But I guess that's what we'll have to do.
Yeah, well I like the concept as it was in my head before - but I also see that its not trivial to have proper behaviour on the server mongdo - since there should be differentiated between native mongo supported types and custom types - so Meteor should deprecate custom types imho.
your schema package + collection2 could make this work but it should be in conjunction with a join pattern... And it should not use ejson to make it work. (really not that difficult and would be safer)
Agreed. We've already been discussing a paradigm for joins, relations, and cascading operations in collection2, and this is giving me some ideas about that. It will not rely on EJSON for storage, but it would be nice to be able to rely on custom EJSON types as far as getting the custom object sent correctly over DDP. That part works correctly and is necessary for server-side validation to succeed in C2, which is why there is still some use for the ejson-file package.
I've been thinking it should actually be a separate add-in package, collection2-relations. Maybe I'll get started on a first draft next week. I have a client project that it would be really useful for.