Sort and search by many_one associations fields

[yoieh]

Is your feature request related to a problem? Please describe.
I'm trying to build a frontend with searchable and sortable tables having one "main" collection with where many of the tables columns are many_one associations.

Simplified example:

MainName	sub1.name	sub1.sub2.name
name1	sub1Name1	sub2Name1
name2	sub1Name2	sub2Name1
name3	sub1Name1	sub2Name2

I need to be able to filter on the sub fields for now I have to move the data of sub fields to the main collection to accomplish this.

This means that I have a fields in the main collection that has to be updated if the associations name filed is updated and in my project there are even worse nested associations that needs to be accessible from the main field like:

Main.sub1.sub2.sub3.name

And every sub doc has its own table on the frontend. In my solution if sub3.name is updated I now need to update every parent after each other in order and doing this in with the update: {post: () => RestHapi.update() } and update: {pre: () => payloade.subName = ... } middleware on each layer.

sub3.model

update: {
  post: async function (request, result, Log) {
    // get sub2s that has sub3s
    const sub2s = await mongoose
      .model("sub2")
      .find({ sub3: result._id });
    
    // uppdate sub2
    await Promise.all(
      sub2s.map(sub2=> 
        RestHapi.update('sub2', sub2._id, {sub3: result._id})
      )
    )
  }
}

sub2.model

update: {
  pre: async function(_id, payload, request, Log) {
    // uppdates sub3 name
    if(payload.sub3) {
      const sub3= await mongoose
        .model("sub3")
        .findOne({ _id: payload.sub3});

      payload.sub3Name = sub3.name;
    }

    return payload
  },
  post: async function (request, result, Log) {
    // get sub1s that has sub2s
    const sub1s = await mongoose
      .model("sub1")
      .find({ sub2: result._id });
    
    // uppdate sub1
    await Promise.all(
      sub1s.map(sub1=> 
        RestHapi.update('sub1', sub1._id, { sub2: result._id })
      )
    )
  }
}

sub1.model

update: {
  pre: async function(_id, payload, request, Log) {
    // uppdates sub2 and sub3 name
    if(payload.sub2) {
      const sub2= await mongoose
        .model("sub2")
        .findOne({ _id: payload.sub2});

      payload.sub3Name = sub2.sub3Name;
      payload.sub2Name = sub2.name;
    }

    return payload
  },
  post: async function (request, result, Log) {
    // get mains that has sub1s
    const mains = await mongoose
      .model("main")
      .find({ sub1: result._id });
    
    // uppdate sub1
    await Promise.all(
      mains.map(main => 
        RestHapi.update('main ', main ._id, { sub1: result._id })
      )
    )
  }
}

main.model

update: {
  pre: async function(_id, payload, request, Log) {
    // uppdates sub1, sub2 and sub3 name
    if(payload.sub1) {
      const sub1= await mongoose
        .model("sub1")
        .findOne({ _id: payload.sub1});

      payload.sub3Name = sub1.sub3Name;
      payload.sub2Name = sub1.sub2Name;
      payload.sub1Name = sub1.name;
    }

    return payload
  },
  post: async function (request, result, Log) {
    // get mains that has sub1s
    const mains = await mongoose
      .model("main")
      .find({ sub1: result._id });
    
    // uppdate sub1
    await Promise.all(
      mains.map(main => 
        RestHapi.update('main ', main ._id, { sub1: result._id })
      )
    )
  }
}

I'm also setting the "moved fields" on create manually in the create middleware on each layer.
and now I'm able to filter and search even on sub3Name from my main collection.

Performance wise this is bad really bad if I have many main documents.. and if any fails I'm screwed.

Describe the solution you'd like
I need to be able to search and sort by sub1.name I'm not sure if this is doable at all even on a lower level..

[Justin Headley]

@yoieh yeah this is a tricky problem. However if I understand you correctly then it sounds like you need to take advantage of the duplicate fields feature. It was added for this type of use case.

[yoieh]

I totally overlooked that feature. It seems to do exactly what I need.

[yoieh]

Ha rebuilt my associates using this feature now and it seems to work. Only Issue is that it dose seem to add my duplicate fields when using RestHapi.create() in my seeder. I'm setting enabelDuplicateFields to true before using it but I have to add the values manually and have them in my schema

[Justin Headley]

@yoieh please check out these docs for using duplicate fields when calling wrapper methods (e.g. RestHapi.create(): https://resthapi.com/docs/mongoose-wrapper-methods.html#simulated-rest-calls

[yoieh]

I missed that the arguments needs to be in a object to use the "restCall=true" also severer has to be registered in the same script if RestHapi.create({restCall: true}) is used in a seeder script.
Also if auth is used credentials needs to be set as such

{
  model,
  payload,
  Log,
  restCall: true,
  credentials: { user: _id }`
}

It took some time to understand how it works but i got it to work now with in my seeders and it seriously made my code much less complex.
Big tanks to you @JKHeadley !

[Justin Headley]

@yoieh good to hear! I'll try to update the docs to make it more clear. Let me know if you have any suggestions.