Indexing (and searching within) nested values
zacklerner opened this issue · comments
I just finished attempting the fix suggested in this recently closed issue, but after rebuilding Flexsearch's index I'm still unable to search for values contained in an array of tags.
To further complicate things the body copy of each post is returned as a value in an array of objects nested within another array of objects. Is it currently possible to index and search nested content?
To clarify things a little, here's an example of one of the posts I'd be searching for (this output was returned by searchResults):
[
{
"index": "SanityPost",
"id": "ddaf3c38-4159-4470-9da8-33e3ead2ec9f",
"path": "/my-first-introduction-to-portable-text/",
"body": [
{
"_key": "8f3691f42e1b",
"_type": "block",
"children": [
{
"_key": "8f3691f42e1b0",
"_type": "span",
"marks": [],
"text": ""
},
{
"_key": "8f3691f42e1b1",
"_type": "span",
"marks": [
"217576d2be65"
],
"text": "Portable Text"
},
{
"_key": "8f3691f42e1b2",
"_type": "span",
"marks": [],
"text": " is a JSON based rich text specification for modern content editing platforms. The rich text editor in Sanity Studio uses it and lets me embed data objects both inline and inbetween my text blocks. I can even have references in them so that my content backend warns me if I try to delete content that is linked to. I can also customize the editor with "
},
{
"_key": "8f3691f42e1b3",
"_type": "span",
"marks": [
"fc7e797c1dd6"
],
"text": "React components and paste-handling overrides"
},
{
"_key": "8f3691f42e1b4",
"_type": "span",
"marks": [],
"text": ". I can even "
},
{
"_key": "8f3691f42e1b5",
"_type": "span",
"marks": [
"2c6896e3acbc"
],
"text": "make it speak"
},
{
"_key": "8f3691f42e1b6",
"_type": "span",
"marks": [],
"text": "!"
}
],
"markDefs": [
{
"_key": "217576d2be65",
"_type": "link",
"href": "https://www.portabletext.org"
},
{
"_key": "fc7e797c1dd6",
"_type": "link",
"href": "https://www.sanity.io/docs/what-you-need-to-know-about-block-text/customization"
},
{
"_key": "2c6896e3acbc",
"_type": "link",
"href": "https://www.smashingmagazine.com/2019/03/sanity-portabletext-speech-synthesis/"
}
],
"style": "normal"
}
],
"tags": [
"Portable Text",
"JSON",
"Rich Text",
"RTE"
],
"title": "My First Introduction to Portable Text"
}
]
There appears to be some documentation on this in the Flexsearch docs, but I'm not sure how this might translate to the Gridsome plugin:
Looks like I may have found the answer I was looking for: nextapps-de/flexsearch#113 (comment)
Is the functionality described in that comment something that would be supported by the current iteration of the plugin? If not is it something you plan on supporting? Thanks!
So you can't search the tags array, when it is added as a search field? That's strange, as I believe it worked on a demo starter I tried - searching through simple arrays like that should be supported out-of-the-box. If that is the case, I'll have a look at it.
And yes - searchFields can be an array or an object, both of which are passed directly to the FlexSearch instance. When 0.7.0 is released, I'll make sure this plugin is compatible.
I think I could make the configuration of this plugin a bit better too, so I'll probably make a few general changes when the new version is released...
Thanks for the confirmation! Was just able to test this again with the following settings:
searchFields: {
tags: {
encode: "extra",
tokenize: "full"
}
}
Also tried:
searchFields: {
tags: {
encode: "advanced",
tokenize: "forward"
}
}
Neither of these yield any results despite tags being a standard array. The only way I'm able to get any results is if I include title in the search fields:
searchFields: {
tags: {
encode: "advanced",
tokenize: "forward"
},
title: {}
}
Even then, only terms indexed within the title field yield results. Here's an example of what Flexsearch returns for the term 'portable' with the above settings:
{
"index": "SanityPost",
"id": "ddaf3c38-4159-4470-9da8-33e3ead2ec9f",
"path": "/my-first-introduction-to-portable-text/",
"body": [
{
"_key": "8f3691f42e1b",
"_type": "block",
"children": [
{
"_key": "8f3691f42e1b0",
"_type": "span",
"marks": [],
"text": ""
},
{
"_key": "8f3691f42e1b1",
"_type": "span",
"marks": [
"217576d2be65"
],
"text": "Portable Text"
},
{
"_key": "8f3691f42e1b2",
"_type": "span",
"marks": [],
"text": " is a JSON based rich text specification for modern content editing platforms. The rich text editor in Sanity Studio uses it and lets me embed data objects both inline and inbetween my text blocks. I can even have references in them so that my content backend warns me if I try to delete content that is linked to. I can also customize the editor with "
},
{
"_key": "8f3691f42e1b3",
"_type": "span",
"marks": [
"fc7e797c1dd6"
],
"text": "React components and paste-handling overrides"
},
{
"_key": "8f3691f42e1b4",
"_type": "span",
"marks": [],
"text": ". I can even "
},
{
"_key": "8f3691f42e1b5",
"_type": "span",
"marks": [
"2c6896e3acbc"
],
"text": "make it speak"
},
{
"_key": "8f3691f42e1b6",
"_type": "span",
"marks": [],
"text": "!"
}
],
"markDefs": [
{
"_key": "217576d2be65",
"_type": "link",
"href": "https://www.portabletext.org"
},
{
"_key": "fc7e797c1dd6",
"_type": "link",
"href": "https://www.sanity.io/docs/what-you-need-to-know-about-block-text/customization"
},
{
"_key": "2c6896e3acbc",
"_type": "link",
"href": "https://www.smashingmagazine.com/2019/03/sanity-portabletext-speech-synthesis/"
}
],
"style": "normal"
}
],
"tags": [
"Portable Text",
"JSON",
"Rich Text",
"RTE"
],
"title": "My First Introduction to Portable Text"
}
Searches for 'rich', 'json', and 'rte' yield nothing, which seems to confirm that values within the tags array are inaccessible. Hope this helps clarify!
Thanks @zacklerner - I have just had time to take a look at it, and you are right. After looking further at this issue it seems others had issues with tags: { encode: "advanced" ... too, so I tried the JSON.stringify(array) suggestion, and that seemed to work.
I have just released v0.1.10, which should handle this automatically for you; please try this, and let me know if it works for you too?
Amazing! Reverting back to the original config searchFields: ['title', 'tags'] works like a charm. Thanks for the quick fix.