zentity-io / zentity

Entity resolution for Elasticsearch.

Home Page:https://zentity.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Dynamic field mapping

marcospassos opened this issue · comments

First of all, thank you for the awesome work done with the library!

We have a use case where the user profile properties are dynamic and we want to allow resolving the identity using one of these fields:

{
    "name": "Marcos",
    "last_name": "Passos",
    "extra": {
        "loyalty_number": "123"
    }
}

In this example, loyalty_number is a dynamic field, unknown at mapping time. However, we would like to match (exactly) this field in some cases.

Is it supported? What is the recommended approach?

@marcospassos

Sorry for the long delay! I'm able to bring my attention back to the project now.

By now you may have answered your own question. I'd be interested to hear your experience.

Zentity appears to do what you're asking. I just did a quick test in the zentity sandbox. I modified one of the entity models by adding a non-existent field under the "indices" section. Then I ran a resolution job with multiple resolvers, one of which used a matcher to query the non-existent field. The job continued without error and matched documents using the other resolvers in my job configuration. I ran a second resolution job using only that resolver, and the job completed with zero results and without error. These are the desired behaviors, given that it's valid and common for documents to lack fields in Elasticsearch. Elasticsearch lets you query any field even if it doesn't exist in index. It will process the query and return zero results without error if the field doesn't exist.

Thank you for the explanation, @davemoore-!