Issue with using k-NN’s Painless Scripting functions for AWS Elasticsearch (7.10)

[Abhilash Goswami]

"script": {
                "lang": "painless",
                "source": "1.0 + cosineSimilarity(params.query_value, params._source.use)",
                "params": {
                    "query_value": [...]
                }
            }
        }

Here is the script which I am using to do a cosine similarity between two vector fields having dimension of 512. The setting index.knn is set to false. I am getting this error which is basically not being able to cast the array list to KNN vector.

"root_cause": [ { "type": "class_cast_exception", "reason": "class_cast_exception: Cannot cast java.util.ArrayList to com.amazon.opendistroforelasticsearch.knn.index.KNNVectorScriptDocValues" } ]

Can someone help with this please?

FYI I know there is the same function which works for "knn" type query, but I don't want to use that as I'll need to add some more custom scoring functions.

[John Mazanec]

Hi @Incrementing-coder , is "params._source.use" the field name containing the vector? If so, could you try doc[params._source.use]?

[Abhilash Goswami]

Hi @jmazanec15, tried it but it's giving me an script exception error. (runtime_error). The values of the field are being picked, I tried doing params._source.use.length and the length of the array is coming, as the score of the document. This is has something to do with casting the query array list to KNNVectorScriptDocValues.

[Vijayan Balasubramanian]

@Incrementing-coder
can you try following approach?

cosineSimilarity(params.query_value, doc['use'])

Also, can you paste the error and mapping to understand more if you still have problem?

Note: Please add size check as mentioned here if all of your documents doesn't have knn vector field.

[Abhilash Goswami]

cosineSimilarity(params.query_value, doc['use'])

This is working! Thanks Vijayan!