Implement a search endpoint using solr

Question

Implement a search endpoint using solr

anitacaron opened this issue 2 years ago · comments

Anita Caron commented 2 years ago

Add solr docker image to the docker-compose
Do a 2 SOLR tutorials like https://www.youtube.com/watch?v=v4GNOnvdgtY (didnt check this)
Use sssom CLI to turn mapping sets to JSON
Write some code to load JSON into SOLR core
Create endpoint to search any field
Create endpoint to search a range for confidence field

Nico Matentzoglu · Answer 1 · Wed Nov 30 2022 22:07:03 GMT+0800 (China Standard Time)

Here is some ancient code I wrote:

https://github.com/VirtualFlyBrain/vfb-pipeline-dumps/blob/master/scripts/obographs-solr.py

Something that's maybe irrelevant: https://github.com/VirtualFlyBrain/vfb-solr

Be very careful with this, don't just copy blindly: https://github.com/VirtualFlyBrain/Docker-VFB-SOLR/blob/master/loadOLS.sh

Anita Caron · Answer 2 · Fri Mar 31 2023 00:02:13 GMT+0800 (China Standard Time)

Use sssom CLI to turn mapping sets to JSON

This also needs a script to flat the data. The decision would be to add all mapping_set information to each of its mappings, but there are the same attributes' names in both levels. Discussing with Nico, he said that the policy (not official) is to prioritise the field at the mapping level. And only take the mapping_set field if it's not available in the mapping. However, this is still in discussion on a case by case.

The list of terms present in mapping set and mapping: TBD