Sparse-first search and hybrid search not working
pragnakalpdev7 opened this issue · comments
Hello,
I am facing issue in sparse-first search and hybrid search:
Dense-First Search is working fine, but when I select the other options it gives the following error:
KeyError: "Unable to open object (object '3580546' doesn't exist)"
I have used the pretrained model and then created custom phrase index for "dev-v1.1"
ERROR:flask.app:Exception on /api [GET]
Traceback (most recent call last):
File "/root/anaconda3/envs/despi/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/root/anaconda3/envs/despi/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/anaconda3/envs/despi/lib/python3.6/site-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/root/anaconda3/envs/despi/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/root/anaconda3/envs/despi/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
raise value
File "/root/anaconda3/envs/despi/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/root/anaconda3/envs/despi/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functionsrule.endpoint
File "open/run_demo.py", line 128, in api
doc_top_k=5)
File "open/run_demo.py", line 94, in search
search_strategy=search_strategy, doc_top_k=5)
File "/root/denspi/open/mips_sparse.py", line 291, in search
doc_top_k=5)
File "/root/denspi/open/mips_sparse.py", line 218, in search_start
(doc_idxs, start_idxs), start_scores = self.search_sparse(query_start, doc_scores, doc_top_k)
File "/root/denspi/open/mips_sparse.py", line 168, in search_sparse
doc_group = self.get_doc_group(doc_idx)
File "/root/denspi/open/mips.py", line 121, in get_doc_group
if len(self.phrase_dumps) == 1:
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "/root/anaconda3/envs/despi/lib/python3.6/site-packages/h5py/_hl/group.py", line 262, in getitem
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5o.pyx", line 190, in h5py.h5o.open
KeyError: "Unable to open object (object '3580546' doesn't exist)"
ERROR:tornado.access:500 GET /api?strat=sparse_first&query=pharmacy%20department%20and%20specialised%20areas%20 (127.0.0.1) 419.57ms
Hi @seominjoon @jhyuklee
Any update on this?
Can you please tell some ideas on how to make SFS and Hybrid search work?
In this version of code, you cannot use sparse first search for dev-v1.1 unless you have the document-level tfidf for SQuAD. See here. Thanks.