xinyadu / nqg

neural question generation for reading comprehension

Home Page:https://arxiv.org/abs/1705.00106

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Set of undevidable words

zharenkov opened this issue · comments

Is there the setting, which can define the list of unsplittable words.
E.g. I have a text about iPhone X. There is an info about Face ID, True Tone flash and so on.
When I gives this text to the model (trained on squad dataset or ms marco dataset ), it asks questions like this:
SENT 1: Face ID is a new authentification method....
PRED 1: What is Face (but we understand that question should be about Face ID)
SENT 2: It also has a quad-LED True Tone flash
PRED 2: What is Tone (instead of True Tone)

And the same situtions with other technology names.
I tried to and -phrase_table setting to translate config with file contains
Face ID ||| FaceID
and some other variation, but got no effect.

Thanks!