XGBoostError on initialization

[Natalie Prange]

When trying to run the example code snippet from the README I get the following error upon initialization of Xrenner:

Traceback (most recent call last):
  File "test.py", line 3, in <module>
    xrenner = Xrenner()
  File "/home/prangen/.local/lib/python3.8/site-packages/xrenner/modules/xrenner_xrenner.py", line 37, in __init__
    self.load(model, override)
  File "/home/prangen/.local/lib/python3.8/site-packages/xrenner/modules/xrenner_xrenner.py", line 57, in load
    self.lex = LexData(self.model, self, self.override, self.rule_based, self.no_seq)
  File "/home/prangen/.local/lib/python3.8/site-packages/xrenner/modules/xrenner_lex.py", line 110, in __init__
    self.speaker_rules, self.non_speaker_rules = self.parse_coref_rules(self.model_files['coref_rules.tab'].read().replace("\r","").split("\n"))
  File "/home/prangen/.local/lib/python3.8/site-packages/xrenner/modules/xrenner_lex.py", line 558, in parse_coref_rules
    clf = load(self.model_files[rule_file_name])
  File "/home/prangen/.local/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 575, in load
    obj = _unpickle(fobj)
  File "/home/prangen/.local/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 504, in _unpickle
    obj = unpickler.load()
  File "/usr/lib/python3.8/pickle.py", line 1210, in load
    dispatch[key[0]](self)
  File "/home/prangen/.local/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 329, in load_build
    Unpickler.load_build(self)
  File "/usr/lib/python3.8/pickle.py", line 1703, in load_build
    setstate(state)
  File "/home/prangen/.local/lib/python3.8/site-packages/xgboost/core.py", line 1211, in __setstate__
    _check_call(
  File "/home/prangen/.local/lib/python3.8/site-packages/xgboost/core.py", line 190, in _check_call
    raise XGBoostError(py_str(_LIB.XGBGetLastError()))
xgboost.core.XGBoostError: [10:06:05] /workspace/src/learner.cc:854: Check failed: header == serialisation_header_: 

  If you are loading a serialized model (like pickle in Python) generated by older
  XGBoost, please export the model by calling `Booster.save_model` from that version
  first, then load it back in current version.  There's a simple script for helping
  the process. See:

    https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html

  for reference to the script, and more details about differences between saving model and
  serializing.


Stack trace:
  [bt] (0) /home/prangen/.local/lib/python3.8/site-packages/xgboost/lib/libxgboost.so(+0xa0c64) [0x7ff515698c64]
  [bt] (1) /home/prangen/.local/lib/python3.8/site-packages/xgboost/lib/libxgboost.so(+0x1b2e81) [0x7ff5157aae81]
  [bt] (2) /home/prangen/.local/lib/python3.8/site-packages/xgboost/lib/libxgboost.so(XGBoosterUnserializeFromBuffer+0x4a) [0x7ff51568760a]
  [bt] (3) /lib/x86_64-linux-gnu/libffi.so.7(+0x6ff5) [0x7ff555fcdff5]
  [bt] (4) /lib/x86_64-linux-gnu/libffi.so.7(+0x640a) [0x7ff555fcd40a]
  [bt] (5) /usr/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so(_ctypes_callproc+0x57c) [0x7ff555fe568c]
  [bt] (6) /usr/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so(+0x12bbc) [0x7ff555fe5bbc]
  [bt] (7) python3(_PyObject_MakeTpCall+0x296) [0x5f2406]
  [bt] (8) python3(_PyEval_EvalFrameDefault+0x5b83) [0x56cb23]

My XGBoost and Xrenner versions:

$ pip3 list | grep "xgboost"
xgboost                1.1.1         
$ pip3 list | grep "xrenner"
xrenner                2.1.2.0      

Thanks for open-sourcing this tool!

[Amir Zeldes]

Hm, yes, this is not really a limitation of the software, but the pre-trained model is from xgboost==0.81, maybe that should be added to the requirements. The quickest fix for you is to use 0.81, since I think they introduced some breaking changes in xgboost V1.0.

Leaving this open as a reminder to update the model.

[Natalie Prange]

Thanks for the reply! Yes, maybe you can add it to the requirements if this is picked up by pip as well. That way a user does not need to manually fix the xgboost version.

[Amir Zeldes]

Probably better to update the model since the system doesn't actually depend on V0.81 in any special way