uqfoundation / dill

serialize all of Python

Home Page:http://dill.rtfd.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

bytes length not a multiple of item size

tHappo opened this issue · comments

i process the .pkl by python 3.11 on windows and load it by python 3.8 on Linux

Cell In[8], line 318, in prepare_dataloaders(opt, device)
316 def prepare_dataloaders(opt, device):
317 batch_size = opt.batch_size
--> 318 data = pickle.load(open(opt.data_pkl, 'rb'))
320 opt.max_token_seq_len = data['settings'].max_len
321 opt.src_pad_idx = data['vocab']['src'].vocab.stoi[PAD_WORD]

File /usr/local/miniconda3/lib/python3.8/site-packages/dill/_dill.py:287, in load(file, ignore, **kwds)
281 def load(file, ignore=None, **kwds):
282 """
283 Unpickle an object from a file.
284
285 See :func:loads for keyword arguments.
286 """
--> 287 return Unpickler(file, ignore=ignore, **kwds).load()

File /usr/local/miniconda3/lib/python3.8/site-packages/dill/_dill.py:442, in Unpickler.load(self)
441 def load(self): #NOTE: if settings change, need to update attributes
--> 442 obj = StockUnpickler.load(self)
443 if type(obj).module == getattr(_main_module, 'name', 'main'):
444 if not self._ignore:
445 # point obj class to main

File /usr/local/miniconda3/lib/python3.8/site-packages/preshed/bloom.pyx:133, in preshed.bloom.unpickle_bloom()

File /usr/local/miniconda3/lib/python3.8/site-packages/preshed/bloom.pyx:52, in preshed.bloom.BloomFilter.from_bytes()

File /usr/local/miniconda3/lib/python3.8/site-packages/preshed/bloom.pyx:73, in preshed.bloom.bloom_from_bytes()

ValueError: bytes length not a multiple of item size