bson.errors.InvalidBSON: objsize too large after update_one

[Vladimir Taratushka]

Getting this error when making any operation after editing database. Using lmdb. Guessed it was after wrong update_one, but not sure about. Anyway, adding original code of editing db:

    record = {'free': 12313232, 'path': '/media/mnt/'}
    col = getattr(db, 'storages')

    record = {**models['storage'], **record}
    if col.count_documents({'path': record['path']}) > 0:
        col.update_one({'path': record['path']}, {'$set': {**record}})
    else:
        col.insert_one(record)

Error text:

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/user/.local/lib/python3.8/site-packages/montydb/cursor.py", line 365, in next if len(self._data) or self._refresh(): File "/home/user/.local/lib/python3.8/site-packages/montydb/cursor.py", line 354, in _refresh self.__query() File "/home/user/.local/lib/python3.8/site-packages/montydb/cursor.py", line 311, in __query for doc in documents: File "/home/user/.local/lib/python3.8/site-packages/montydb/storage/lightning.py", line 253, in <genexpr> docs = (self._decode_doc(doc) for doc in self._conn.iter_docs()) File "/home/user/.local/lib/python3.8/site-packages/montydb/storage/__init__.py", line 227, in _decode_doc return bson.document_decode( File "/home/user/.local/lib/python3.8/site-packages/montydb/types/_bson.py", line 64, in document_decode return cls.BSON(doc).decode(codec_options) File "/home/user/.local/lib/python3.8/site-packages/bson/__init__.py", line 1258, in decode return decode(self, codec_options) File "/home/user/.local/lib/python3.8/site-packages/bson/__init__.py", line 970, in decode return _bson_to_dict(data, codec_options) bson.errors.InvalidBSON: objsize too large

That's how db looks like in plain:

$ cat db/storages.mdb @ @ ~60c9a9cf368c720edc2668a6{"path": "/mnt/hdd4", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a6"}}~60c9a9cf368c720edc2668a5{"path": "/boot/efi", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a5"}}y60c9a9cf368c720edc2668a4{"path": "/run", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a4"}}v60c9a9cf368c720edc2668a3{"path": "/", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a3"}} f*~s60c9a9cf368c720edc2668a3{"path": "/", "total": 9999, "used": 99, "free": 0, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a3"}}2~60c9a9cf368c720edc2668a6{"path": "/mnt/hdd4", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a6"}}~60c9a9cf368c720edc2668a5{"path": "/boot/efi", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a5"}}y60c9a9cf368c720edc2668a4{"path": "/run", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a4"} f*~sr60c9a9cf368c720edc2668a3{"path": "/", "total": 9999, "used": 99, "free": 0, "status": "busy", "_id": {"$oid": "60c9a9cf368c720edc2668a3"}}~60c9a9cf368c720edc2668a6{"path": "/mnt/hdd4", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a6"}}~60c9a9cf368c720edc2668a5{"path": "/boot/efi", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a5"}}y60c9a9cf368c720edc2668a4{"path": "/run", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a4"}}

Or this:

$ cat db/storages.mdb @ @ 0 documents 0 document 0 ̝φħ^sTb_id̝φħ^sTpath/media/user/ssd1totalusedfreestatusreadyr60c9a9cf368c720edc2668a3{"path": "/", "total": 9999, "used": 99, "free": 0, "status": "busy", "_id": {"$oid": "60c9a9cf368c720edc2668a3"}}~60c9a9cf368c720edc2668a6{"path": "/mnt/hdd4", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a6"}}~60c9a9cf368c720edc2668a5{"path": "/boot/efi", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a5"}}y60c9a9cf368c720edc2668a4{"path": "/run", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a4"}} 0 documents ̝φħ^sTb_id̝φħ^sTpath/media/user/ssd1totalusedfreestatusreadyr60c9a9cf368c720edc2668a3{"path": "/", "total": 9999, "used": 99, "free": 0, "status": "busy", "_id": {"$oid": "60c9a9cf368c720edc2668a3"}}~60c9a9cf368c720edc2668a6{"path": "/mnt/hdd4", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a6"}}~60c9a9cf368c720edc2668a5{"path": "/boot/efi", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a5"}}y60c9a9cf368c720edc2668a4{"path": "/run", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a4"} 0 0 ̝φħ^sTb_id̝φħ^sTpath/media/user/ssd1totalusedfreestatusreadyr60c9a9cf368c720edc2668a3{"path": "/", "total": 9999, "used": 99, "free": 0, "status": "busy", "_id": {"$oid": "60c9a9cf368c720edc2668a3"}}~60c9a9cf368c720edc2668a6{"path": "/mnt/hdd4", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a6"}}~60c9a9cf368c720edc2668a5{"path": "/boot/efi", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a5"}}y60c9a9cf368c720edc2668a4{"path": "/run", "total": 9999, "used": 99, "free": 9900, "status": "ready", "_id": {"$oid": "60c9a9cf368c720edc2668a4"}}

Does it stores changes after updating?

[David Lai]

Hi, @rewiaca
The example code you gave does not define what variable models is, and could not reproduce the error at my end.

record = {**models['storage'], **record}

Could you update the example code ?

[Vladimir Taratushka]

@davidlatwe Guess it's resolved with 2.3.9 update. Thanks for being reactive!