hornc / ia-bulkmarc-importer

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

On unexpected error, output the item, file, and offset for retrying and debugging

hornc opened this issue · comments

On the unexpected error line, output the item, file, and offset for retrying and debugging

log output:

marc_scms/20220805_ADAM_MARC_records.mrc:104021904:1204: 200 -- {'authors': [{'key': '/authors/OL4281082A', 'name': 'Paul Henry Grummann', 'status': 'matched'}], 'success': True, 'edition': {'key': '/books/OL40254835M', 'status': 'created'}, 'work': {'key': '/works/OL17942155W', 'status': 'matched'}, 'next_record_offset': 104023108, 'next_record_length': 1295}
UNEXPECTED ERROR 500; [<class 'AssertionError'> at /api/import/ia] WRITTEN TO: error_45.html
104024403:5

For the above example, the record which caused the error is at offset 104023108 , i.e. next_record_offset of the last successful row.

104024403:5 is the next record to attempt after skipping a problem record.

In practice, these 500 errors are caused by something in the existing matched Open Library record, not the source MARC.

e.g.:

key DUP/works/OL1085128W-DUP does not start with '

but it would still be good to be clearer in the logs about which record triggered the problem.