Giters

abg / dbsake

Collection of mysql tools

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Uncaught exception `ValueError: 245 is not a valid MySQLType` when using `dbsake frmdump`

golflima opened this issue · comments

commented

I'm using your excellent tool to extract the schema data from the .frm files of a database to recreate it on another server (the plan is to have a disaster recovery procedure in case every other backup options failed, and to copy prod data to a BI server with Percona XtraDB Backup).
It works very well for most tables, on few I have the following error and I cant't get the schema for them:

# dbsake frmdump company_user_csv_import_configuration.frm
Uncaught exception!Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/local/bin/dbsake/dbsake/cli/__init__.py", line 106, in handle_uncaught_exception
UnicodeDecodeError: 'ascii' codec can't decode byte 0xb0 in position 8: ordinal not in range(128)

Original exception was:
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/dbsake/__main__.py", line 22, in <module>
  File "/usr/local/bin/dbsake/__main__.py", line 18, in main
  File "/usr/local/bin/dbsake/dbsake/cli/__init__.py", line 123, in main
  File "/usr/local/bin/dbsake/click/core.py", line 722, in __call__
  File "/usr/local/bin/dbsake/click/core.py", line 697, in main
  File "/usr/local/bin/dbsake/click/core.py", line 1066, in invoke
  File "/usr/local/bin/dbsake/click/core.py", line 895, in invoke
  File "/usr/local/bin/dbsake/click/core.py", line 535, in invoke
  File "/usr/local/bin/dbsake/click/decorators.py", line 17, in new_func
  File "/usr/local/bin/dbsake/dbsake/cli/cmd/frm.py", line 71, in frmdump
  File "/usr/local/bin/dbsake/dbsake/cli/cmd/frm.py", line 26, in parse_and_print
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/__init__.py", line 41, in parse
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/binaryfrm.py", line 402, in parse
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/binaryfrm.py", line 287, in unpack_columns
  File "/usr/local/bin/dbsake/dbsake/util/enum.py", line 295, in __call__
  File "/usr/local/bin/dbsake/dbsake/util/enum.py", line 575, in __new__
ValueError: 245 is not a valid MySQLType

By default, we are (still) using Python 2.7.16, so I tried with Python 3.7.3 too:

# python3.7 /usr/local/bin/dbsake frmdump company_user_csv_import_configuration.frm
Uncaught exception! (╯°□°)╯ ︵ ┻━┻
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/bin/dbsake/__main__.py", line 22, in <module>
    sys.exit(main())
  File "/usr/local/bin/dbsake/__main__.py", line 18, in main
    sys.exit(dbsake.cli.main())
  File "/usr/local/bin/dbsake/dbsake/cli/__init__.py", line 123, in main
    dbsake(args=argv, auto_envvar_prefix='DBSAKE', obj={})
  File "/usr/local/bin/dbsake/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/bin/dbsake/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/bin/dbsake/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/bin/dbsake/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/bin/dbsake/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/bin/dbsake/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/bin/dbsake/dbsake/cli/cmd/frm.py", line 71, in frmdump
    failures += parse_and_print(name, type_codes, replace)
  File "/usr/local/bin/dbsake/dbsake/cli/cmd/frm.py", line 26, in parse_and_print
    table = frm.parse(frm_path)
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/__init__.py", line 41, in parse
    return dispatch(path)
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/binaryfrm.py", line 402, in parse
    columns = list(unpack_columns(packed_frm_data.columns, table))
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/binaryfrm.py", line 287, in unpack_columns
    type_code=constants.MySQLType(metadata.uint8_at(13, os.SEEK_CUR))
  File "/usr/local/bin/dbsake/dbsake/util/enum.py", line 295, in __call__
    return cls.__new__(cls, value)
  File "/usr/local/bin/dbsake/dbsake/util/enum.py", line 575, in __new__
    raise ValueError("%s is not a valid %s" % (value, cls.__name__))
ValueError: 245 is not a valid MySQLType
It's okay. ┬─┬ノ( º_ ºノ)
Consider filing a bug report at https://github.com/abg/dbsake/issues

OS is: Linux 4.19.0-8-cloud-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux.
dbsake version is: dbsake, version 2.1.2 (git: 50de953)

For information, DBeaver generate this query to recreate the table:

CREATE TABLE `company_user_csv_import_configuration` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `company_id` int(11) DEFAULT NULL,
  `directory` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
  `file_pattern` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
  `report_emails` json DEFAULT NULL,
  `error_emails` json DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `UNIQ_27AB5994979B1AD6` (`company_id`),
  CONSTRAINT `FK_27AB5994979B1AD6` FOREIGN KEY (`company_id`) REFERENCES `companies` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

And DESCRIBE returns this:

DESCRIBE company_user_csv_import_configuration;

Field        |Type        |Null|Key|Default|Extra         |
-------------|------------|----|---|-------|--------------|
id           |int(11)     |NO  |PRI|       |auto_increment|
company_id   |int(11)     |YES |UNI|       |              |
directory    |varchar(255)|NO  |   |       |              |
file_pattern |varchar(255)|NO  |   |       |              |
report_emails|json        |YES |   |       |              |
error_emails |json        |YES |   |       |              |
commented

Ok, I just had a look into the other 2 tables having the same issue, they all use the MySQL type json, which is not used on the other tables. So maybe dbsake doesn't support yet this json type?

commented

Indeed, just checked your sources: https://github.com/abg/dbsake/blob/master/dbsake/core/mysql/frm/constants.py#L152-L182
Perhaps you should add something like: JSON = 245 ?