operatorai / modelstore

🏬 modelstore is a Python library that allows you to version, export, and save a machine learning model to your filesystem or a cloud storage provider.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Cannot download latest model version

erosenthal-square opened this issue · comments

Very small bug: if you call model_store.download without a model_id (in order to get the latest model version), an error is raised since the logging code is erroneously trying to format a string as a number.

I was going to fix this myself. As part of the fix, I thought I'd add a test to capture this bug. But then I saw that there's no test for this function and testing seemed nontrivial, so now I'm lazily creating this issue 🙃

Stack trace:

$ model_store.download("some_dir", "my_domain", model_id=None)
--- Logging error ---
Traceback (most recent call last):
  File "/Users/erosenthal/.pyenv/versions/3.7.8/lib/python3.7/logging/__init__.py", line 1025, in emit
    msg = self.format(record)
  File "/Users/erosenthal/.pyenv/versions/3.7.8/lib/python3.7/logging/__init__.py", line 869, in format
    return fmt.format(record)
  File "/Users/erosenthal/.pyenv/versions/3.7.8/lib/python3.7/logging/__init__.py", line 608, in format
    record.message = record.getMessage()
  File "/Users/erosenthal/.pyenv/versions/3.7.8/lib/python3.7/logging/__init__.py", line 369, in getMessage
    msg = msg % self.args
TypeError: must be real number, not str
Call stack:
  File "/Users/erosenthal/.pyenv/versions/modelstore_tests/bin/ipython", line 8, in <module>
    sys.exit(start_ipython())
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/IPython/__init__.py", line 126, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/traitlets/config/application.py", line 846, in launch_instance
    app.start()
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/IPython/terminal/ipapp.py", line 356, in start
    self.shell.mainloop()
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/IPython/terminal/interactiveshell.py", line 563, in mainloop
    self.interact()
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/IPython/terminal/interactiveshell.py", line 554, in interact
    self.run_cell(code, store_history=True)
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2902, in run_cell
    raw_cell, store_history, silent, shell_futures)
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2947, in _run_cell
    return runner(coro)
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 68, in _pseudo_sync_runner
    coro.send(None)
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3173, in run_cell_async
    interactivity=interactivity, compiler=compiler, result=result)
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3364, in run_ast_nodes
    if (await self.run_code(code, result,  async_=asy)):
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3444, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-5-c8bdaf682d10>", line 1, in <module>
    model_store.download("some_dir", domain, model_id=None)
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/modelstore/model_store.py", line 145, in download
    archive_path = self.storage.download(local_path, domain, model_id)
  File "/Users/erosenthal/.pyenv/versions/3.7.8/envs/modelstore_tests/lib/python3.7/site-packages/modelstore/storage/blob_storage.py", line 121, in download
    logger.info("Latest model is: %f", model_meta["model"]["model_id"])
Message: 'Latest model is: %f'

Yikes! Yeah testing is not super trivial as I've got to mock the different storage layers in different parts. Will look into this one 👀

For this one, since it's just about the logger, I'm going to go with the quick fix and not extend the tests - but I have extended the examples (which I used to replicate the bug). I've just merged the PR above with both. Thanks for flagging @erosenthal-square!