piskvorky / smart_open

Utils for streaming large files (S3, HDFS, gzip, bz2...)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Version 7.0.0 issue - import botocore error

albertferras-vrf opened this issue · comments

Problem description

Hello, we're using jsf (https://github.com/ghandic/jsf) library, which for some reason uses smart_open.

On our previous builds (until 1h ago when 7.0.0 was released), jsf was taking smart-open==6.4.0 and all was working fine. However, now pip decides to install jsf with the latest smart-open version (7.0.0).

The version 7.0.0 seems to have a hard dependency on botocore because when you import smart_open it fails if you don't have this installed. Should this now be a dependency that should come in this package requirements OR should it not fail if botocore isn't installed?

Steps/code to reproduce the problem

  • New virtualenv
  • pip install smart_open==7.0.0
  • python -c 'import smart_open'

If you install 6.4.0 instead it just works, without having botocore installed.

# Traceback from `import jsf` but the same happens with normal `import smart_open`
/usr/local/lib/python3.11/site-packages/jsf/__init__.py:1: in <module>
    from jsf.parser import JSF
/usr/local/lib/python3.11/site-packages/jsf/parser.py:13: in <module>
    from smart_open import open as s_open
/usr/local/lib/python3.11/site-packages/smart_open/__init__.py:34: in <module>
    from .smart_open_lib import open, parse_uri, smart_open, register_compressor  # noqa: E402
/usr/local/lib/python3.11/site-packages/smart_open/smart_open_lib.py:35: in <module>
    from smart_open import doctools
/usr/local/lib/python3.11/site-packages/smart_open/doctools.py:21: in <module>
    from . import transport
/usr/local/lib/python3.11/site-packages/smart_open/transport.py:104: in <module>
    register_transport("smart_open.s3")
/usr/local/lib/python3.11/site-packages/smart_open/transport.py:49: in register_transport
    submodule = importlib.import_module(submodule)
/usr/local/lib/python3.11/site-packages/smart_open/s3.py:121: in <module>
    RETRY = Retry()
/usr/local/lib/python3.11/site-packages/smart_open/s3.py:86: in __init__
    self.exceptions: List[Exception] = [botocore.exceptions.EndpointConnectionError]
E   NameError: name 'botocore' is not defined

Versions

6.4.0 -> OK
7.0.0 -> Does not work without botocore installed

Checklist

Before you create the issue, please make sure you have:

  • Described the problem clearly
  • Provided a minimal reproducible example, including any required data
  • Provided the version numbers of the relevant software

OK, thank you for letting me know. I can see what the problem is, and will release a bugfix soon.

I've released 7.0.1, please try it.

Works for me 👍 Thanks for the quick fix!

Should this issue be closed?

Yes, thank you @browniebroke