miurahr / py7zr

7zip in python3 with ZStandard, PPMd, LZMA2, LZMA1, Delta, BCJ, BZip2, and Deflate compressions, and AES encryption.

Home Page:https://pypi.org/project/py7zr/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

AttributeError: module 'resource' has no attribute 'RLIMIT_AS' on OpenBSD

neonknight opened this issue · comments

Describe the bug
While trying to extract a file from 7z archive an exception "AttributeError: module 'resource' has no attribute 'RLIMIT_AS'" is raised.

Full traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/7zrvenv/lib/python3.8/site-packages/py7zr/py7zr.py", line 967, in read
    return self._extract(path=None, targets=targets, return_dict=True)
  File "/tmp/7zrvenv/lib/python3.8/site-packages/py7zr/py7zr.py", line 624, in _extract
    self.worker.extract(
  File "/tmp/7zrvenv/lib/python3.8/site-packages/py7zr/py7zr.py", line 1204, in extract
    self.extract_single(
  File "/tmp/7zrvenv/lib/python3.8/site-packages/py7zr/py7zr.py", line 1292, in extract_single
    raise e
  File "/tmp/7zrvenv/lib/python3.8/site-packages/py7zr/py7zr.py", line 1289, in extract_single
    self._extract_single(fp, files, path, src_end, q, skip_notarget)
  File "/tmp/7zrvenv/lib/python3.8/site-packages/py7zr/py7zr.py", line 1326, in _extract_single
    self._check(fp, just_check, src_end)
  File "/tmp/7zrvenv/lib/python3.8/site-packages/py7zr/py7zr.py", line 1381, in _check
    crc32 = self.decompress(fp, f.folder, ofp, f.uncompressed, f.compressed, src_end)
  File "/tmp/7zrvenv/lib/python3.8/site-packages/py7zr/py7zr.py", line 1409, in decompress
    max_block_size = get_memory_limit()
  File "/tmp/7zrvenv/lib/python3.8/site-packages/py7zr/properties.py", line 100, in get_memory_limit
    soft, _ = resource.getrlimit(resource.RLIMIT_AS)
AttributeError: module 'resource' has no attribute 'RLIMIT_AS'

To Reproduce

import py7zr
from io import BytesIO
archive = py7zr.SevenZipFile(BytesIO(open('/tmp/2.7z','rb').read()))
archive.read('some/file/in/some/dir/filename.txt')

After a while of calculating the exception mentioned above is raised. The archive is 6MB in size, the file is 51 bytes uncompressed. The issue occurs with any file of any size within the archive.

Expected behavior
Either file gets unpacked or a clear error is returned. In any case available attributes of module resource should be checked before accessing them, as they may differ depending on the underlying platform/OS.

resource on OpenBSD:

import resource
>>> dir(resource)
['RLIMIT_CORE', 'RLIMIT_CPU', 'RLIMIT_DATA', 'RLIMIT_FSIZE', 'RLIMIT_MEMLOCK', 'RLIMIT_NOFILE', 'RLIMIT_NPROC', 'RLIMIT_RSS', 'RLIMIT_STACK', 'RLIM_INFINITY', 'RUSAGE_CHILDREN', 'RUSAGE_SELF', 'RUSAGE_THREAD', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'error', 'getpagesize', 'getrlimit', 'getrusage', 'setrlimit', 'struct_rusage']

Environment (please complete the following information):

  • OS: OpenBSD 7.0
  • Python 3.8.12
  • py7zr version: 0.20.5

Test data(please attach in the report):
Sadly affected archive contains confidential customer data. However, it contains 31137 files of total 460MB uncompressed size in a large directory tree with many subdirectories. The resulting archive is only 6MB.

@neonknight Could you tell me how to detect OpenBSD?
Does sys.platform.startswith("openbsd") detect OpenBSD?

Does #522 work for you?

@neonknight Could you tell me how to detect OpenBSD? Does sys.platform.startswith("openbsd") detect OpenBSD?

yes, it does.

Does #522 work for you?

yes, get_memory_limit now works without raising any exception. Thank you for providing the fix!