Issue with Installation - lmdb

[Hassan Naveed]

Instructions To Reproduce the Issue:

I installed it on OSX, and got install successful (although I did have to remove the torch-lighting version requirement from the requirements.txt)

However, the tests from pytest ./tests/ mention in https://mmf.sh/docs/ do not pass and get the same issue when I try to run a model.

what exact command I ran:

I ran the following command:
pytest ./tests/

and here is the error I received:

/opt/anaconda3/envs/dlproject/lib/python3.9/site-packages/lmdb/__pycache__/lmdb_cffi.c:268:14: fatal error: 'preload.h' file not found
    #include "preload.h"
             ^~~~~~~~~~~
1 error generated.
ImportError while loading conftest '/Users/hassan/Documents/OMSCS_2nd_Sem/CS7643 Deep Learning/Project/git_repo/mmf/tests/conftest.py'.
tests/__init__.py:2: in <module>
    from mmf.utils.patch import patch_transformers
mmf/__init__.py:8: in <module>
    from mmf import utils, common, modules, datasets, models
mmf/modules/__init__.py:3: in <module>
    import mmf.modules.metrics
mmf/modules/metrics.py:53: in <module>
    from mmf.datasets.processors.processors import EvalAIAnswerProcessor
mmf/datasets/__init__.py:8: in <module>
    from .mmf_dataset import MMFDataset
mmf/datasets/mmf_dataset.py:6: in <module>
    from mmf.datasets.databases.annotation_database import AnnotationDatabase
mmf/datasets/databases/__init__.py:5: in <module>
    from .features_database import FeaturesDatabase
mmf/datasets/databases/features_database.py:7: in <module>
    from mmf.datasets.databases.readers.feature_readers import FeatureReader
mmf/datasets/databases/readers/feature_readers.py:8: in <module>
    import lmdb
/opt/anaconda3/envs/dlproject/lib/python3.9/site-packages/lmdb/__init__.py:48: in <module>
    from lmdb.cffi import *
/opt/anaconda3/envs/dlproject/lib/python3.9/site-packages/lmdb/cffi.py:361: in <module>
    _lib = _ffi.verify(_CFFI_VERIFY,
/opt/anaconda3/envs/dlproject/lib/python3.9/site-packages/cffi/api.py:468: in verify
    lib = self.verifier.load_library()
/opt/anaconda3/envs/dlproject/lib/python3.9/site-packages/cffi/verifier.py:105: in load_library
    self._compile_module()
/opt/anaconda3/envs/dlproject/lib/python3.9/site-packages/cffi/verifier.py:202: in _compile_module
    outputfilename = ffiplatform.compile(tmpdir, self.get_extension())
/opt/anaconda3/envs/dlproject/lib/python3.9/site-packages/cffi/ffiplatform.py:22: in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
/opt/anaconda3/envs/dlproject/lib/python3.9/site-packages/cffi/ffiplatform.py:58: in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
E   cffi.VerificationError: CompileError: command '/usr/bin/clang' failed with exit code 1

Expected behavior:

I am confused why this happened, even when the install of mmf turned out to be successful. It looks like an issue with lmdb, which does seem to be installed. Here is what I see in conda list:
lmdb 0.98 pypi_0 pypi

Environment:

Provide your environment information using the following command:

python -m torch.utils.collect_env

Collecting environment information...
PyTorch version: 1.9.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.3.1 (x86_64)
GCC version: Could not collect
Clang version: 11.0.3 (clang-1103.0.32.62)
CMake version: version 3.22.0
Libc version: N/A

Python version: 3.9 (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.2
[pip3] pytorch-lightning==1.6.1
[pip3] torch==1.9.0
[pip3] torchaudio==0.9.0
[pip3] torchmetrics==0.8.0
[pip3] torchtext==0.5.0
[pip3] torchvision==0.10.0
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           hecd8cb5_637  
[conda] mkl-service               2.4.0            py39h9ed2024_0  
[conda] mkl_fft                   1.3.1            py39h4ab4a9b_0  
[conda] mkl_random                1.2.2            py39hb2f4e1b_0  
[conda] numpy                     1.21.2           py39h4b4dc7a_0  
[conda] numpy-base                1.21.2           py39he0bd621_0  
[conda] pytorch-lightning         1.6.1                    pypi_0    pypi
[conda] torch                     1.9.0                    pypi_0    pypi
[conda] torchaudio                0.9.0                    pypi_0    pypi
[conda] torchmetrics              0.8.0                    pypi_0    pypi
[conda] torchtext                 0.5.0                    pypi_0    pypi
[conda] torchvision               0.10.0                   pypi_0    pypi

[adampicwell]

I had this same issue and was able to resolve it by upgrading to 1.3.0 of lmdb.

[Hang Yan]

I had this same issue and was able to resolve it by upgrading to 1.3.0 of lmdb.

Thanks, fix my issue

[Takuma256]

I got the same error using lmdb 1.4.1