Facing an issue with mmcv for object detection model (ModuleNotFoundError: No module named 'mmcv._ext')

[AliGhadiri]

Describe the bug
I am following the instructions here to install otx and train an object detection model with it.
After installing the package just as mentioned in the above link, I am getting this error when I try to set up an object detection workspace running the following command:

otx build --workspace "C:\aervision\OV_training\HHDetection_workspace" ^
--model YOLOX-L --task DETECTION --backbone torchvision.resnet50 ^
--train-data-roots "C:\aervision\OV_training\HardHat_dataset" ^
--val-data-roots "C:\aervision\OV_training\HardHat_dataset"

The output with the error at the end:

[*] Workspace Path: C:\aervision\OV_training\HHDetection_workspace
[*] Load Model Template ID: Object_Detection_YOLOX_L
[*] Load Model Name: YOLOX-L
No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7'
C:\aervision\OV_training\training_extensions\venv\otx\lib\site-packages\mmcv\__init__.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
  warnings.warn(
C:\aervision\OV_training\training_extensions\venv\otx\lib\site-packages\mmcv\cnn\bricks\transformer.py:33: UserWarning: Fail to import ``MultiScaleDeformableAttention`` from ``mmcv.ops.multi_scale_deform_attn``, You should install ``mmcv-full`` if you need this module.
  warnings.warn('Fail to import ``MultiScaleDeformableAttention`` from '
Traceback (most recent call last):
  File "C:\aervision\OV_training\training_extensions\src\otx\cli\manager\config_manager.py", line 674, in _copy_config_files
    from otx.algorithms.common.adapters.mmcv.utils.config_utils import (
  File "C:\aervision\OV_training\training_extensions\src\otx\algorithms\common\adapters\mmcv\__init__.py", line 39, in <module>
    from .ops import multi_scale_deformable_attn_pytorch
  File "C:\aervision\OV_training\training_extensions\src\otx\algorithms\common\adapters\mmcv\ops\__init__.py", line 6, in <module>
    from .multi_scale_deformable_attn_pytorch import multi_scale_deformable_attn_pytorch
  File "C:\aervision\OV_training\training_extensions\src\otx\algorithms\common\adapters\mmcv\ops\multi_scale_deformable_attn_pytorch.py", line 9, in <module>
    from mmcv.ops import multi_scale_deform_attn
  File "C:\aervision\OV_training\training_extensions\venv\otx\lib\site-packages\mmcv\ops\__init__.py", line 2, in <module>
    from .active_rotated_filter import active_rotated_filter
  File "C:\aervision\OV_training\training_extensions\venv\otx\lib\site-packages\mmcv\ops\active_rotated_filter.py", line 10, in <module>
    ext_module = ext_loader.load_ext(
  File "C:\aervision\OV_training\training_extensions\venv\otx\lib\site-packages\mmcv\utils\ext_loader.py", line 13, in load_ext
    ext = importlib.import_module('mmcv.' + name)
  File "C:\Users\frd\AppData\Local\Programs\Python\Python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'mmcv._ext'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\frd\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\frd\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\aervision\OV_training\training_extensions\venv\otx\Scripts\otx.exe\__main__.py", line 7, in <module>
  File "C:\aervision\OV_training\training_extensions\src\otx\cli\tools\cli.py", line 77, in main
    results = globals()[f"otx_{name}"]()
  File "C:\aervision\OV_training\training_extensions\src\otx\cli\tools\build.py", line 110, in main
    config_manager.build_workspace(new_workspace_path=args.workspace)
  File "C:\aervision\OV_training\training_extensions\src\otx\cli\manager\config_manager.py", line 636, in build_workspace
    self._copy_config_files(target_dir, file_name, dest_dir)
  File "C:\aervision\OV_training\training_extensions\src\otx\cli\manager\config_manager.py", line 682, in _copy_config_files
    raise CliException(f"{self.task_type} requires mmcv-full to be installed.") from exc
otx.cli.utils.errors.CliException: DETECTION requires mmcv-full to be installed.

I've tried both of these methods to create the virtual environment according to the link above, yet both yield the same error I described.

1)
# Create virtual env.
python -m venv .otx

# Activate virtual env.
source .otx/bin/activate

2)
pip install tox
# -- need to replace '310' below if another python version needed
tox devenv venv/otx -e tests-all-py310
source venv/otx/bin/activate

I also doubled-checked the successul installation of mmcv-full as the pip list showes, it's been installed correctly:

mmaction2                 0.24.1
mmcls                     0.25.0
mmcv-full                 1.7.0
mmdeploy                  0.14.0
mmdet                     2.28.1
mmengine                  0.7.4
mmsegmentation            0.30.0

Steps to Reproduce

1. set up the environment according to this documentation

2.Install PyTorch
pip install torch==1.13.1 torchvision==0.14.1 --extra-index-url https://download.pytorch.org/whl/cu117

3.Install OpenVINO™ Training Extensions package
pip install -e .[full]

4. Run the otx build command with your own dataset and paths:

otx build --workspace "C:\aervision\OV_training\HHDetection_workspace" ^
--model YOLOX-L --task DETECTION --backbone torchvision.resnet50 ^
--train-data-roots "C:\aervision\OV_training\HardHat_dataset" ^
--val-data-roots "C:\aervision\OV_training\HardHat_dataset"

Environment:

• OS:
• Framework version:
• Python version: 3.10
• OpenVINO version:
• CUDA/cuDNN version: 11.7
• GPU model and memory: Nvidia TITAN Xp

[Harim Kang]

As far as I know, that error message means that MMCV is not installed properly. It's probably related to your environment being CUDA related.
Maybe below works:

pip uninstall mmcv-full
pip install -U openmim
mim install mmcv-full==1.7.0

[Yunchu Lee]

closing this issue because there is no interactions for 4 months. please reopen it if there is same issue again. thanks.