ModuleNotFoundError: No module named 'opensora.acceleration.shardformer'
BountyMage opened this issue · comments
I ran into this ModuleNotFoundError issue while trying to train, please help to check.
error log:
(opensora) root@9e1193352f5c:/data/workspace_zj/Open-Sora-main# torchrun --nnodes=1 --nproc_per_node=1 scripts/train.py configs/opensora/train/16x256x256.py --data-path /data/workspace_zj/Open-Sora-main/dataset/output.csv
/opt/conda/envs/opensora/lib/python3.10/site-packages/colossalai/pipeline/schedule/_utils.py:19: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_register_pytree_node(OrderedDict, _odict_flatten, _odict_unflatten)
/opt/conda/envs/opensora/lib/python3.10/site-packages/torch/utils/_pytree.py:254: UserWarning: <class 'collections.OrderedDict'> is already registered as pytree node. Overwriting the previous registration.
warnings.warn(
Config (path: configs/opensora/train/16x256x256.py): {'num_frames': 16, 'frame_interval': 3, 'image_size': (256, 256), 'root': None, 'data_path': '/data/workspace_zj/Open-Sora-main/dataset/output.csv', 'use_image_transform': False, 'num_workers': 4, 'dtype': 'bf16', 'grad_checkpoint': True, 'plugin': 'zero2', 'sp_size': 1, 'model': {'type': 'STDiT-XL/2', 'space_scale': 0.5, 'time_scale': 1.0, 'from_pretrained': 'PixArt-XL-2-512x512.pth', 'enable_flashattn': True, 'enable_layernorm_kernel': True}, 'vae': {'type': 'VideoAutoencoderKL', 'from_pretrained': 'stabilityai/sd-vae-ft-ema'}, 'text_encoder': {'type': 't5', 'from_pretrained': './pretrained_models/t5_ckpts', 'model_max_length': 120, 'shardformer': True}, 'scheduler': {'type': 'iddpm', 'timestep_respacing': ''}, 'seed': 42, 'outputs': 'outputs', 'wandb': False, 'epochs': 1000, 'log_every': 10, 'ckpt_every': 1000, 'load': None, 'batch_size': 8, 'lr': 2e-05, 'grad_clip': 1.0, 'multi_resolution': False}
/opt/conda/envs/opensora/lib/python3.10/site-packages/colossalai/initialize.py:48: UserWarning: config is deprecated and will be removed soon.
warnings.warn("config is deprecated and will be removed soon.")
[03/28/24 05:33:03] INFO colossalai - colossalai - INFO: /opt/conda/envs/opensora/lib/python3.10/site-packages/colossalai/initialize.py:67 launch
INFO colossalai - colossalai - INFO: Distributed environment is initialized, world size: 1
[2024-03-28 05:33:03] Experiment directory created at outputs/002-F16S3-STDiT-XL-2
[2024-03-28 05:33:03] Dataset contains 183 videos (/data/workspace_zj/Open-Sora-main/dataset/output.csv)
[2024-03-28 05:33:03] Total batch size: 8
./pretrained_models/t5_ckpts/t5-v1_1-xxl
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:19<00:00, 9.64s/it]
Traceback (most recent call last):
File "/data/workspace_zj/Open-Sora-main/scripts/train.py", line 287, in
main()
File "/data/workspace_zj/Open-Sora-main/scripts/train.py", line 132, in main
text_encoder = build_module(cfg.text_encoder, MODELS, device=device) # T5 must be fp32
File "/opt/conda/envs/opensora/lib/python3.10/site-packages/opensora/registry.py", line 22, in build_module
return builder.build(cfg)
File "/opt/conda/envs/opensora/lib/python3.10/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
File "/opt/conda/envs/opensora/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
File "/opt/conda/envs/opensora/lib/python3.10/site-packages/opensora/models/text_encoder/t5.py", line 326, in init
self.shardformer_t5()
File "/opt/conda/envs/opensora/lib/python3.10/site-packages/opensora/models/text_encoder/t5.py", line 331, in shardformer_t5
from opensora.acceleration.shardformer.policy.t5_encoder import T5EncoderPolicy
ModuleNotFoundError: No module named 'opensora.acceleration.shardformer'
[2024-03-28 05:33:38,682] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 2645088) of binary: /opt/conda/envs/opensora/bin/python
Traceback (most recent call last):
File "/opt/conda/envs/opensora/bin/torchrun", line 33, in
sys.exit(load_entry_point('torch==2.2.1', 'console_scripts', 'torchrun')())
File "/opt/conda/envs/opensora/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 347, in wrapper
return f(*args, **kwargs)
File "/opt/conda/envs/opensora/lib/python3.10/site-packages/torch/distributed/run.py", line 812, in main
run(args)
File "/opt/conda/envs/opensora/lib/python3.10/site-packages/torch/distributed/run.py", line 803, in run
elastic_launch(
File "/opt/conda/envs/opensora/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 135, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/opt/conda/envs/opensora/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 268, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
scripts/train.py FAILED
Failures:
<NO_OTHER_FAILURES>
Root Cause (first observed failure):
[0]:
time : 2024-03-28_05:33:38
host : 9e1193352f5c
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 2645088)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
opensora install log:
(opensora) root@9e1193352f5c:/data/workspace_zj/Open-Sora-main# pip install -v .
Using pip 23.3.1 from /opt/conda/envs/opensora/lib/python3.10/site-packages/pip (python 3.10)
Processing /data/workspace_zj/Open-Sora-main
Running command python setup.py egg_info
running egg_info
creating /tmp/pip-pip-egg-info-_p8hdr3v/opensora.egg-info
writing /tmp/pip-pip-egg-info-_p8hdr3v/opensora.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-pip-egg-info-_p8hdr3v/opensora.egg-info/dependency_links.txt
writing requirements to /tmp/pip-pip-egg-info-_p8hdr3v/opensora.egg-info/requires.txt
writing top-level names to /tmp/pip-pip-egg-info-_p8hdr3v/opensora.egg-info/top_level.txt
writing manifest file '/tmp/pip-pip-egg-info-_p8hdr3v/opensora.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-pip-egg-info-_p8hdr3v/opensora.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file '/tmp/pip-pip-egg-info-_p8hdr3v/opensora.egg-info/SOURCES.txt'
Preparing metadata (setup.py) ... done
Requirement already satisfied: colossalai in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (0.3.6)
Requirement already satisfied: accelerate in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (0.28.0)
Requirement already satisfied: diffusers in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (0.27.2)
Requirement already satisfied: ftfy in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (6.2.0)
Requirement already satisfied: gdown in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (5.1.0)
Requirement already satisfied: mmengine in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (0.10.3)
Requirement already satisfied: pre-commit in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (3.6.2)
Requirement already satisfied: pyav in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (12.0.5)
Requirement already satisfied: tensorboard in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (2.16.2)
Requirement already satisfied: timm in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (0.9.16)
Requirement already satisfied: tqdm in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (4.66.2)
Requirement already satisfied: transformers in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (4.39.0)
Requirement already satisfied: wandb in /opt/conda/envs/opensora/lib/python3.10/site-packages (from opensora==1.0.0) (0.16.4)
Requirement already satisfied: numpy>=1.17 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from accelerate->opensora==1.0.0) (1.26.4)
Requirement already satisfied: packaging>=20.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from accelerate->opensora==1.0.0) (24.0)
Requirement already satisfied: psutil in /opt/conda/envs/opensora/lib/python3.10/site-packages (from accelerate->opensora==1.0.0) (5.9.8)
Requirement already satisfied: pyyaml in /opt/conda/envs/opensora/lib/python3.10/site-packages (from accelerate->opensora==1.0.0) (6.0.1)
Requirement already satisfied: torch>=1.10.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from accelerate->opensora==1.0.0) (2.2.1)
Requirement already satisfied: huggingface-hub in /opt/conda/envs/opensora/lib/python3.10/site-packages (from accelerate->opensora==1.0.0) (0.21.4)
Requirement already satisfied: safetensors>=0.3.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from accelerate->opensora==1.0.0) (0.4.2)
Requirement already satisfied: rich in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (13.7.1)
Requirement already satisfied: click in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (8.1.7)
Requirement already satisfied: fabric in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (3.2.2)
Requirement already satisfied: contexttimer in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (0.3.3)
Requirement already satisfied: ninja in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (1.11.1.1)
Requirement already satisfied: einops in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (0.7.0)
Requirement already satisfied: pydantic in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (2.6.4)
Requirement already satisfied: ray in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (2.9.3)
Requirement already satisfied: sentencepiece in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (0.2.0)
Requirement already satisfied: google in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (3.0.0)
Requirement already satisfied: protobuf in /opt/conda/envs/opensora/lib/python3.10/site-packages (from colossalai->opensora==1.0.0) (4.25.3)
Requirement already satisfied: importlib-metadata in /opt/conda/envs/opensora/lib/python3.10/site-packages (from diffusers->opensora==1.0.0) (7.1.0)
Requirement already satisfied: filelock in /opt/conda/envs/opensora/lib/python3.10/site-packages (from diffusers->opensora==1.0.0) (3.13.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from diffusers->opensora==1.0.0) (2023.12.25)
Requirement already satisfied: requests in /opt/conda/envs/opensora/lib/python3.10/site-packages (from diffusers->opensora==1.0.0) (2.31.0)
Requirement already satisfied: Pillow in /opt/conda/envs/opensora/lib/python3.10/site-packages (from diffusers->opensora==1.0.0) (10.2.0)
Requirement already satisfied: wcwidth<0.3.0,>=0.2.12 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from ftfy->opensora==1.0.0) (0.2.13)
Requirement already satisfied: beautifulsoup4 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from gdown->opensora==1.0.0) (4.12.3)
Requirement already satisfied: addict in /opt/conda/envs/opensora/lib/python3.10/site-packages (from mmengine->opensora==1.0.0) (2.4.0)
Requirement already satisfied: matplotlib in /opt/conda/envs/opensora/lib/python3.10/site-packages (from mmengine->opensora==1.0.0) (3.8.3)
Requirement already satisfied: termcolor in /opt/conda/envs/opensora/lib/python3.10/site-packages (from mmengine->opensora==1.0.0) (2.4.0)
Requirement already satisfied: yapf in /opt/conda/envs/opensora/lib/python3.10/site-packages (from mmengine->opensora==1.0.0) (0.40.2)
Requirement already satisfied: opencv-python>=3 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from mmengine->opensora==1.0.0) (4.9.0.80)
Requirement already satisfied: cfgv>=2.0.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from pre-commit->opensora==1.0.0) (3.4.0)
Requirement already satisfied: identify>=1.0.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from pre-commit->opensora==1.0.0) (2.5.35)
Requirement already satisfied: nodeenv>=0.11.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from pre-commit->opensora==1.0.0) (1.8.0)
Requirement already satisfied: virtualenv>=20.10.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from pre-commit->opensora==1.0.0) (20.25.1)
Requirement already satisfied: absl-py>=0.4 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from tensorboard->opensora==1.0.0) (2.1.0)
Requirement already satisfied: grpcio>=1.48.2 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from tensorboard->opensora==1.0.0) (1.62.1)
Requirement already satisfied: markdown>=2.6.8 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from tensorboard->opensora==1.0.0) (3.6)
Requirement already satisfied: setuptools>=41.0.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from tensorboard->opensora==1.0.0) (68.2.2)
Requirement already satisfied: six>1.9 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from tensorboard->opensora==1.0.0) (1.16.0)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from tensorboard->opensora==1.0.0) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from tensorboard->opensora==1.0.0) (3.0.1)
Requirement already satisfied: torchvision in /opt/conda/envs/opensora/lib/python3.10/site-packages (from timm->opensora==1.0.0) (0.17.1)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from transformers->opensora==1.0.0) (0.15.2)
Requirement already satisfied: GitPython!=3.1.29,>=1.0.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from wandb->opensora==1.0.0) (3.1.42)
Requirement already satisfied: sentry-sdk>=1.0.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from wandb->opensora==1.0.0) (1.43.0)
Requirement already satisfied: docker-pycreds>=0.4.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from wandb->opensora==1.0.0) (0.4.0)
Requirement already satisfied: setproctitle in /opt/conda/envs/opensora/lib/python3.10/site-packages (from wandb->opensora==1.0.0) (1.3.3)
Requirement already satisfied: appdirs>=1.4.3 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from wandb->opensora==1.0.0) (1.4.4)
Requirement already satisfied: gitdb<5,>=4.0.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from GitPython!=3.1.29,>=1.0.0->wandb->opensora==1.0.0) (4.0.11)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from huggingface-hub->accelerate->opensora==1.0.0) (2024.3.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from huggingface-hub->accelerate->opensora==1.0.0) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from requests->diffusers->opensora==1.0.0) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from requests->diffusers->opensora==1.0.0) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from requests->diffusers->opensora==1.0.0) (2.1.0)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from requests->diffusers->opensora==1.0.0) (2024.2.2)
Requirement already satisfied: sympy in /opt/conda/envs/opensora/lib/python3.10/site-packages (from torch>=1.10.0->accelerate->opensora==1.0.0) (1.12)
Requirement already satisfied: networkx in /opt/conda/envs/opensora/lib/python3.10/site-packages (from torch>=1.10.0->accelerate->opensora==1.0.0) (3.1)
Requirement already satisfied: jinja2 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from torch>=1.10.0->accelerate->opensora==1.0.0) (3.1.3)
Requirement already satisfied: distlib<1,>=0.3.7 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from virtualenv>=20.10.0->pre-commit->opensora==1.0.0) (0.3.8)
Requirement already satisfied: platformdirs<5,>=3.9.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from virtualenv>=20.10.0->pre-commit->opensora==1.0.0) (4.2.0)
Requirement already satisfied: MarkupSafe>=2.1.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from werkzeug>=1.0.1->tensorboard->opensora==1.0.0) (2.1.3)
Requirement already satisfied: soupsieve>1.2 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from beautifulsoup4->gdown->opensora==1.0.0) (2.5)
Requirement already satisfied: invoke>=2.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from fabric->colossalai->opensora==1.0.0) (2.2.0)
Requirement already satisfied: paramiko>=2.4 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from fabric->colossalai->opensora==1.0.0) (3.4.0)
Requirement already satisfied: decorator>=5 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from fabric->colossalai->opensora==1.0.0) (5.1.1)
Requirement already satisfied: deprecated>=1.2 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from fabric->colossalai->opensora==1.0.0) (1.2.14)
Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from importlib-metadata->diffusers->opensora==1.0.0) (3.18.1)
Requirement already satisfied: contourpy>=1.0.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from matplotlib->mmengine->opensora==1.0.0) (1.2.0)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from matplotlib->mmengine->opensora==1.0.0) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from matplotlib->mmengine->opensora==1.0.0) (4.50.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from matplotlib->mmengine->opensora==1.0.0) (1.4.5)
Requirement already satisfied: pyparsing>=2.3.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from matplotlib->mmengine->opensora==1.0.0) (3.1.2)
Requirement already satisfied: python-dateutil>=2.7 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from matplotlib->mmengine->opensora==1.0.0) (2.9.0.post0)
Requirement already satisfied: annotated-types>=0.4.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from pydantic->colossalai->opensora==1.0.0) (0.6.0)
Requirement already satisfied: pydantic-core==2.16.3 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from pydantic->colossalai->opensora==1.0.0) (2.16.3)
Requirement already satisfied: jsonschema in /opt/conda/envs/opensora/lib/python3.10/site-packages (from ray->colossalai->opensora==1.0.0) (4.21.1)
Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from ray->colossalai->opensora==1.0.0) (1.0.8)
Requirement already satisfied: aiosignal in /opt/conda/envs/opensora/lib/python3.10/site-packages (from ray->colossalai->opensora==1.0.0) (1.3.1)
Requirement already satisfied: frozenlist in /opt/conda/envs/opensora/lib/python3.10/site-packages (from ray->colossalai->opensora==1.0.0) (1.4.1)
Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from requests[socks]->gdown->opensora==1.0.0) (1.7.1)
Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from rich->colossalai->opensora==1.0.0) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from rich->colossalai->opensora==1.0.0) (2.17.2)
Requirement already satisfied: tomli>=2.0.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from yapf->mmengine->opensora==1.0.0) (2.0.1)
Requirement already satisfied: wrapt<2,>=1.10 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from deprecated>=1.2->fabric->colossalai->opensora==1.0.0) (1.16.0)
Requirement already satisfied: smmap<6,>=3.0.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->opensora==1.0.0) (5.0.1)
Requirement already satisfied: mdurl~=0.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich->colossalai->opensora==1.0.0) (0.1.2)
Requirement already satisfied: bcrypt>=3.2 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from paramiko>=2.4->fabric->colossalai->opensora==1.0.0) (4.1.2)
Requirement already satisfied: cryptography>=3.3 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from paramiko>=2.4->fabric->colossalai->opensora==1.0.0) (42.0.5)
Requirement already satisfied: pynacl>=1.5 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from paramiko>=2.4->fabric->colossalai->opensora==1.0.0) (1.5.0)
Requirement already satisfied: attrs>=22.2.0 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from jsonschema->ray->colossalai->opensora==1.0.0) (23.2.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from jsonschema->ray->colossalai->opensora==1.0.0) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from jsonschema->ray->colossalai->opensora==1.0.0) (0.34.0)
Requirement already satisfied: rpds-py>=0.7.1 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from jsonschema->ray->colossalai->opensora==1.0.0) (0.18.0)
Requirement already satisfied: mpmath>=0.19 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from sympy->torch>=1.10.0->accelerate->opensora==1.0.0) (1.3.0)
Requirement already satisfied: cffi>=1.12 in /opt/conda/envs/opensora/lib/python3.10/site-packages (from cryptography>=3.3->paramiko>=2.4->fabric->colossalai->opensora==1.0.0) (1.16.0)
Requirement already satisfied: pycparser in /opt/conda/envs/opensora/lib/python3.10/site-packages (from cffi>=1.12->cryptography>=3.3->paramiko>=2.4->fabric->colossalai->opensora==1.0.0) (2.21)
Building wheels for collected packages: opensora
Running command python setup.py bdist_wheel
running bdist_wheel
running build
running build_py
/opt/conda/envs/opensora/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` directly.
Instead, use pypa/build, pypa/installer or other
standards-based tools.
See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
********************************************************************************
!!
self.initialize_options()
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/tools
creating build/bdist.linux-x86_64/wheel/tools/caption
copying build/lib/tools/caption/caption_llava.py -> build/bdist.linux-x86_64/wheel/tools/caption
copying build/lib/tools/caption/utils.py -> build/bdist.linux-x86_64/wheel/tools/caption
copying build/lib/tools/caption/init.py -> build/bdist.linux-x86_64/wheel/tools/caption
copying build/lib/tools/caption/caption_gpt4.py -> build/bdist.linux-x86_64/wheel/tools/caption
creating build/bdist.linux-x86_64/wheel/tools/datasets
copying build/lib/tools/datasets/convert_dataset.py -> build/bdist.linux-x86_64/wheel/tools/datasets
copying build/lib/tools/datasets/init.py -> build/bdist.linux-x86_64/wheel/tools/datasets
copying build/lib/tools/datasets/csvutil.py -> build/bdist.linux-x86_64/wheel/tools/datasets
creating build/bdist.linux-x86_64/wheel/opensora
creating build/bdist.linux-x86_64/wheel/opensora/utils
copying build/lib/opensora/utils/misc.py -> build/bdist.linux-x86_64/wheel/opensora/utils
copying build/lib/opensora/utils/train_utils.py -> build/bdist.linux-x86_64/wheel/opensora/utils
copying build/lib/opensora/utils/config_utils.py -> build/bdist.linux-x86_64/wheel/opensora/utils
copying build/lib/opensora/utils/ckpt_utils.py -> build/bdist.linux-x86_64/wheel/opensora/utils
copying build/lib/opensora/utils/init.py -> build/bdist.linux-x86_64/wheel/opensora/utils
creating build/bdist.linux-x86_64/wheel/opensora/schedulers
creating build/bdist.linux-x86_64/wheel/opensora/schedulers/iddpm
copying build/lib/opensora/schedulers/iddpm/respace.py -> build/bdist.linux-x86_64/wheel/opensora/schedulers/iddpm
copying build/lib/opensora/schedulers/iddpm/timestep_sampler.py -> build/bdist.linux-x86_64/wheel/opensora/schedulers/iddpm
copying build/lib/opensora/schedulers/iddpm/diffusion_utils.py -> build/bdist.linux-x86_64/wheel/opensora/schedulers/iddpm
copying build/lib/opensora/schedulers/iddpm/gaussian_diffusion.py -> build/bdist.linux-x86_64/wheel/opensora/schedulers/iddpm
copying build/lib/opensora/schedulers/iddpm/init.py -> build/bdist.linux-x86_64/wheel/opensora/schedulers/iddpm
copying build/lib/opensora/schedulers/init.py -> build/bdist.linux-x86_64/wheel/opensora/schedulers
creating build/bdist.linux-x86_64/wheel/opensora/schedulers/dpms
copying build/lib/opensora/schedulers/dpms/dpm_solver.py -> build/bdist.linux-x86_64/wheel/opensora/schedulers/dpms
copying build/lib/opensora/schedulers/dpms/init.py -> build/bdist.linux-x86_64/wheel/opensora/schedulers/dpms
creating build/bdist.linux-x86_64/wheel/opensora/acceleration
copying build/lib/opensora/acceleration/parallel_states.py -> build/bdist.linux-x86_64/wheel/opensora/acceleration
copying build/lib/opensora/acceleration/plugin.py -> build/bdist.linux-x86_64/wheel/opensora/acceleration
copying build/lib/opensora/acceleration/init.py -> build/bdist.linux-x86_64/wheel/opensora/acceleration
copying build/lib/opensora/acceleration/checkpoint.py -> build/bdist.linux-x86_64/wheel/opensora/acceleration
copying build/lib/opensora/acceleration/communications.py -> build/bdist.linux-x86_64/wheel/opensora/acceleration
creating build/bdist.linux-x86_64/wheel/opensora/datasets
copying build/lib/opensora/datasets/utils.py -> build/bdist.linux-x86_64/wheel/opensora/datasets
copying build/lib/opensora/datasets/datasets.py -> build/bdist.linux-x86_64/wheel/opensora/datasets
copying build/lib/opensora/datasets/init.py -> build/bdist.linux-x86_64/wheel/opensora/datasets
copying build/lib/opensora/datasets/video_transforms.py -> build/bdist.linux-x86_64/wheel/opensora/datasets
copying build/lib/opensora/registry.py -> build/bdist.linux-x86_64/wheel/opensora
copying build/lib/opensora/init.py -> build/bdist.linux-x86_64/wheel/opensora
creating build/bdist.linux-x86_64/wheel/opensora/models
creating build/bdist.linux-x86_64/wheel/opensora/models/dit
copying build/lib/opensora/models/dit/dit.py -> build/bdist.linux-x86_64/wheel/opensora/models/dit
copying build/lib/opensora/models/dit/init.py -> build/bdist.linux-x86_64/wheel/opensora/models/dit
creating build/bdist.linux-x86_64/wheel/opensora/models/latte
copying build/lib/opensora/models/latte/init.py -> build/bdist.linux-x86_64/wheel/opensora/models/latte
copying build/lib/opensora/models/latte/latte.py -> build/bdist.linux-x86_64/wheel/opensora/models/latte
creating build/bdist.linux-x86_64/wheel/opensora/models/text_encoder
copying build/lib/opensora/models/text_encoder/t5.py -> build/bdist.linux-x86_64/wheel/opensora/models/text_encoder
copying build/lib/opensora/models/text_encoder/clip.py -> build/bdist.linux-x86_64/wheel/opensora/models/text_encoder
copying build/lib/opensora/models/text_encoder/init.py -> build/bdist.linux-x86_64/wheel/opensora/models/text_encoder
copying build/lib/opensora/models/text_encoder/classes.py -> build/bdist.linux-x86_64/wheel/opensora/models/text_encoder
creating build/bdist.linux-x86_64/wheel/opensora/models/pixart
copying build/lib/opensora/models/pixart/init.py -> build/bdist.linux-x86_64/wheel/opensora/models/pixart
copying build/lib/opensora/models/pixart/pixart.py -> build/bdist.linux-x86_64/wheel/opensora/models/pixart
creating build/bdist.linux-x86_64/wheel/opensora/models/stdit
copying build/lib/opensora/models/stdit/stdit.py -> build/bdist.linux-x86_64/wheel/opensora/models/stdit
copying build/lib/opensora/models/stdit/init.py -> build/bdist.linux-x86_64/wheel/opensora/models/stdit
creating build/bdist.linux-x86_64/wheel/opensora/models/layers
copying build/lib/opensora/models/layers/init.py -> build/bdist.linux-x86_64/wheel/opensora/models/layers
copying build/lib/opensora/models/layers/blocks.py -> build/bdist.linux-x86_64/wheel/opensora/models/layers
copying build/lib/opensora/models/init.py -> build/bdist.linux-x86_64/wheel/opensora/models
creating build/bdist.linux-x86_64/wheel/opensora/models/vae
copying build/lib/opensora/models/vae/init.py -> build/bdist.linux-x86_64/wheel/opensora/models/vae
copying build/lib/opensora/models/vae/vae.py -> build/bdist.linux-x86_64/wheel/opensora/models/vae
running install_egg_info
running egg_info
writing opensora.egg-info/PKG-INFO
writing dependency_links to opensora.egg-info/dependency_links.txt
writing requirements to opensora.egg-info/requires.txt
writing top-level names to opensora.egg-info/top_level.txt
reading manifest file 'opensora.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'opensora.egg-info/SOURCES.txt'
Copying opensora.egg-info to build/bdist.linux-x86_64/wheel/opensora-1.0.0-py3.10.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/opensora-1.0.0.dist-info/WHEEL
creating '/tmp/pip-wheel-6g9x6zvw/opensora-1.0.0-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'opensora/init.py'
adding 'opensora/registry.py'
adding 'opensora/acceleration/init.py'
adding 'opensora/acceleration/checkpoint.py'
adding 'opensora/acceleration/communications.py'
adding 'opensora/acceleration/parallel_states.py'
adding 'opensora/acceleration/plugin.py'
adding 'opensora/datasets/init.py'
adding 'opensora/datasets/datasets.py'
adding 'opensora/datasets/utils.py'
adding 'opensora/datasets/video_transforms.py'
adding 'opensora/models/init.py'
adding 'opensora/models/dit/init.py'
adding 'opensora/models/dit/dit.py'
adding 'opensora/models/latte/init.py'
adding 'opensora/models/latte/latte.py'
adding 'opensora/models/layers/init.py'
adding 'opensora/models/layers/blocks.py'
adding 'opensora/models/pixart/init.py'
adding 'opensora/models/pixart/pixart.py'
adding 'opensora/models/stdit/init.py'
adding 'opensora/models/stdit/stdit.py'
adding 'opensora/models/text_encoder/init.py'
adding 'opensora/models/text_encoder/classes.py'
adding 'opensora/models/text_encoder/clip.py'
adding 'opensora/models/text_encoder/t5.py'
adding 'opensora/models/vae/init.py'
adding 'opensora/models/vae/vae.py'
adding 'opensora/schedulers/init.py'
adding 'opensora/schedulers/dpms/init.py'
adding 'opensora/schedulers/dpms/dpm_solver.py'
adding 'opensora/schedulers/iddpm/init.py'
adding 'opensora/schedulers/iddpm/diffusion_utils.py'
adding 'opensora/schedulers/iddpm/gaussian_diffusion.py'
adding 'opensora/schedulers/iddpm/respace.py'
adding 'opensora/schedulers/iddpm/timestep_sampler.py'
adding 'opensora/utils/init.py'
adding 'opensora/utils/ckpt_utils.py'
adding 'opensora/utils/config_utils.py'
adding 'opensora/utils/misc.py'
adding 'opensora/utils/train_utils.py'
adding 'tools/caption/init.py'
adding 'tools/caption/caption_gpt4.py'
adding 'tools/caption/caption_llava.py'
adding 'tools/caption/utils.py'
adding 'tools/datasets/init.py'
adding 'tools/datasets/convert_dataset.py'
adding 'tools/datasets/csvutil.py'
adding 'opensora-1.0.0.dist-info/LICENSE'
adding 'opensora-1.0.0.dist-info/METADATA'
adding 'opensora-1.0.0.dist-info/WHEEL'
adding 'opensora-1.0.0.dist-info/top_level.txt'
adding 'opensora-1.0.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Building wheel for opensora (setup.py) ... done
Created wheel for opensora: filename=opensora-1.0.0-py3-none-any.whl size=103285 sha256=69d652fc515c568703ee7908d711c0232f9a35a45aff6bf897c82c0fad6b7c75
Stored in directory: /root/.cache/pip/wheels/3d/a5/75/e5b3d324d2b2a2057cad933b4eb78180650040e18ca644d79b
Successfully built opensora
Installing collected packages: opensora
Attempting uninstall: opensora
Found existing installation: opensora 1.0.0
Uninstalling opensora-1.0.0:
Removing file or directory /opt/conda/envs/opensora/lib/python3.10/site-packages/opensora-1.0.0.dist-info/
Removing file or directory /opt/conda/envs/opensora/lib/python3.10/site-packages/opensora/
Removing file or directory /opt/conda/envs/opensora/lib/python3.10/site-packages/tools/caption/
Removing file or directory /opt/conda/envs/opensora/lib/python3.10/site-packages/tools/datasets/
Successfully uninstalled opensora-1.0.0
Successfully installed opensora-1.0.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root cause： no init.py file under shardformer folder.