Build and run on CPU

[kovmir]

My CPU: AMD Ryzen 5800X
My GPU: AMD Radeon RX 6600 XT

My exact build commands:

git clone https://github.com/hpcaitech/Open-Sora.git
cd Open-Sora
git switch opensora/v1.3
podman build -t opensora .

The container build aborts after STEP 6/11: RUN pip install flash-attn --no-build-isolation:

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [291 lines of output]
      No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'

...

      /opt/conda/envs/pytorch/lib/python3.10/site-packages/torch/utils/cpp_extension.py:424: UserWarning: There are no g++ version bounds defined for CUDA version 12.1
        warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')

1. Can I build and run this container without Nvidia GPU? How?
2. Can I run this on CPU? How?

[kovmir]

RUN pip install flash-attn==2.5.8 --no-build-isolation hardcoded version fixes the build errors.

[kovmir]

The python script fails at run-time with Nvidia GPU:

$ python scripts/inference.py configs/opensora-v1-3/inference/t2v.py \  --num-frames 97 --resolution 720p --aspect-ratio 9:16 \  --prompt "a beautiful waterfall"
/opt/conda/envs/pytorch/lib/python3.10/site-packages/mmengine/optim/optimizer/zero_optimizer.py:11: DeprecationWarning: `TorchScript` support for functional optimizers is deprecated and will be removed in a future PyTorch release. Consider using the `torch.compile` optimizer instead.
  from torch.distributed.optim import \
/opt/conda/envs/pytorch/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/opt/conda/envs/pytorch/lib/python3.10/site-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
Traceback (most recent call last):
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1967, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/opt/conda/envs/pytorch/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/models/bloom/modeling_bloom.py", line 38, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/modeling_utils.py", line 62, in <module>
    from .integrations.flash_attention import flash_attention_forward
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/integrations/flash_attention.py", line 5, in <module>
    from ..modeling_flash_attention_utils import _flash_attention_forward, flash_attn_supports_top_left_mask
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/modeling_flash_attention_utils.py", line 36, in <module>
    from flash_attn.bert_padding import index_first_axis, pad_input, unpad_input  # noqa
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/flash_attn/__init__.py", line 3, in <module>
    from flash_attn.flash_attn_interface import (
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 10, in <module>
    import flash_attn_2_cuda as flash_attn_cuda
ImportError: /opt/conda/envs/pytorch/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/Open-Sora/scripts/inference.py", line 15, in <module>
    from opensora.models.text_encoder.t5 import text_preprocessing
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/opensora/models/__init__.py", line 1, in <module>
    from .vae_v1_3 import *
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/opensora/models/vae_v1_3/__init__.py", line 2, in <module>
    from .discriminator import N_LAYER_DISCRIMINATOR_3D
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/opensora/models/vae_v1_3/discriminator.py", line 7, in <module>
    from opensora.utils.ckpt_utils import load_checkpoint
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/opensora/utils/ckpt_utils.py", line 13, in <module>
    from colossalai.booster import Booster
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/colossalai/booster/__init__.py", line 2, in <module>
    from .booster import Booster
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/colossalai/booster/booster.py", line 14, in <module>
    import peft
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/peft/__init__.py", line 22, in <module>
    from .auto import (
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/peft/auto.py", line 31, in <module>
    from .config import PeftConfig
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/peft/config.py", line 24, in <module>
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/peft/utils/__init__.py", line 23, in <module>
    from .other import (
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/peft/utils/other.py", line 33, in <module>
    from .constants import (
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/peft/utils/constants.py", line 16, in <module>
    from transformers import BloomPreTrainedModel
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1956, in __getattr__
    value = getattr(module, name)
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1955, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1969, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.bloom.modeling_bloom because of the following error (look up to see its traceback):
/opt/conda/envs/pytorch/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE

apt-get install libpng-dev in Dockerfile does not help.

[github-actions]

This issue is stale because it has been open for 7 days with no activity.