CUDA Stream Activated: False 库达流激活：假[Bug]:

Question

CUDA Stream Activated: False 库达流激活：假[Bug]:

kgdbtydx opened this issue 2 months ago · comments

Checklist

The issue exists after disabling all extensions
The issue exists on a clean installation of webui
The issue is caused by an extension, but I believe it is caused by a bug in the webui
The issue exists in the current version of the webui
The issue has not been reported before recently
The issue has been reported before but has not been fixed yet

What happened?

Total VRAM 6144 MB, total RAM 16298 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 2060 : native
Hint: your device supports --pin-shared-memory for potential speed improvements.
Hint: your device supports --cuda-malloc for potential speed improvements.
Hint: your device supports --cuda-stream for potential speed improvements.
VAE dtype: torch.float32
CUDA Stream Activated: False
Using pytorch cross attention
ControlNet preprocessor location: H:\forge\webui_forge_cu121_torch21\webui\models\ControlNetPreprocessor
Checkpoint realisticVisionV51_v51VAE.safetensors not found; loading fallback realisticVisionV60B1_v51HyperVAE.safetensors [f47e942ad4]
Loading weights [f47e942ad4] from H:\forge\webui_forge_cu121_torch21\webui\models\Stable-diffusion\realisticVisionV60B1_v51HyperVAE.safetensors
2024-06-07 10:47:25,426 - ControlNet - INFO - ControlNet UI callback registered.
model_type EPS
UNet ADM Dimension 0
Running on local URL: http://127.0.0.1:7860

Steps to reproduce the problem

Total VRAM 6144 MB, total RAM 16298 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 2060 : native
Hint: your device supports --pin-shared-memory for potential speed improvements.
Hint: your device supports --cuda-malloc for potential speed improvements.
Hint: your device supports --cuda-stream for potential speed improvements.
VAE dtype: torch.float32
CUDA Stream Activated: False
Using pytorch cross attention
ControlNet preprocessor location: H:\forge\webui_forge_cu121_torch21\webui\models\ControlNetPreprocessor
Checkpoint realisticVisionV51_v51VAE.safetensors not found; loading fallback realisticVisionV60B1_v51HyperVAE.safetensors [f47e942ad4]
Loading weights [f47e942ad4] from H:\forge\webui_forge_cu121_torch21\webui\models\Stable-diffusion\realisticVisionV60B1_v51HyperVAE.safetensors
2024-06-07 10:47:25,426 - ControlNet - INFO - ControlNet UI callback registered.
model_type EPS
UNet ADM Dimension 0
Running on local URL: http://127.0.0.1:7860

What should have happened?

Total VRAM 6144 MB, total RAM 16298 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 2060 : native
Hint: your device supports --pin-shared-memory for potential speed improvements.
Hint: your device supports --cuda-malloc for potential speed improvements.
Hint: your device supports --cuda-stream for potential speed improvements.
VAE dtype: torch.float32
CUDA Stream Activated: False
Using pytorch cross attention
ControlNet preprocessor location: H:\forge\webui_forge_cu121_torch21\webui\models\ControlNetPreprocessor
Checkpoint realisticVisionV51_v51VAE.safetensors not found; loading fallback realisticVisionV60B1_v51HyperVAE.safetensors [f47e942ad4]
Loading weights [f47e942ad4] from H:\forge\webui_forge_cu121_torch21\webui\models\Stable-diffusion\realisticVisionV60B1_v51HyperVAE.safetensors
2024-06-07 10:47:25,426 - ControlNet - INFO - ControlNet UI callback registered.
model_type EPS
UNet ADM Dimension 0
Running on local URL: http://127.0.0.1:7860

What browsers do you use to access the UI ?

Microsoft Edge

Sysinfo

{
"Platform": "Windows-10-10.0.22631-SP0",
"Python": "3.10.6",
"Version": "f0.0.17v1.8.0rc-latest-276-g29be1da7",
"Commit": "29be1da7cf2b5dccfc70fbdd33eb35c56a31ffb7",
"Script path": "H:\forge\webui_forge_cu121_torch21\webui",
"Data path": "H:\forge\webui_forge_cu121_torch21\webui",
"Extensions dir": "H:\forge\webui_forge_cu121_torch21\webui\extensions",
"Checksum": "0b2ded8aba8dc76f09443ff1468f267c3e3f2b8ba4e180ca587ba4975632268a",
"Commandline": [
"launch.py"
],
"Torch env info": {
"torch_version": "2.1.2+cu121",
"is_debug_build": "False",
"cuda_compiled_version": "12.1",
"gcc_version": null,
"clang_version": null,
"cmake_version": null,
"os": "Microsoft Windows 11 专业工作站版",
"libc_version": "N/A",
"python_version": "3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] (64-bit runtime)",
"python_platform": "Windows-10-10.0.22631-SP0",
"is_cuda_available": "True",
"cuda_runtime_version": "11.8.89\r",
"cuda_module_loading": "LAZY",
"nvidia_driver_version": "552.44",
"nvidia_gpu_models": "GPU 0: NVIDIA GeForce RTX 2060",
"cudnn_version": "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\cudnn_ops_train64_8.dll",
"pip_version": "pip3",
"pip_packages": [
"numpy==1.26.2",
"open-clip-torch==2.20.0",
"pytorch-lightning==1.9.4",
"torch==2.1.2+cu121",
"torchdiffeq==0.2.3",
"torchmetrics==1.3.0.post0",
"torchsde==0.2.6",
"torchvision==0.16.2+cu121"
],
"conda_packages": null,
"hip_compiled_version": "N/A",
"hip_runtime_version": "N/A",
"miopen_runtime_version": "N/A",
"caching_allocator_config": "",
"is_xnnpack_available": "True",
"cpu_info": [
"Architecture=9",
"CurrentClockSpeed=3696",
"DeviceID=CPU0",
"Family=205",
"L2CacheSize=1536",
"L2CacheSpeed=",
"Manufacturer=GenuineIntel",
"MaxClockSpeed=3696",
"Name=Intel(R) Core(TM) i5-9600KF CPU @ 3.70GHz",
"ProcessorType=3",
"Revision="
]
},
"Exceptions": [],
"CPU": {
"model": "Intel64 Family 6 Model 158 Stepping 12, GenuineIntel",
"count logical": 6,
"count physical": 6
},
"RAM": {
"total": "16GB",
"used": "9GB",
"free": "7GB"
},
"Extensions": [],
"Inactive extensions": [],
"Environment": {
"GRADIO_ANALYTICS_ENABLED": "False"
},
"Config": {
"ldsr_steps": 100,
"ldsr_cached": false,
"SCUNET_tile": 256,
"SCUNET_tile_overlap": 8,
"SWIN_tile": 192,
"SWIN_tile_overlap": 8,
"SWIN_torch_compile": false,
"control_net_detectedmap_dir": "detected_maps",
"control_net_models_path": "",
"control_net_modules_path": "",
"control_net_unit_count": 3,
"control_net_model_cache_size": 5,
"control_net_no_detectmap": false,
"control_net_detectmap_autosaving": false,
"control_net_allow_script_control": false,
"control_net_sync_field_args": true,
"controlnet_show_batch_images_in_ui": false,
"controlnet_increment_seed_during_batch": false,
"controlnet_disable_openpose_edit": false,
"controlnet_disable_photopea_edit": false,
"controlnet_photopea_warning": true,
"controlnet_input_thumbnail": true,
"sd_checkpoint_hash": "f47e942ad4c30d863ad7f53cb60145ffcd2118845dfa705ce8bd6b42e90c4a13",
"sd_model_checkpoint": "realisticVisionV51_v51VAE.safetensors"
},
"Startup": {
"total": 34.55004620552063,
"records": {
"initial startup": 0.04303550720214844,
"prepare environment/checks": 0.009553194046020508,
"prepare environment/git version info": 0.2873556613922119,
"prepare environment/torch GPU test": 7.733046531677246,
"prepare environment/clone repositores": 0.21545815467834473,
"prepare environment/run extensions installers": 0.010408401489257812,
"prepare environment/run extensions_builtin installers/canvas-zoom-and-pan": 0.0,
"prepare environment/run extensions_builtin installers/extra-options-section": 0.0,
"prepare environment/run extensions_builtin installers/forge_legacy_preprocessors": 4.431233167648315,
"prepare environment/run extensions_builtin installers/forge_preprocessor_inpaint": 0.0,
"prepare environment/run extensions_builtin installers/forge_preprocessor_marigold": 0.0,
"prepare environment/run extensions_builtin installers/forge_preprocessor_normalbae": 0.0,
"prepare environment/run extensions_builtin installers/forge_preprocessor_recolor": 0.0,
"prepare environment/run extensions_builtin installers/forge_preprocessor_reference": 0.0,
"prepare environment/run extensions_builtin installers/forge_preprocessor_revision": 0.0,
"prepare environment/run extensions_builtin installers/forge_preprocessor_tile": 0.0,
"prepare environment/run extensions_builtin installers/LDSR": 0.0,
"prepare environment/run extensions_builtin installers/Lora": 0.0,
"prepare environment/run extensions_builtin installers/mobile": 0.0,
"prepare environment/run extensions_builtin installers/prompt-bracket-checker": 0.0010232925415039062,
"prepare environment/run extensions_builtin installers/ScuNET": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_controlllite": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_controlnet": 3.171245813369751,
"prepare environment/run extensions_builtin installers/sd_forge_controlnet_example": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_dynamic_thresholding": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_fooocus_inpaint": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_freeu": 0.0010251998901367188,
"prepare environment/run extensions_builtin installers/sd_forge_hypertile": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_ipadapter": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_kohya_hrfix": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_latent_modifier": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_multidiffusion": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_neveroom": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_photomaker": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_sag": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_stylealign": 0.0,
"prepare environment/run extensions_builtin installers/sd_forge_svd": 0.0009701251983642578,
"prepare environment/run extensions_builtin installers/sd_forge_z123": 0.0,
"prepare environment/run extensions_builtin installers/soft-inpainting": 0.0,
"prepare environment/run extensions_builtin installers/SwinIR": 0.0,
"prepare environment/run extensions_builtin installers": 7.605497598648071,
"prepare environment": 16.15416979789734,
"launcher": 0.008207082748413086,
"import torch": 6.838769435882568,
"import gradio": 2.37813663482666,
"setup paths": 2.809051513671875,
"import ldm": 0.014148235321044922,
"import sgm": 0.0,
"initialize shared": 0.20048165321350098,
"other imports": 1.9917569160461426,
"opts onchange": 0.0,
"setup SD model": 0.0,
"setup codeformer": 0.0027005672454833984,
"setup gfpgan": 0.03402423858642578,
"set samplers": 0.0,
"list extensions": 0.00598597526550293,
"restore config state file": 0.0,
"list SD models": 0.0241243839263916,
"list localizations": 0.001722097396850586,
"load scripts/custom_code.py": 0.019344329833984375,
"load scripts/img2imgalt.py": 0.0012302398681640625,
"load scripts/loopback.py": 0.0012526512145996094,
"load scripts/outpainting_mk_2.py": 0.0012249946594238281,
"load scripts/poor_mans_outpainting.py": 0.0004062652587890625,
"load scripts/postprocessing_caption.py": 0.0013477802276611328,
"load scripts/postprocessing_codeformer.py": 0.0007927417755126953,
"load scripts/postprocessing_create_flipped_copies.py": 0.0007781982421875,
"load scripts/postprocessing_focal_crop.py": 0.01024174690246582,
"load scripts/postprocessing_gfpgan.py": 0.0006551742553710938,
"load scripts/postprocessing_split_oversized.py": 0.0008542537689208984,
"load scripts/postprocessing_upscale.py": 0.0013535022735595703,
"load scripts/processing_autosized_crop.py": 0.000827789306640625,
"load scripts/prompt_matrix.py": 0.0007784366607666016,
"load scripts/prompts_from_file.py": 0.0010755062103271484,
"load scripts/sd_upscale.py": 0.0,
"load scripts/xyz_grid.py": 0.0027680397033691406,
"load scripts/ldsr_model.py": 1.0670418739318848,
"load scripts/lora_script.py": 0.42612767219543457,
"load scripts/scunet_model.py": 0.07183957099914551,
"load scripts/swinir_model.py": 0.06600236892700195,
"load scripts/hotkey_config.py": 0.001714944839477539,
"load scripts/extra_options_section.py": 0.0004153251647949219,
"load scripts/legacy_preprocessors.py": 0.018901824951171875,
"load scripts/preprocessor_inpaint.py": 0.028310775756835938,
"load scripts/preprocessor_marigold.py": 0.21159911155700684,
"load scripts/preprocessor_normalbae.py": 0.00893402099609375,
"load scripts/preprocessor_recolor.py": 0.001308441162109375,
"load scripts/forge_reference.py": 0.0006542205810546875,
"load scripts/preprocessor_revision.py": 0.000637054443359375,
"load scripts/preprocessor_tile.py": 0.0006167888641357422,
"load scripts/forge_controllllite.py": 0.015354633331298828,
"load scripts/controlnet.py": 0.7611112594604492,
"load scripts/xyz_grid_support.py": 0.0011532306671142578,
"load scripts/sd_forge_controlnet_example.py": 0.0010852813720703125,
"load scripts/forge_dynamic_thresholding.py": 0.005865812301635742,
"load scripts/forge_fooocus_inpaint.py": 0.0013659000396728516,
"load scripts/forge_freeu.py": 0.006141185760498047,
"load scripts/forge_hypertile.py": 0.005309104919433594,
"load scripts/forge_ipadapter.py": 0.013051748275756836,
"load scripts/kohya_hrfix.py": 0.00599217414855957,
"load scripts/forge_latent_modifier.py": 0.012321233749389648,
"load scripts/forge_multidiffusion.py": 0.013607978820800781,
"load scripts/forge_never_oom.py": 0.0016160011291503906,
"load scripts/forge_photomaker.py": 0.005373239517211914,
"load scripts/forge_sag.py": 0.005951881408691406,
"load scripts/forge_stylealign.py": 0.0009291172027587891,
"load scripts/forge_svd.py": 0.08934760093688965,
"load scripts/forge_z123.py": 0.07997870445251465,
"load scripts/soft_inpainting.py": 0.0016083717346191406,
"load scripts/comments.py": 0.07356572151184082,
"load scripts/refiner.py": 0.0009694099426269531,
"load scripts/seed.py": 0.0011920928955078125,
"load scripts": 3.051927328109741,
"load upscalers": 0.005713701248168945,
"refresh VAE": 0.0019965171813964844,
"refresh textual inversion templates": 0.0,
"scripts list_optimizers": 0.04787254333496094,
"scripts list_unets": 0.0,
"reload hypernetworks": 0.0,
"initialize extra networks": 0.03012871742248535,
"scripts before_ui_callback": 0.0010442733764648438,
"create ui": 0.8883819580078125,
"gradio launch": 0.305774450302124,
"add APIs": 0.006952047348022461,
"app_started_callback/lora_script.py": 0.0,
"app_started_callback/controlnet.py": 0.001995086669921875,
"app_started_callback": 0.001995086669921875
}
},
"Packages": [
"absl-py==2.1.0",
"accelerate==0.21.0",
"addict==2.4.0",
"aenum==3.1.15",
"aiofiles==23.2.1",
"aiohttp==3.9.5",
"aiosignal==1.3.1",
"albumentations==1.3.1",
"altair==5.3.0",
"antlr4-python3-runtime==4.9.3",
"anyio==3.7.1",
"async-timeout==4.0.3",
"attrs==23.2.0",
"basicsr==1.4.2",
"blendmodes==2022",
"certifi==2024.2.2",
"cffi==1.16.0",
"chardet==5.2.0",
"charset-normalizer==3.3.2",
"clean-fid==0.1.35",
"click==8.1.7",
"clip==1.0",
"colorama==0.4.6",
"coloredlogs==15.0.1",
"colorlog==6.8.2",
"contourpy==1.2.0",
"cssselect2==0.7.0",
"cycler==0.12.1",
"cython==3.0.8",
"deprecation==2.1.0",
"depth-anything==2024.1.22.0",
"diffusers==0.25.0",
"easydict==1.11",
"einops==0.4.1",
"embreex==2.17.7.post4",
"exceptiongroup==1.2.0",
"facexlib==0.3.0",
"fastapi==0.94.0",
"ffmpy==0.3.1",
"filelock==3.13.1",
"filterpy==1.4.5",
"flatbuffers==24.3.25",
"fonttools==4.47.2",
"frozenlist==1.4.1",
"fsspec==2024.2.0",
"ftfy==6.1.3",
"future==0.18.3",
"fvcore==0.1.5.post20221221",
"gitdb==4.0.11",
"gitpython==3.1.32",
"gradio-client==0.5.0",
"gradio==3.41.2",
"grpcio==1.60.1",
"h11==0.12.0",
"handrefinerportable==2024.2.12.0",
"httpcore==0.15.0",
"httpx==0.24.1",
"huggingface-hub==0.20.3",
"humanfriendly==10.0",
"idna==3.6",
"imageio==2.33.1",
"importlib-metadata==7.0.1",
"importlib-resources==6.4.0",
"inflection==0.5.1",
"insightface==0.7.3",
"iopath==0.1.9",
"jax==0.4.28",
"jaxlib==0.4.28",
"jinja2==3.1.3",
"joblib==1.3.2",
"jsonmerge==1.8.0",
"jsonschema-specifications==2023.12.1",
"jsonschema==4.21.1",
"kiwisolver==1.4.5",
"kornia==0.6.7",
"lark==1.1.2",
"lazy-loader==0.3",
"lightning-utilities==0.10.1",
"llvmlite==0.42.0",
"lmdb==1.4.1",
"lxml==5.2.2",
"mapbox-earcut==1.0.1",
"markdown==3.5.2",
"markupsafe==2.1.5",
"matplotlib==3.8.2",
"mediapipe==0.10.14",
"ml-dtypes==0.4.0",
"mpmath==1.3.0",
"multidict==6.0.5",
"networkx==3.2.1",
"numba==0.59.0",
"numpy==1.26.2",
"omegaconf==2.2.3",
"onnx==1.15.0",
"onnxruntime==1.17.0",
"open-clip-torch==2.20.0",
"opencv-contrib-python==4.9.0.80",
"opencv-python-headless==4.9.0.80",
"opencv-python==4.10.0.82",
"opt-einsum==3.3.0",
"orjson==3.9.13",
"packaging==23.2",
"pandas==2.2.0",
"piexif==1.1.3",
"pillow==9.5.0",
"pip==24.0",
"platformdirs==4.2.0",
"portalocker==2.8.2",
"prettytable==3.9.0",
"protobuf==4.25.3",
"psutil==5.9.5",
"pycollada==0.8",
"pycparser==2.22",
"pydantic==1.10.15",
"pydub==0.25.1",
"pyparsing==3.1.1",
"pyreadline3==3.4.1",
"python-dateutil==2.8.2",
"python-multipart==0.0.7",
"pytorch-lightning==1.9.4",
"pytz==2024.1",
"pywavelets==1.5.0",
"pywin32==306",
"pyyaml==6.0.1",
"qudida==0.0.4",
"referencing==0.33.0",
"regex==2023.12.25",
"reportlab==4.2.0",
"requests==2.31.0",
"resize-right==0.0.2",
"rpds-py==0.17.1",
"rtree==1.2.0",
"safetensors==0.4.2",
"scikit-image==0.21.0",
"scikit-learn==1.4.0",
"scipy==1.12.0",
"semantic-version==2.10.0",
"sentencepiece==0.2.0",
"setuptools==69.0.3",
"shapely==2.0.2",
"six==1.16.0",
"smmap==5.0.1",
"sniffio==1.3.0",
"sounddevice==0.4.7",
"spandrel==0.1.6",
"starlette==0.26.1",
"svg.path==6.3",
"svglib==1.5.1",
"sympy==1.12",
"tabulate==0.9.0",
"tb-nightly==2.16.0a20240204",
"tensorboard-data-server==0.7.2",
"termcolor==2.4.0",
"tf-keras-nightly==2.16.0.dev2024020410",
"threadpoolctl==3.2.0",
"tifffile==2024.1.30",
"timm==1.0.3",
"tinycss2==1.3.0",
"tokenizers==0.13.3",
"tomesd==0.1.3",
"tomli==2.0.1",
"toolz==0.12.1",
"torch==2.1.2+cu121",
"torchdiffeq==0.2.3",
"torchmetrics==1.3.0.post0",
"torchsde==0.2.6",
"torchvision==0.16.2+cu121",
"tqdm==4.66.4",
"trampoline==0.1.2",
"transformers==4.30.2",
"trimesh==4.1.3",
"typing-extensions==4.9.0",
"tzdata==2023.4",
"urllib3==2.2.0",
"uvicorn==0.27.0.post1",
"vhacdx==0.0.5",
"wcwidth==0.2.13",
"webencodings==0.5.1",
"websockets==11.0.3",
"werkzeug==3.0.1",
"wheel==0.42.0",
"xxhash==3.4.1",
"yacs==0.1.8",
"yapf==0.40.2",
"yarl==1.9.4",
"zipp==3.17.0"
]
}

Console logs

To create a public link, set `share=True` in `launch()`.
Startup time: 34.6s (prepare environment: 16.2s, import torch: 6.8s, import gradio: 2.4s, setup paths: 2.8s, initialize shared: 0.2s, other imports: 2.0s, load scripts: 3.1s, create ui: 0.9s, gradio launch: 0.3s).
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.

Additional information

To create a public link, set share=True in launch().
Startup time: 34.6s (prepare environment: 16.2s, import torch: 6.8s, import gradio: 2.4s, setup paths: 2.8s, initialize shared: 0.2s, other imports: 2.0s, load scripts: 3.1s, create ui: 0.9s, gradio launch: 0.3s).
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.

HITTRAKKZ · Answer 1 · Fri Jul 05 2024 22:20:47 GMT+0800 (China Standard Time)

it says in the terminal
Hint: your device supports --cuda-stream for potential speed improvements
if your seeing that that means you don't have the commandline arg --cuda-stream being used, once you add that commandline arg to your other args then that'll change to true

WITHOUT THE COMMANDLINE_ARGS

(forge) c:\sdforge>python launch.py
Launching Web UI with arguments: --allow-code --api --enable-insecure-extension-access --administrator --theme dark --disable-nan-check --cors-allow-origins=https://zero01101.github.io --cors-allow-origins-regex=https://.*.github.io$ --disable-safe-unpickle --timeout-keep-alive 240 --opt-sdp-attention --skip-prepare-environment
Total VRAM 12288 MB, total RAM 49040 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3060 : native
Hint: your device supports --pin-shared-memory for potential speed improvements.
Hint: your device supports --cuda-malloc for potential speed improvements.
Hint: your device supports --cuda-stream for potential speed improvements.
VAE dtype: torch.bfloat16
CUDA Stream Activated: False

WITH THE COMMANDLINE_ARGS

(forge) c:\sdforge>python launch.py
Launching Web UI with arguments: --allow-code --api --enable-insecure-extension-access --administrator --theme dark --disable-nan-check --cors-allow-origins=https://zero01101.github.io --cors-allow-origins-regex=https://.*.github.io$ --disable-safe-unpickle --timeout-keep-alive 240 --opt-sdp-attention --pin-shared-memory --cuda-malloc --cuda-stream --skip-prepare-environment
Using cudaMallocAsync backend.
Total VRAM 12288 MB, total RAM 49040 MB
Set vram state to: NORMAL_VRAM
Always pin shared GPU memory
Device: cuda:0 NVIDIA GeForce RTX 3060 : cudaMallocAsync
VAE dtype: torch.bfloat16
CUDA Stream Activated: True