Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

No available kernel. Aborting execution #40

Open
AGhalibH opened this issue Sep 25, 2024 · 2 comments
Open

No available kernel. Aborting execution #40

AGhalibH opened this issue Sep 25, 2024 · 2 comments

Comments

@AGhalibH
Copy link

AGhalibH commented Sep 25, 2024 •
edited
Loading

ComfyUI Error Report

Error Details

  • Node Type: Sam2Segmentation
  • Exception Type: RuntimeError
  • Exception Message: No available kernel. Aborting execution.

Stack Trace

  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)

  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/nodes.py", line 302, in segment
    out_masks, scores, logits = model.predict(
                                ^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/sam2_image_predictor.py", line 271, in predict
    masks, iou_predictions, low_res_masks = self._predict(
                                            ^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/sam2_image_predictor.py", line 400, in _predict
    low_res_masks, iou_predictions, _, _ = self.model.sam_mask_decoder(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/mask_decoder.py", line 136, in forward
    masks, iou_pred, mask_tokens_out, object_score_logits = self.predict_masks(
                                                            ^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/mask_decoder.py", line 213, in predict_masks
    hs, src = self.transformer(src, pos_src, tokens)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 113, in forward
    queries, keys = layer(
                    ^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 179, in forward
    queries = self.self_attn(q=queries, k=queries, v=queries)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 265, in forward
    out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

System Information

  • ComfyUI Version: v0.2.2-69-g6ad0ddb
  • Arguments: main.py
  • OS: posix
  • Python Version: 3.11.9 (main, Jul 23 2024, 17:22:20) [GCC 14.1.1 20240522]
  • Embedded Python: false
  • PyTorch Version: 2.3.1+rocm6.0

Devices

  • Name: cuda:0 AMD Radeon Graphics : native
    • Type: cuda
    • VRAM Total: 8573157376
    • VRAM Free: 7838681088
    • Torch VRAM Total: 2845835264
    • Torch VRAM Free: 2570635264

Logs

2024-09-25 21:02:57,161 - root - INFO - Total VRAM 8176 MB, total RAM 80358 MB
2024-09-25 21:02:57,161 - root - INFO - pytorch version: 2.3.1+rocm6.0
2024-09-25 21:02:57,161 - root - INFO - Set vram state to: NORMAL_VRAM
2024-09-25 21:02:57,161 - root - INFO - Device: cuda:0 AMD Radeon Graphics : native
2024-09-25 21:02:58,046 - root - INFO - Using sub quadratic optimization for cross attention, if you have memory or speed issues try using: --use-split-cross-attention
2024-09-25 21:02:58,829 - root - INFO - [Prompt Server] web root: /home/aghalib/Downloads/git/ComfyUI/web
2024-09-25 21:02:59,518 - dinov2 - WARNING - xFormers not available
2024-09-25 21:02:59,520 - dinov2 - WARNING - xFormers not available
2024-09-25 21:03:00,640 - root - INFO - Total VRAM 8176 MB, total RAM 80358 MB
2024-09-25 21:03:00,641 - root - INFO - pytorch version: 2.3.1+rocm6.0
2024-09-25 21:03:00,641 - root - INFO - Set vram state to: NORMAL_VRAM
2024-09-25 21:03:00,641 - root - INFO - Device: cuda:0 AMD Radeon Graphics : native
2024-09-25 21:03:02,075 - root - WARNING - Traceback (most recent call last):
  File "/home/aghalib/Downloads/git/ComfyUI/nodes.py", line 1998, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 936, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1073, in get_code
  File "<frozen importlib._bootstrap_external>", line 1130, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Yolain-Workflows/__init__.py'

2024-09-25 21:03:02,075 - root - WARNING - Cannot import /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Yolain-Workflows module for custom nodes: [Errno 2] No such file or directory: '/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Yolain-Workflows/__init__.py'
2024-09-25 21:03:03,345 - numexpr.utils - INFO - NumExpr defaulting to 8 threads.
2024-09-25 21:03:03,982 - root - INFO - 
Import times for custom nodes:
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/quick-connections
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/SD-Latent-Interposer
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/cg-use-everywhere
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/Skimmed_CFG
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/websocket_image_save.py
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Inpaint-CropAndStitch
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/SD-Latent-Upscaler
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_Cutoff
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/masquerade-nodes-comfyui
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-WD14-Tagger
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_TiledKSampler
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-ELLA
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds (IMPORT FAILED): /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Yolain-Workflows
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Florence2
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/comfyui-inpaint-nodes
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-TiledDiffusion
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_FizzNodes
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Custom-Scripts
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-DDColor
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-IC-Light
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_UltimateSDUpscale
2024-09-25 21:03:03,983 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Frame-Interpolation
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/OneButtonPrompt
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/rgthree-comfy
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-KJNodes
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/efficiency-nodes-comfyui
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-GGUF
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_Comfyroll_CustomNodes
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV2
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_FaceAnalysis
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_essentials
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/Flux-Prompt-Generator
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Impact-Pack
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_tinyterraNodes
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Manager
2024-09-25 21:03:03,984 - root - INFO -    0.0 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_ExtraModels
2024-09-25 21:03:03,984 - root - INFO -    0.1 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Inspire-Pack
2024-09-25 21:03:03,984 - root - INFO -    0.1 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-DynamiCrafterWrapper
2024-09-25 21:03:03,984 - root - INFO -    0.1 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_LayerStyle
2024-09-25 21:03:03,984 - root - INFO -    0.1 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Easy-Use
2024-09-25 21:03:03,984 - root - INFO -    0.1 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite
2024-09-25 21:03:03,984 - root - INFO -    0.1 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-BrushNet
2024-09-25 21:03:03,984 - root - INFO -    0.2 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-IF_AI_tools
2024-09-25 21:03:03,984 - root - INFO -    0.2 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Fluxpromptenhancer
2024-09-25 21:03:03,984 - root - INFO -    0.2 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/comfyui-ollama
2024-09-25 21:03:03,985 - root - INFO -    0.2 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-AdvancedLivePortrait
2024-09-25 21:03:03,985 - root - INFO -    0.3 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/comfyui_controlnet_aux
2024-09-25 21:03:03,985 - root - INFO -    0.3 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/comfyui_segment_anything
2024-09-25 21:03:03,985 - root - INFO -    0.4 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-Crystools
2024-09-25 21:03:03,985 - root - INFO -    0.9 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/comfy_mtb
2024-09-25 21:03:03,985 - root - INFO -    1.2 seconds: /home/aghalib/Downloads/git/ComfyUI/custom_nodes/was-node-suite-comfyui
2024-09-25 21:03:03,985 - root - INFO - 
2024-09-25 21:03:03,998 - root - INFO - Starting server

2024-09-25 21:03:03,998 - root - INFO - To see the GUI go to: http://127.0.0.1:8188
2024-09-25 21:03:26,629 - root - ERROR - [ERROR] An error occurred while retrieving information for the 'IF_ChatPrompt' node.
2024-09-25 21:03:26,630 - root - ERROR - Traceback (most recent call last):
  File "/home/aghalib/Downloads/git/ComfyUI/server.py", line 564, in get_object_info
    out[x] = node_info(x)
             ^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/server.py", line 531, in node_info
    info['input'] = obj_class.INPUT_TYPES()
                    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-IF_AI_tools/IFChatPromptNode.py", line 109, in INPUT_TYPES
    node = cls()
           ^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-IF_AI_tools/IFChatPromptNode.py", line 53, in __init__
    self.agent_tools = self.load_agent_tools()
                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-IF_AI_tools/IFChatPromptNode.py", line 78, in load_agent_tools
    for filename in os.listdir(self.agents_dir):
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/aghalib/Downloads/git/ComfyUI/input/IF_AI/presets/agents'

2024-09-25 21:03:26,655 - root - ERROR - [ERROR] An error occurred while retrieving information for the 'CR Select Font' node.
2024-09-25 21:03:26,655 - root - ERROR - Traceback (most recent call last):
  File "/home/aghalib/Downloads/git/ComfyUI/server.py", line 564, in get_object_info
    out[x] = node_info(x)
             ^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/server.py", line 531, in node_info
    info['input'] = obj_class.INPUT_TYPES()
                    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI_Comfyroll_CustomNodes/nodes/nodes_graphics_text.py", line 467, in INPUT_TYPES
    file_list = [f for f in os.listdir(font_dir) if os.path.isfile(os.path.join(font_dir, f)) and f.lower().endswith(".ttf")]
                            ^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/fonts/truetype'

2024-09-25 21:03:52,137 - root - INFO - got prompt
2024-09-25 21:03:55,949 - root - ERROR - !!! Exception during processing !!! No available kernel. Aborting execution.
2024-09-25 21:03:55,951 - root - ERROR - Traceback (most recent call last):
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/nodes.py", line 302, in segment
    out_masks, scores, logits = model.predict(
                                ^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/sam2_image_predictor.py", line 271, in predict
    masks, iou_predictions, low_res_masks = self._predict(
                                            ^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/sam2_image_predictor.py", line 400, in _predict
    low_res_masks, iou_predictions, _, _ = self.model.sam_mask_decoder(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/mask_decoder.py", line 136, in forward
    masks, iou_pred, mask_tokens_out, object_score_logits = self.predict_masks(
                                                            ^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/mask_decoder.py", line 213, in predict_masks
    hs, src = self.transformer(src, pos_src, tokens)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 113, in forward
    queries, keys = layer(
                    ^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 179, in forward
    queries = self.self_attn(q=queries, k=queries, v=queries)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 265, in forward
    out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: No available kernel. Aborting execution.

2024-09-25 21:03:55,952 - root - INFO - Prompt executed in 3.81 seconds
2024-09-25 21:04:21,957 - root - INFO - got prompt
2024-09-25 21:04:24,158 - root - ERROR - !!! Exception during processing !!! No available kernel. Aborting execution.
2024-09-25 21:04:24,160 - root - ERROR - Traceback (most recent call last):
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/nodes.py", line 302, in segment
    out_masks, scores, logits = model.predict(
                                ^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/sam2_image_predictor.py", line 271, in predict
    masks, iou_predictions, low_res_masks = self._predict(
                                            ^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/sam2_image_predictor.py", line 400, in _predict
    low_res_masks, iou_predictions, _, _ = self.model.sam_mask_decoder(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/mask_decoder.py", line 136, in forward
    masks, iou_pred, mask_tokens_out, object_score_logits = self.predict_masks(
                                                            ^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/mask_decoder.py", line 213, in predict_masks
    hs, src = self.transformer(src, pos_src, tokens)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 113, in forward
    queries, keys = layer(
                    ^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 179, in forward
    queries = self.self_attn(q=queries, k=queries, v=queries)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 265, in forward
    out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: No available kernel. Aborting execution.

2024-09-25 21:04:24,162 - root - INFO - Prompt executed in 2.20 seconds
2024-09-25 21:15:42,792 - root - INFO - got prompt
2024-09-25 21:15:42,814 - root - INFO - Prompt executed in 0.02 seconds
2024-09-25 21:16:17,011 - root - INFO - got prompt
2024-09-25 21:16:17,116 - root - INFO - Prompt executed in 0.10 seconds
2024-09-25 21:28:33,571 - root - INFO - got prompt
2024-09-25 21:28:48,275 - root - INFO - Prompt executed in 14.70 seconds
2024-09-25 21:30:11,811 - root - INFO - got prompt
2024-09-25 21:30:23,112 - root - INFO - Prompt executed in 11.30 seconds
2024-09-25 21:31:43,657 - root - INFO - got prompt
2024-09-25 21:31:47,883 - root - ERROR - !!! Exception during processing !!! No available kernel. Aborting execution.
2024-09-25 21:31:47,885 - root - ERROR - Traceback (most recent call last):
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/nodes.py", line 302, in segment
    out_masks, scores, logits = model.predict(
                                ^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/sam2_image_predictor.py", line 271, in predict
    masks, iou_predictions, low_res_masks = self._predict(
                                            ^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/sam2_image_predictor.py", line 400, in _predict
    low_res_masks, iou_predictions, _, _ = self.model.sam_mask_decoder(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/mask_decoder.py", line 136, in forward
    masks, iou_pred, mask_tokens_out, object_score_logits = self.predict_masks(
                                                            ^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/mask_decoder.py", line 213, in predict_masks
    hs, src = self.transformer(src, pos_src, tokens)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 113, in forward
    queries, keys = layer(
                    ^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 179, in forward
    queries = self.self_attn(q=queries, k=queries, v=queries)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 265, in forward
    out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: No available kernel. Aborting execution.

2024-09-25 21:31:47,886 - root - INFO - Prompt executed in 4.23 seconds
2024-09-25 21:31:51,480 - root - INFO - got prompt
2024-09-25 21:31:55,176 - root - ERROR - !!! Exception during processing !!! No available kernel. Aborting execution.
2024-09-25 21:31:55,178 - root - ERROR - Traceback (most recent call last):
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/home/aghalib/Downloads/git/ComfyUI/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/nodes.py", line 302, in segment
    out_masks, scores, logits = model.predict(
                                ^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/sam2_image_predictor.py", line 271, in predict
    masks, iou_predictions, low_res_masks = self._predict(
                                            ^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/sam2_image_predictor.py", line 400, in _predict
    low_res_masks, iou_predictions, _, _ = self.model.sam_mask_decoder(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/mask_decoder.py", line 136, in forward
    masks, iou_pred, mask_tokens_out, object_score_logits = self.predict_masks(
                                                            ^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/mask_decoder.py", line 213, in predict_masks
    hs, src = self.transformer(src, pos_src, tokens)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 113, in forward
    queries, keys = layer(
                    ^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 179, in forward
    queries = self.self_attn(q=queries, k=queries, v=queries)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aghalib/Downloads/git/ComfyUI/custom_nodes/ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py", line 265, in forward
    out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: No available kernel. Aborting execution.

2024-09-25 21:31:55,179 - root - INFO - Prompt executed in 3.70 seconds

Attached Workflow

Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

{"last_node_id":22,"last_link_id":37,"nodes":[{"id":1,"type":"LoadImage","pos":{"0":30,"1":230},"size":[320,310],"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[{"name":"IMAGE","type":"IMAGE","links":[7,33],"slot_index":0,"shape":3},{"name":"MASK","type":"MASK","links":null,"shape":3}],"properties":{"Node name for S&R":"LoadImage"},"widgets_values":["ComfyUI_00001_.png","image"]},{"id":20,"type":"Florence2ModelLoader","pos":{"0":20,"1":620},"size":{"0":320,"1":110},"flags":{},"order":1,"mode":0,"inputs":[{"name":"lora","type":"PEFTLORA","link":null}],"outputs":[{"name":"florence2_model","type":"FL2MODEL","links":[34],"shape":3,"slot_index":0}],"properties":{"Node name for S&R":"Florence2ModelLoader"},"widgets_values":["Florence-2-large","fp16","sdpa"]},{"id":21,"type":"Florence2toCoordinates","pos":{"0":880,"1":390},"size":{"0":320,"1":100},"flags":{},"order":4,"mode":0,"inputs":[{"name":"data","type":"JSON","link":35}],"outputs":[{"name":"center_coordinates","type":"STRING","links":null,"shape":3},{"name":"bboxes","type":"BBOX","links":[36],"shape":3,"slot_index":1}],"properties":{"Node name for S&R":"Florence2toCoordinates"},"widgets_values":["0",false]},{"id":22,"type":"MaskPreview+","pos":{"0":1620,"1":210},"size":{"0":210,"1":250},"flags":{},"order":6,"mode":0,"inputs":[{"name":"mask","type":"MASK","link":37}],"outputs":[],"properties":{"Node name for S&R":"MaskPreview+"}},{"id":6,"type":"Sam2Segmentation","pos":{"0":1240,"1":210},"size":{"0":320,"1":190},"flags":{},"order":5,"mode":0,"inputs":[{"name":"sam2_model","type":"SAM2MODEL","link":8},{"name":"image","type":"IMAGE","link":7},{"name":"bboxes","type":"BBOX","link":36},{"name":"mask","type":"MASK","link":null},{"name":"coordinates_positive","type":"STRING","link":null,"widget":{"name":"coordinates_positive"}},{"name":"coordinates_negative","type":"STRING","link":null,"widget":{"name":"coordinates_negative"}}],"outputs":[{"name":"mask","type":"MASK","links":[37],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"Sam2Segmentation"},"widgets_values":[true,"","",false]},{"id":5,"type":"DownloadAndLoadSAM2Model","pos":{"0":710,"1":20},"size":{"0":320,"1":130},"flags":{},"order":2,"mode":0,"inputs":[],"outputs":[{"name":"sam2_model","type":"SAM2MODEL","links":[8],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"DownloadAndLoadSAM2Model"},"widgets_values":["sam2_hiera_base_plus.safetensors","single_image","cuda","bf16"]},{"id":19,"type":"Florence2Run","pos":{"0":420,"1":600},"size":{"0":400,"1":352},"flags":{},"order":3,"mode":0,"inputs":[{"name":"image","type":"IMAGE","link":33},{"name":"florence2_model","type":"FL2MODEL","link":34}],"outputs":[{"name":"image","type":"IMAGE","links":null,"shape":3},{"name":"mask","type":"MASK","links":null,"shape":3},{"name":"caption","type":"STRING","links":null,"shape":3},{"name":"data","type":"JSON","links":[35],"shape":3,"slot_index":3}],"properties":{"Node name for S&R":"Florence2Run"},"widgets_values":["face","caption_to_phrase_grounding",true,false,1024,3,true,"",281130921333049,"fixed"]}],"links":[[7,1,0,6,1,"IMAGE"],[8,5,0,6,0,"SAM2MODEL"],[33,1,0,19,0,"IMAGE"],[34,20,0,19,1,"FL2MODEL"],[35,19,3,21,0,"JSON"],[36,21,1,6,2,"BBOX"],[37,6,0,22,0,"MASK"]],"groups":[],"config":{},"extra":{},"version":0.4}

Additional Context

(Please add any additional context or steps to reproduce the error here)

20240925-troubleshooting

OS: Arch Linux
Python: 3.11.9
ROCM: 6.0
PYTORCH: 2.3.1+rocm6.0
CPU: AMD Ryzen 3 3100
GPU: AMD RX 6600 XT 8GB
RAM: 80 GB DDR4

$ python -m torch.utils.collect_env
amdgpu.ids: No such file or directory
<frozen runpy>:128: RuntimeWarning: 'torch.utils.collect_env' found in sys.modules after import of package 'torch.utils', but prior to execution of 'torch.utils.collect_env'; this may result in unpredictable behaviour
Collecting environment information...
PyTorch version: 2.3.1+rocm6.0
Is debug build: False
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: 6.0.32830-d62f6a171

OS: Arch Linux (x86_64)
GCC version: (GCC) 14.2.1 20240805
Clang version: Could not collect
CMake version: version 3.30.2
Libc version: glibc-2.40

Python version: 3.11.9 (main, Jul 23 2024, 17:22:20) [GCC 14.1.1 20240522] (64-bit runtime)
Python platform: Linux-6.6.48-1-lts-x86_64-with-glibc2.40
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: AMD Radeon Graphics (gfx1032)
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: 6.0.32830
MIOpen runtime version: 3.0.0
Is XNNPACK available: True

CPU:
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Address sizes:                        43 bits physical, 48 bits virtual
Byte Order:                           Little Endian
CPU(s):                               8
On-line CPU(s) list:                  0-7
Vendor ID:                            AuthenticAMD
Model name:                           AMD Ryzen 3 3100 4-Core Processor
CPU family:                           23
Model:                                113
Thread(s) per core:                   2
Core(s) per socket:                   4
Socket(s):                            1
Stepping:                             0
Frequency boost:                      enabled
CPU(s) scaling MHz:                   76%
CPU max MHz:                          3905.8589
CPU min MHz:                          2200.0000
BogoMIPS:                             7189.67
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sev sev_es
Virtualization:                       AMD-V
L1d cache:                            128 KiB (4 instances)
L1i cache:                            128 KiB (4 instances)
L2 cache:                             2 MiB (4 instances)
L3 cache:                             16 MiB (2 instances)
NUMA node(s):                         1
NUMA node0 CPU(s):                    0-7
Vulnerability Gather data sampling:   Not affected
Vulnerability Itlb multihit:          Not affected
Vulnerability L1tf:                   Not affected
Vulnerability Mds:                    Not affected
Vulnerability Meltdown:               Not affected
Vulnerability Mmio stale data:        Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Mitigation; untrained return thunk; SMT enabled with STIBP protection
Vulnerability Spec rstack overflow:   Mitigation; Safe RET
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; Retpolines; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Not affected

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.25.2
[pip3] onnx==1.16.2
[pip3] onnxruntime==1.19.2
[pip3] onnxruntime-gpu==1.15.1
[pip3] open_clip_torch==2.26.1
[pip3] pytorch-lightning==2.4.0
[pip3] pytorch-triton-rocm==2.3.1
[pip3] torch==2.3.1+rocm6.0
[pip3] torchaudio==2.3.1+rocm6.0
[pip3] torchmetrics==1.4.1
[pip3] torchscale==0.3.0
[pip3] torchsde==0.2.6
[pip3] torchvision==0.18.1+rocm6.0
[conda] Could not collect
@kijai
Copy link
Owner

kijai commented Oct 2, 2024

That's AMD card yeah? I have never been able to test with one so I don't know if it's even supposed to work, maybe someone who has can comment.

@ProtoBelisarius
Copy link

Im running a 6800XT on linux and the following made it work. I got the same No available kernel. Aborting execution error

Im running the nightly torch version, though I didnt test it with other versions. Before my fix neither rocm5.7, 6.1, 6.2 or nightly worked, always giving the exact same error, so Im not sure if nightly is needed.
pytorch version: 2.7.0.dev20250105+rocm6.2.4

I added AMD specific handling to: ComfyUI-segment-anything-2/sam2/modeling/sam/transformer.py

def forward(self, q: Tensor, k: Tensor, v: Tensor) -> Tensor:
    # Existing projection code remains the same
    q = self.q_proj(q)
    k = self.k_proj(k)
    v = self.v_proj(v)
    
    # Separate into heads
    q = self._separate_heads(q, self.num_heads)
    k = self._separate_heads(k, self.num_heads)
    v = self._separate_heads(v, self.num_heads)
    
    #use basic attention implementation
    dropout_p = self.dropout_p if self.training else 0.0
    
    #fallback for AMD
    with torch.backends.cuda.sdp_kernel(enable_math=True):
        out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
    
    out = self._recombine_heads(out)
    out = self.out_proj(out)
    return out

I also set these environment variables, Im not sure if they are necessary but I use them for something else as well, so they are always active for me.

export PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.8,max_split_size_mb:512

export HSA_OVERRIDE_GFX_VERSION=10.3.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@AGhalibH @kijai @ProtoBelisarius