Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Encounter some problems when building wheel #1418

Open
ZarkPanda opened this issue Jan 2, 2025 · 0 comments
Open

Encounter some problems when building wheel #1418

ZarkPanda opened this issue Jan 2, 2025 · 0 comments

Comments

@ZarkPanda
Copy link

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting flash-attn
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/83/29/48df18cb51902a7cb7a0ee13327bb2cf50b6ba24bd2e8283d0a9538dde52/flash_attn-2.7.2.post1.tar.gz (3.1 MB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: torch in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from flash-attn) (2.4.0)
Requirement already satisfied: einops in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from flash-attn) (0.8.0)
Requirement already satisfied: filelock in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (3.16.1)
Requirement already satisfied: typing-extensions>=4.8.0 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (4.12.2)
Requirement already satisfied: sympy in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (1.13.3)
Requirement already satisfied: networkx in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (3.4.2)
Requirement already satisfied: jinja2 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (3.1.5)
Requirement already satisfied: fsspec in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (2024.9.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (12.1.105)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (12.1.105)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (9.1.0.70)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (12.1.3.1)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (11.0.2.54)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (10.3.2.106)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (11.4.5.107)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (12.1.0.106)
Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (2.20.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (12.1.105)
Requirement already satisfied: triton==3.0.0 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from torch->flash-attn) (3.0.0)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch->flash-attn) (12.6.85)
Requirement already satisfied: MarkupSafe>=2.0 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from jinja2->torch->flash-attn) (3.0.2)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages (from sympy->torch->flash-attn) (1.3.0)
Building wheels for collected packages: flash-attn
Building wheel for flash-attn (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [232 lines of output]
fatal: 不是 git 仓库（或者任何父目录）：.git

  torch.__version__  = 2.4.0+cu121
  
  
  /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/__init__.py:94: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
  !!
  
          ********************************************************************************
          Requirements should be satisfied by a PEP 517 installer.
          If you are using pip, you can try `pip install --use-pep517`.
          ********************************************************************************
  
  !!
    dist.fetch_build_eggs(dist.setup_requires)
  running bdist_wheel
  Guessing wheel URL:  https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.2.post1/flash_attn-2.7.2.post1+cu12torch2.4cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
  Precompiled wheel not found. Building from source...
  running build
  running build_py
  creating build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/flash_attn_triton.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/flash_blocksparse_attn_interface.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/bert_padding.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/flash_attn_interface.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/fused_softmax.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/flash_attn_triton_og.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/flash_blocksparse_attention.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  creating build/lib.linux-x86_64-cpython-310/hopper
  copying hopper/flash_attn_interface.py -> build/lib.linux-x86_64-cpython-310/hopper
  copying hopper/benchmark_split_kv.py -> build/lib.linux-x86_64-cpython-310/hopper
  copying hopper/benchmark_flash_attention_fp8.py -> build/lib.linux-x86_64-cpython-310/hopper
  copying hopper/__init__.py -> build/lib.linux-x86_64-cpython-310/hopper
  copying hopper/test_flash_attn.py -> build/lib.linux-x86_64-cpython-310/hopper
  copying hopper/test_kvcache.py -> build/lib.linux-x86_64-cpython-310/hopper
  copying hopper/test_attn_kvcache.py -> build/lib.linux-x86_64-cpython-310/hopper
  copying hopper/setup.py -> build/lib.linux-x86_64-cpython-310/hopper
  copying hopper/benchmark_attn.py -> build/lib.linux-x86_64-cpython-310/hopper
  creating build/lib.linux-x86_64-cpython-310/flash_attn/utils
  copying flash_attn/utils/benchmark.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
  copying flash_attn/utils/generation.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
  copying flash_attn/utils/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
  copying flash_attn/utils/pretrained.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
  copying flash_attn/utils/distributed.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
  creating build/lib.linux-x86_64-cpython-310/flash_attn/losses
  copying flash_attn/losses/cross_entropy.py -> build/lib.linux-x86_64-cpython-310/flash_attn/losses
  copying flash_attn/losses/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/losses
  creating build/lib.linux-x86_64-cpython-310/flash_attn/modules
  copying flash_attn/modules/mha.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
  copying flash_attn/modules/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
  copying flash_attn/modules/embedding.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
  copying flash_attn/modules/block.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
  copying flash_attn/modules/mlp.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
  creating build/lib.linux-x86_64-cpython-310/flash_attn/ops
  copying flash_attn/ops/activations.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
  copying flash_attn/ops/layer_norm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
  copying flash_attn/ops/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
  copying flash_attn/ops/fused_dense.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
  copying flash_attn/ops/rms_norm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
  creating build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/llama.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/opt.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/gpt.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/vit.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/falcon.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/bert.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/gptj.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/bigcode.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/btlm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/gpt_neox.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/baichuan.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  creating build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/fwd_ref.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/interface_torch.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/fwd_decode.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/bwd_ref.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/fwd_prefill.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/test.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/utils.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/bwd_prefill.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/bench.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  copying flash_attn/flash_attn_triton_amd/interface_fa.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
  creating build/lib.linux-x86_64-cpython-310/flash_attn/layers
  copying flash_attn/layers/rotary.py -> build/lib.linux-x86_64-cpython-310/flash_attn/layers
  copying flash_attn/layers/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/layers
  copying flash_attn/layers/patch_embed.py -> build/lib.linux-x86_64-cpython-310/flash_attn/layers
  creating build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/cross_entropy.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/k_activations.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/layer_norm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/rotary.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/linear.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/mlp.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  running build_ext
  /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/utils/cpp_extension.py:414: UserWarning: The detected CUDA version (12.2) has a minor version mismatch with the version that was used to compile PyTorch (12.1). Most likely this shouldn't be a problem.
    warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
  /ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/utils/cpp_extension.py:424: UserWarning: There are no g++ version bounds defined for CUDA version 12.2
    warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
  building 'flash_attn_2_cuda' extension
  creating /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn
  creating /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src
  Emitting ninja build file /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/build.ninja...
  Compiling objects...
  Using envvar MAX_JOBS (4) as the number of workers...
  [1/85] c++ -MMD -MF /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/flash_api.o.d -pthread -B /ssd/zhangz/Anaconda3/envs/medm/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /ssd/zhangz/Anaconda3/envs/medm/include -fPIC -O2 -isystem /ssd/zhangz/Anaconda3/envs/medm/include -fPIC -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/cutlass/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/TH -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/THC -I/home/zhangz/cuda/cuda-12.2/include -I/ssd/zhangz/Anaconda3/envs/medm/include/python3.10 -c -c /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/flash_api.cpp -o /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/flash_api.o -O3 -std=c++17 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=flash_attn_2_cuda -D_GLIBCXX_USE_CXX11_ABI=0
  [2/85] /home/zhangz/cuda/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_fp16_causal_sm80.o.d -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/cutlass/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/TH -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/THC -I/home/zhangz/cuda/cuda-12.2/include -I/ssd/zhangz/Anaconda3/envs/medm/include/python3.10 -c -c /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src/flash_bwd_hdim128_fp16_causal_sm80.cu -o /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_fp16_causal_sm80.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 --threads 2 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=flash_attn_2_cuda -D_GLIBCXX_USE_CXX11_ABI=0
  FAILED: /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_fp16_causal_sm80.o
  /home/zhangz/cuda/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_fp16_causal_sm80.o.d -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/cutlass/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/TH -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/THC -I/home/zhangz/cuda/cuda-12.2/include -I/ssd/zhangz/Anaconda3/envs/medm/include/python3.10 -c -c /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src/flash_bwd_hdim128_fp16_causal_sm80.cu -o /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_fp16_causal_sm80.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 --threads 2 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=flash_attn_2_cuda -D_GLIBCXX_USE_CXX11_ABI=0
  Killed
  [3/85] /home/zhangz/cuda/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_bf16_causal_sm80.o.d -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/cutlass/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/TH -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/THC -I/home/zhangz/cuda/cuda-12.2/include -I/ssd/zhangz/Anaconda3/envs/medm/include/python3.10 -c -c /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src/flash_bwd_hdim128_bf16_causal_sm80.cu -o /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_bf16_causal_sm80.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 --threads 2 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=flash_attn_2_cuda -D_GLIBCXX_USE_CXX11_ABI=0
  [4/85] /home/zhangz/cuda/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_bf16_sm80.o.d -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/cutlass/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/TH -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/THC -I/home/zhangz/cuda/cuda-12.2/include -I/ssd/zhangz/Anaconda3/envs/medm/include/python3.10 -c -c /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src/flash_bwd_hdim128_bf16_sm80.cu -o /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_bf16_sm80.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 --threads 2 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=flash_attn_2_cuda -D_GLIBCXX_USE_CXX11_ABI=0
  FAILED: /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_bf16_sm80.o
  /home/zhangz/cuda/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_bf16_sm80.o.d -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/cutlass/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/TH -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/THC -I/home/zhangz/cuda/cuda-12.2/include -I/ssd/zhangz/Anaconda3/envs/medm/include/python3.10 -c -c /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src/flash_bwd_hdim128_bf16_sm80.cu -o /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_bf16_sm80.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 --threads 2 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=flash_attn_2_cuda -D_GLIBCXX_USE_CXX11_ABI=0
  Killed
  [5/85] /home/zhangz/cuda/cuda-12.2/bin/nvcc --generate-dependencies-with-compile --dependency-output /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_fp16_sm80.o.d -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src -I/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/cutlass/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/TH -I/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/include/THC -I/home/zhangz/cuda/cuda-12.2/include -I/ssd/zhangz/Anaconda3/envs/medm/include/python3.10 -c -c /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/csrc/flash_attn/src/flash_bwd_hdim128_fp16_sm80.cu -o /tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src/flash_bwd_hdim128_fp16_sm80.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 --threads 2 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=flash_attn_2_cuda -D_GLIBCXX_USE_CXX11_ABI=0
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/urllib/request.py", line 1348, in do_open
      h.request(req.get_method(), req.selector, req.data, headers,
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/http/client.py", line 1283, in request
      self._send_request(method, url, body, headers, encode_chunked)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/http/client.py", line 1329, in _send_request
      self.endheaders(body, encode_chunked=encode_chunked)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/http/client.py", line 1278, in endheaders
      self._send_output(message_body, encode_chunked=encode_chunked)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/http/client.py", line 1038, in _send_output
      self.send(msg)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/http/client.py", line 976, in send
      self.connect()
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/http/client.py", line 1448, in connect
      super().connect()
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/http/client.py", line 942, in connect
      self.sock = self._create_connection(
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/socket.py", line 857, in create_connection
      raise err
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/socket.py", line 845, in create_connection
      sock.connect(sa)
  ConnectionRefusedError: [Errno 111] Connection refused
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/setup.py", line 465, in run
      urllib.request.urlretrieve(wheel_url, wheel_filename)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/urllib/request.py", line 241, in urlretrieve
      with contextlib.closing(urlopen(url, data)) as fp:
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/urllib/request.py", line 216, in urlopen
      return opener.open(url, data, timeout)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/urllib/request.py", line 519, in open
      response = self._open(req, data)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/urllib/request.py", line 536, in _open
      result = self._call_chain(self.handle_open, protocol, protocol +
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/urllib/request.py", line 496, in _call_chain
      result = func(*args)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/urllib/request.py", line 1391, in https_open
      return self.do_open(http.client.HTTPSConnection, req,
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/urllib/request.py", line 1351, in do_open
      raise URLError(err)
  urllib.error.URLError: <urlopen error [Errno 111] Connection refused>
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2105, in _run_ninja_build
      subprocess.run(
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/subprocess.py", line 526, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v', '-j', '4']' returned non-zero exit status 1.
  
  The above exception was the direct cause of the following exception:
  
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/setup.py", line 505, in <module>
      setup(
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/__init__.py", line 117, in setup
      return distutils.core.setup(**attrs)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 183, in setup
      return run_commands(dist)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
      dist.run_commands()
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
      self.run_command(cmd)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/dist.py", line 950, in run_command
      super().run_command(command)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-sukldczv/flash-attn_cd141d5ea1a243cbbe601fdc5298daca/setup.py", line 482, in run
      super().run()
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/wheel/_bdist_wheel.py", line 378, in run
      self.run_command("build")
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/dist.py", line 950, in run_command
      super().run_command(command)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
      cmd_obj.run()
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/dist.py", line 950, in run_command
      super().run_command(command)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
      cmd_obj.run()
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 98, in run
      _build_ext.run(self)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
      self.build_extensions()
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 866, in build_extensions
      build_ext.build_extensions(self)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 476, in build_extensions
      self._build_extensions_serial()
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 502, in _build_extensions_serial
      self.build_extension(ext)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 263, in build_extension
      _build_ext.build_extension(self, ext)
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 557, in build_extension
      objects = self.compiler.compile(
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 679, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1785, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/ssd/zhangz/Anaconda3/envs/medm/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2121, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for flash-attn
Running setup.py clean for flash-attn
Failed to build flash-attn
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (flash-attn)

Did anyone know how to solve this?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ZarkPanda