Procedure (e.g. Allenamd)
HW: Choose RTX 3x 以上 GPU
OS: Ubuntu 22.04 still ok, 不過最好 upgrade to 24.04?
install CUDA choose cuda 12.1
anaconda_2025 install. 改成 mini-forge 使用 mamba.
=> conda create –name torch26 –clone base (python 13)
先裝 Pytorch!
再來裝 transformers and openi
uv pip install transformers, openai
Transformers 裝完最好先 pytest ！
Source
Good reference 安裝 mamba: https://zhuanlan.zhihu.com/p/25916604332
用 nvcc check cuda 版本，不要用 nvidia-smi! 我用 cuda 12.1, python 3.10, pytorch 2.5!
nvcc –version Cuda compilation tools, release 12.1, V12.1.66 Build cuda_12.1.r12.1/compiler.32415258_0
conda create –name mamba2 python=3.10 conda activate mamba2
安裝: torch
pip install torch torchvision torchaudio –index-url https://download.pytorch.org/whl/cu121
安裝: causal_conv1d!
Releases · Dao-AILab/causal-conv1d
使用命令查看是否支持abi python -c “import torch; print(torch._C._GLIBCXX_USE_CXX11_ABI)” 我的输出 false，就选择abiFALSE的 记得点show all选择对应的torch版本 download from: https://github.com/Dao-AILab/causal-conv1d/releases then uv pip install causal_conv1d-1.5.2+cu12torch2.5cxx11abiFALSE-cp310-cp310-linux_x86_64.whl 做完後 run pytest: 8794 passed, 3888 skipped in 133.40s
安裝: mamba-ssm
同樣的 trick, download: pip install mamba_ssm-2.2.5+cu12torch2.5cxx11abiFALSE-cp310-cp310-linux_x86_64.whl git clone mamba_ssm… run pytest
安裝: Flash attention
同樣的 trick, 但是官網： https://github.com/Dao-AILab/flash-attention/releases
pip install flash_attn-2.8.1+cu12torch2.5cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
Wrong
From llama3 開始 source code installation
https://blog.csdn.net/yyywxk/article/details/144790950
git clone https://github.com/Dao-AILab/causal-conv1d.git cd causal-conv1d git checkout v1.1.1 # 用你想要的版本 CAUSAL_CONV1D_FORCE_BUILD=TRUE pip install . cd .. git clone https://github.com/state-spaces/mamba.git cd mamba git checkout v1.1.1 # 用你想要的版本 pip install . # 方式一，下载whl安装，两种方式选择一个即可 MAMBA_FORCE_BUILD=TRUE pip install . # 方式二，强制在本地编译安装，Win 下无法识别此命令
re-built, conda 都不行。最後這版好像 ok.
1 | |