diff --git a/README.md b/README.md index 4aa55b9..5755c0a 100644 --- a/README.md +++ b/README.md @@ -112,6 +112,7 @@ MOSS是一个支持中英双语和多种插件的开源对话语言模型,`mos ## :robot: 本地部署 + ### 硬件要求 下表提供了一个batch size=1时本地部署MOSS进行推理所需的显存大小。**量化模型暂时不支持模型并行。** @@ -123,6 +124,7 @@ MOSS是一个支持中英双语和多种插件的开源对话语言模型,`mos | Int4 | 7.8GB | 12GB | 26GB | ### 下载安装 + 1. 下载本仓库内容至本地/远程服务器 ```bash @@ -130,22 +132,16 @@ git clone https://github.com/OpenLMLab/MOSS.git cd MOSS ``` -2. 创建conda环境 +2. 创建 conda 环境 ```bash -conda create --name moss python=3.8 +conda env create --file conda-recipe.yaml # or `mamba env create --file conda-recipe.yaml` conda activate moss ``` -3. 安装依赖 - -```bash -pip install -r requirements.txt -``` - -其中`torch`和`transformers`版本不建议低于推荐版本。 +其中 `torch` 和 `transformers` 版本不建议低于推荐版本。 -目前triton仅支持Linux及WSL,暂不支持Windows及Mac OS,请等待后续更新。 +目前 triton 仅支持 Linux 及 WSL,暂不支持 Windows 及 macOS,请等待后续更新。 ### 使用示例 diff --git a/README_en.md b/README_en.md index e60f132..c4db646 100644 --- a/README_en.md +++ b/README_en.md @@ -108,6 +108,7 @@ MOSS is an open-sourced plugin-augmented conversational language model. `moss-mo ## :robot: Chat with MOSS + ### GPU Requirements The table below shows the minimal GPU memory required by performing MOSS inference when batch size is 1. Please note that **currently the quantized models do not support model parallism**. @@ -119,6 +120,7 @@ The table below shows the minimal GPU memory required by performing MOSS inferen | Int4 | 7.8GB | 12GB | 26GB | ### Installation + 1. Clone this repo to your local/remote machine. ```bash @@ -129,17 +131,11 @@ cd MOSS 2. Create a new conda environment ```bash -conda create --name moss python=3.8 +conda env create --file conda-recipe.yaml # or `mamba env create --file conda-recipe.yaml` conda activate moss ``` -3. Install requirements - -```bash -pip install -r requirements.txt -``` - -4. (Optional) 4/8-bit quantization requirement +3. (Optional) 4/8-bit quantization requirement ```bash pip install triton @@ -147,7 +143,7 @@ pip install triton Note that the version of `torch` and `transformers` should be equal or higher than recommended. -Currently triton only supports Linux and WSL. Please wait for later updates if you are using Windows/MacOS. +Currently triton only supports Linux and WSL. Please wait for later updates if you are using Windows/macOS. ### Try MOSS diff --git a/conda-recipe.yaml b/conda-recipe.yaml new file mode 100644 index 0000000..04dd906 --- /dev/null +++ b/conda-recipe.yaml @@ -0,0 +1,33 @@ +# Create virtual environment with command: +# +# $ conda env create --file conda-recipe.yaml +# + +name: moss + +channels: + - pytorch + - huggingface + - nvidia/label/cuda-11.7.1 + - defaults + - conda-forge + +dependencies: + - python = 3.10 + - pip + + - pytorch::pytorch >= 1.13 + - pytorch::pytorch-mutex = *=*cuda* + - nvidia/label/cuda-11.7.1::cuda-toolkit = 11.7 + + - huggingface::transformers >= 4.25 + - huggingface::datasets + - accelerate + - huggingface_hub + - sentencepiece + + - matplotlib-base + - gradio + - streamlit + - pip: + - mdtex2html