From 7cd13e2328dc1c9d6c85713fc80bda1b4d0e33b8 Mon Sep 17 00:00:00 2001 From: Xuehai Pan Date: Wed, 26 Apr 2023 19:32:43 +0800 Subject: [PATCH 1/2] Add conda environment recipe --- conda-recipe.yaml | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 conda-recipe.yaml diff --git a/conda-recipe.yaml b/conda-recipe.yaml new file mode 100644 index 0000000..97b1d4a --- /dev/null +++ b/conda-recipe.yaml @@ -0,0 +1,32 @@ +# Create virtual environment with command: +# +# $ conda env create --file conda-recipe.yaml +# + +name: moss + +channels: + - pytorch + - huggingface + - nvidia/label/cuda-11.7.1 + - defaults + - conda-forge + +dependencies: + - python = 3.10 + - pip + + - pytorch::pytorch >= 1.13 + - pytorch::pytorch-mutex = *=*cuda* + - nvidia/label/cuda-11.7.1::cuda-toolkit = 11.7 + + - huggingface::transformers >= 4.25 + - huggingface::datasets + - accelerate + - huggingface_hub + - sentencepiece + + - matplotlib-base + - gradio + - pip: + - mdtex2html From 8f20fe03b6c0323b9494335cf84385374746f587 Mon Sep 17 00:00:00 2001 From: Xuehai Pan Date: Wed, 26 Apr 2023 19:35:18 +0800 Subject: [PATCH 2/2] Update installation instructions --- README.md | 18 +++++++----------- README_en.md | 14 +++++--------- 2 files changed, 12 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 3ca10d3..7e2d776 100644 --- a/README.md +++ b/README.md @@ -112,6 +112,7 @@ MOSS是一个支持中英双语和多种插件的开源对话语言模型，`mos ## :robot: 本地部署 + ### 硬件要求 下表提供了一个batch size=1时本地部署MOSS进行推理所需的显存大小。**量化模型暂时不支持模型并行。** @@ -123,6 +124,7 @@ MOSS是一个支持中英双语和多种插件的开源对话语言模型，`mos | Int4 | 7.8GB | 12GB | 26GB | ### 下载安装 + 1. 下载本仓库内容至本地/远程服务器 ```bash @@ -130,28 +132,22 @@ git clone https://github.com/OpenLMLab/MOSS.git cd MOSS ``` -2. 创建conda环境 +2. 创建 conda 环境 ```bash -conda create --name moss python=3.8 +conda env create --file conda-recipe.yaml # or `mamba env create --file conda-recipe.yaml` conda activate moss ``` -3. 安装依赖 - -```bash -pip install -r requirements.txt -``` - -4. (可选) 4/8-bit 量化环境 +3. (可选) 4/8-bit 量化环境 ```bash pip install triton ``` -其中`torch`和`transformers`版本不建议低于推荐版本。 +其中 `torch` 和 `transformers` 版本不建议低于推荐版本。 -目前triton仅支持Linux及WSL，暂不支持Windows及Mac OS，请等待后续更新。 +目前 triton 仅支持 Linux 及 WSL，暂不支持 Windows 及 macOS，请等待后续更新。 ### 使用示例 diff --git a/README_en.md b/README_en.md index 3c50cc0..dc3bdc8 100644 --- a/README_en.md +++ b/README_en.md @@ -108,6 +108,7 @@ MOSS is an open-sourced plugin-augmented conversational language model. `moss-mo ## :robot: Chat with MOSS + ### GPU Requirements The table below shows the minimal GPU memory required by performing MOSS inference when batch size is 1. Please note that **currently the quantized models do not support model parallism**. @@ -119,6 +120,7 @@ The table below shows the minimal GPU memory required by performing MOSS inferen | Int4 | 7.8GB | 12GB | 26GB | ### Installation + 1. Clone this repo to your local/remote machine. ```bash @@ -129,17 +131,11 @@ cd MOSS 2. Create a new conda environment ```bash -conda create --name moss python=3.8 +conda env create --file conda-recipe.yaml # or `mamba env create --file conda-recipe.yaml` conda activate moss ``` -3. Install requirements - -```bash -pip install -r requirements.txt -``` - -4. (Optional) 4/8-bit quantization requirement +3. (Optional) 4/8-bit quantization requirement ```bash pip install triton @@ -147,7 +143,7 @@ pip install triton Note that the version of `torch` and `transformers` should be equal or higher than recommended. -Currently triton only supports Linux and WSL. Please wait for later updates if you are using Windows/MacOS. +Currently triton only supports Linux and WSL. Please wait for later updates if you are using Windows/macOS. ### Try MOSS