diff --git a/README.md b/README.md index bf3eb0b76e1da..2233078556204 100644 --- a/README.md +++ b/README.md @@ -791,48 +791,59 @@ Finally, copy the `llama` binary and the model files to your device storage. Her https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4 -#### Building the Project using Termux (F-Droid) -Termux from F-Droid offers an alternative route to execute the project on an Android device. This method empowers you to construct the project right from within the terminal, negating the requirement for a rooted device or SD Card. +#### Building the Project in Termux (F-Droid) +[Termux](https://termux.dev/) is a way to run `llama.cpp` on Android devices. -Outlined below are the directives for installing the project using OpenBLAS and CLBlast. This combination is specifically designed to deliver peak performance on recent devices that feature a GPU. - -If you opt to utilize OpenBLAS, you'll need to install the corresponding package. +Ensure Termux is up to date and clone the repo: ``` -apt install libopenblas +apt update && apt upgrade +cd +git clone https://github.com/ggerganov/llama.cpp ``` -Subsequently, if you decide to incorporate CLBlast, you'll first need to install the requisite OpenCL packages: +Build `llama.cpp`: ``` -apt install ocl-icd opencl-headers opencl-clhpp clinfo +cd llama.cpp +make ``` -In order to compile CLBlast, you'll need to first clone the respective Git repository, which can be found at this URL: https://github.com/CNugteren/CLBlast. Alongside this, clone this repository into your home directory. Once this is done, navigate to the CLBlast folder and execute the commands detailed below: +It's possible to enable `OpenBlas` while building: ``` -cmake . -make -cp libclblast.so* $PREFIX/lib -cp ./include/clblast.h ../llama.cpp +pkg install libopenblas +cd llama.cpp +make LLAMA_OPENBLAS=1 ``` -Following the previous steps, navigate to the LlamaCpp directory. To compile it with OpenBLAS and CLBlast, execute the command provided below: +Move your model to the home directory (`~/`), for example: ``` -cp /data/data/com.termux/files/usr/include/openblas/cblas.h . -cp /data/data/com.termux/files/usr/include/openblas/openblas_config.h . -make LLAMA_CLBLAST=1 //(sometimes you need to run this command twice) +cd +cd storage/downloads +mv 7b-model.gguf.q4_0.bin ~/ ``` -Upon completion of the aforementioned steps, you will have successfully compiled the project. To run it using CLBlast, a slight adjustment is required: a command must be issued to direct the operations towards your device's physical GPU, rather than the virtual one. The necessary command is detailed below: +Usage example: ``` -GGML_OPENCL_PLATFORM=0 -GGML_OPENCL_DEVICE=0 -export LD_LIBRARY_PATH=/vendor/lib64:$LD_LIBRARY_PATH +./main -m ~/7b-model.gguf.q4_0.bin --color -c 2048 --keep -1 -n -2 -b 7 -ins -p 'Below is an instruction that describes a task. Write a response that appropriately completes the request.'\n\n'### Instruction:'\n'Hi!'\n\n'### Response:Hi! How may I assist you?' ``` -(Note: some Android devices, like the Zenfone 8, need the following command instead - "export LD_LIBRARY_PATH=/system/vendor/lib64:$LD_LIBRARY_PATH". Source: https://www.reddit.com/r/termux/comments/kc3ynp/opencl_working_in_termux_more_in_comments/ ) +For building with `OpenCL` then install the requisite packages: +``` +pkg install ocl-icd opencl-headers clblast +cd llama.cpp +make LLAMA_CLBLAST=1 +``` -For easy and swift re-execution, consider documenting this final part in a .sh script file. This will enable you to rerun the process with minimal hassle. +Use one of the following to enable GPU: +``` +export LD_LIBRARY_PATH=/vendor/lib64 +``` +or +``` +export LD_LIBRARY_PATH=/system/vendor/lib64 +``` +then `./main ... --gpu-layers 1` -Place your desired model into the `~/llama.cpp/models/` directory and execute the `./main (...)` script. +(Note: Use `unset LD_LIBRARY_PATH` to re-link executables.) ### Docker