ed
/
rwkv.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #14 from hypnopump/update_macos 

Update macOS, better instructions, streaming output
This commit is contained in:
Alexander 2023-04-04 21:42:45 +05:00 committed by GitHub
parent 77e19980e9 d3801340f3
commit dc679bf971
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@ -57,6 +57,10 @@ cmake -DBUILD_SHARED_LIBS=ON .
cmake --build . --config Release
```
* **Anconda & M1 users**: Please verify that `CMAKE_SYSTEM_PROCESSOR: arm64` after running `cmake -DBUILD_SHARED_LIBS=ON .`
* If it detects `x86_64`, edit the `CMakeLists.txt` file under the `# Compile flags` to add `set(CMAKE_SYSTEM_PROCESSOR "arm64")`
If everything went OK, `librwkv.so` (Linux) or `rwkv.o` (MacOS) file should appear in the base repo folder.
@ -66,10 +70,10 @@ If everything went OK, `librwkv.so` (Linux) or `rwkv.o` (MacOS) file should appe
```commandline
# Windows
python rwkv\convert_rwkv_to_ggml.py C:\RWKV-4b-Pile-169M-20220807-8023.pth C:\rwkv.cpp-169M.bin float32
python rwkv\convert_rwkv_to_ggml.py C:\RWKV-4-Pile-169M-20220807-8023.pth C:\rwkv.cpp-169M.bin float16
# Linux / MacOS
python rwkv/convert_pytorch_to_ggml.py ~/Downloads/RWKV-4b-Pile-169M-20220807-8023.pth ~/Downloads/rwkv.cpp-169M.bin float32
python rwkv/convert_pytorch_to_ggml.py ~/Downloads/RWKV-4-Pile-169M-20220807-8023.pth ~/Downloads/rwkv.cpp-169M.bin float16
```
#### 3.1. Optionally, quantize the model

2
rwkv/chat_with_bot.py
View File

@ -79,7 +79,7 @@ while True:
decoded = tokenizer.decode([token])
print(decoded, end='')
print(decoded, end='', flush=True)
if '\n' in decoded:
break