rwkv.cpp

Commit Graph

Author	SHA1	Message	Date
Mathmagician8191	82c4ac78f4	Add support for the world tokenizer (#86 ) * Add support for the world tokenizer * Move tokenizer logic to rwkv_tokenizer.py * Added test for the tokenizer	2023-06-08 16:37:18 +05:00
Alex	3587ff9e58	Sync ggml with upstream (#38 ) * Sync ggml with upstream * Remove file filters from Actions triggers * Update ggml * Add Q4_2 and Q4_3 support * Improve output of perplexity measuring script * Add tests for new formats * Add token limit argument to perplexity measuring script * Update README * Update README * Update ggml * Use master branch of ggml	2023-04-22 20:25:29 +05:00
saharNooby	85db23c7de	Add script that measures perplexity	2023-04-08 10:41:16 +04:00

Author

SHA1

Message

Date

Mathmagician8191

82c4ac78f4

Add support for the world tokenizer (#86 )

* Add support for the world tokenizer

* Move tokenizer logic to rwkv_tokenizer.py

* Added test for the tokenizer

2023-06-08 16:37:18 +05:00

Alex

3587ff9e58

Sync ggml with upstream (#38 )

* Sync ggml with upstream

* Remove file filters from Actions triggers

* Update ggml

* Add Q4_2 and Q4_3 support

* Improve output of perplexity measuring script

* Add tests for new formats

* Add token limit argument to perplexity measuring script

* Update README

* Update README

* Update ggml

* Use master branch of ggml

2023-04-22 20:25:29 +05:00

saharNooby

85db23c7de

Add script that measures perplexity

2023-04-08 10:41:16 +04:00

3 Commits