Commit Graph

5 Commits

Author SHA1 Message Date
Mathmagician8191 82c4ac78f4
Add support for the world tokenizer (#86)
* Add support for the world tokenizer

* Move tokenizer logic to rwkv_tokenizer.py

* Added test for the tokenizer
2023-06-08 16:37:18 +05:00
LoganDark b61d94aef0
Flush output every token in generate_completions.py (#73) 2023-05-26 17:23:58 +05:00
hypnopump 6f3fb01913
suggestions 2023-04-03 08:25:54 +02:00
hypnopump a64aaa81ec
initial addition 2023-04-03 00:52:26 +02:00
saharNooby e0684e8104 Add text generation and chat scripts 2023-04-02 15:03:31 +04:00