3587ff9e58
Sync ggml with upstream ( #38 )
...
* Sync ggml with upstream
* Remove file filters from Actions triggers
* Update ggml
* Add Q4_2 and Q4_3 support
* Improve output of perplexity measuring script
* Add tests for new formats
* Add token limit argument to perplexity measuring script
* Update README
* Update README
* Update ggml
* Use master branch of ggml
2023-04-22 20:25:29 +05:00
1be9fda248
Add robust automatic testing ( #33 )
2023-04-20 11:00:35 +05:00
05825d2370
Fix GitHub Actions
2023-04-17 19:04:55 +04:00
28e354c183
Delete Makefile and make workflows
2023-04-17 17:37:09 +04:00
5662bf4b4f
chore: make the asset file at the root of the zip file
2023-04-09 00:32:32 +09:00
a3fe1c63d8
chore: align asset file name
2023-04-09 00:21:30 +09:00
37f890ff3e
chore: update github action
...
chore: update github action
chore: update github action
2023-04-08 23:18:31 +09:00
f2b1dad22b
Add GitHub workflows file
2023-04-02 16:56:04 +04:00
1ecbad3a65
Remove unused files
2023-04-02 12:53:41 +04:00
9cbc404ba6
ci : re-enable AVX512 testing (Windows-MSVC) ( #584 )
...
* CI: Re-enable AVX512 testing (Windows-MSVC)
Now with 100% less base64 encoding
* plain __cpuid is enough here
2023-03-29 23:44:39 +03:00
f1217055ea
CI: fix subdirectory path globbing ( #546 )
...
- Changes in subdirectories will now be detecter properly
- (Windows-MSVC) AVX512 tests temporarily disabled
2023-03-28 22:43:25 +03:00
96f9c0506f
ci : make ctest verbose, hopefully we see what is wrong with the sanitizer
2023-03-28 20:01:09 +03:00
34c1072e49
ci: add debug build to sanitizer build matrix ( #527 )
2023-03-26 15:48:40 +00:00
8c2ec5e21d
Add support for linux/arm64 platform during Docker Builds ( #514 )
...
* Add support for linux/arm64 platform
* Add platform to versioned builds
2023-03-26 14:48:42 +00:00
19726169b3
CI: Run other sanitizer builds even if one fails ( #511 )
...
applies only to sanitizer builds so they wont be cancelled
2023-03-26 00:13:28 +02:00
2f7bf7dd7c
CMake / CI additions ( #497 )
...
* CMake: Add AVX512 option
* CI: Add AVX/AVX512 builds (Windows)
(AVX512 tests can only be run when the worker happens to support it, building works anyway)
* CMake: Fix sanitizer linkage ( merged #468 )
* CI: Add sanitizer builds (Ubuntu)
* CI: Fix release tagging
(change @zendesk/action-create-release to @anzz1/action-create-release until upstream PR Added commitish as input zendesk/action-create-release#32 is merged)
2023-03-25 23:38:11 +02:00
e4412b45e3
CI: CMake: Separate build and test steps ( #376 )
...
* CI: Separate Build and Test steps (CMake)
* CI: Make sure build passes before running tests (CMake)
* CI: Standardise step id names
2023-03-23 04:20:34 +02:00
69c92298a9
Deduplicate q4 quantization functions ( #383 )
...
* Deduplicate q4 quantization functions
* Use const; add basic test
* Re-enable quantization test
* Disable AVX2 flags in CI
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-03-22 19:29:06 +02:00
e6c9e0986c
Fix bin dir for win ci
2023-03-22 00:01:08 +02:00
01a297b099
specify build type for ctest on windows ( #371 )
2023-03-21 23:34:25 +02:00
eb34620aec
Add tokenizer test + revert to C++11 ( #355 )
...
* Add test-tokenizer-0 to do a few tokenizations - feel free to expand
* Added option to convert-pth-to-ggml.py script to dump just the vocabulary
* Added ./models/ggml-vocab.bin containing just LLaMA vocab data (used for tests)
* Added utility to load vocabulary file from previous point (temporary implementation)
* Avoid using std::string_view and drop back to C++11 (hope I didn't break something)
* Rename gpt_vocab -> llama_vocab
* All CMake binaries go into ./bin/ now
2023-03-21 17:29:41 +02:00
0f1b21cb90
Docker - Fix publish docker image in GitHub Registry ( #235 )
...
* fix publish permission
* try to fix docker pipeline using as password github_token & username repository_owner
2023-03-20 18:05:20 +01:00
b2de7f18df
CI Improvements ( #230 )
...
* CI Improvements
Manual build feature, autoreleases for Windows
* better CI naming convention
use branch name in releases and tags
2023-03-18 09:27:12 +02:00
6b0df5ccf3
add ptread link to fix cmake build under linux ( #114 )
...
* add ptread link to fix cmake build under linux
* add cmake to linux and macos platform
* separate make and cmake workflow
---------
Co-authored-by: Sebastián A <sebastian.aedo29@gmail.com>
2023-03-17 13:38:24 -03:00
2af23d3043
🚀 Dockerize llamacpp ( #132 )
...
* feat: dockerize llamacpp
* feat: split build & runtime stages
* split dockerfile into main & tools
* add quantize into tool docker image
* Update .devops/tools.sh
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* add docker action pipeline
* change CI to publish at github docker registry
* fix name runs-on macOS-latest is macos-latest (lowercase)
* include docker versioned images
* fix github action docker
* fix docker.yml
* feat: include all-in-one command tool & update readme.md
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-03-17 10:47:06 +01:00
2f700a2738
Add windows to the CI ( #98 )
2023-03-13 22:29:10 +02:00
2d555e5b42
Add CI ( #60 )
2023-03-12 22:08:24 +02:00
Alex
saharNooby
YorkZero
anzz1
Georgi Gerganov
Erik Scholz
Juan Calderon-Perez
Stephan Walter
Bernat Vadell
mmyjona
Sebastián A