readme : add GPT4All instructions (close #588)
This commit is contained in:
		
							parent
							
								
									53635c081c
								
							
						
					
					
						commit
						516d88e75c
					
				
							
								
								
									
										21
									
								
								README.md
								
								
								
								
							
							
						
						
									
										21
									
								
								README.md
								
								
								
								
							|  | @ -10,9 +10,7 @@ Inference of [LLaMA](https://arxiv.org/abs/2302.13971) model in pure C/C++ | ||||||
| **Hot topics:** | **Hot topics:** | ||||||
| 
 | 
 | ||||||
| - [Roadmap (short-term)](https://github.com/ggerganov/llama.cpp/discussions/457) | - [Roadmap (short-term)](https://github.com/ggerganov/llama.cpp/discussions/457) | ||||||
| - New C-style API is now available: https://github.com/ggerganov/llama.cpp/pull/370 | - Support for [GPT4All](https://github.com/ggerganov/llama.cpp#using-gpt4all) | ||||||
| - Cache input prompts for faster initialization: https://github.com/ggerganov/llama.cpp/issues/64 |  | ||||||
| - Create a `llama.cpp` logo: https://github.com/ggerganov/llama.cpp/issues/105 |  | ||||||
| 
 | 
 | ||||||
| ## Description | ## Description | ||||||
| 
 | 
 | ||||||
|  | @ -37,6 +35,12 @@ Supported platforms: | ||||||
| - [X] Windows (via CMake) | - [X] Windows (via CMake) | ||||||
| - [X] Docker | - [X] Docker | ||||||
| 
 | 
 | ||||||
|  | Supported models: | ||||||
|  | 
 | ||||||
|  | - [X] LLaMA | ||||||
|  | - [X] [Alpaca](https://github.com/ggerganov/llama.cpp#instruction-mode-with-alpaca) | ||||||
|  | - [X] [GPT4All](https://github.com/ggerganov/llama.cpp#using-gpt4all) | ||||||
|  | 
 | ||||||
| --- | --- | ||||||
| 
 | 
 | ||||||
| Here is a typical run using LLaMA-7B: | Here is a typical run using LLaMA-7B: | ||||||
|  | @ -222,6 +226,17 @@ cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach. | ||||||
| >  | >  | ||||||
| ``` | ``` | ||||||
| 
 | 
 | ||||||
|  | ### Using [GPT4All](https://github.com/nomic-ai/gpt4all) | ||||||
|  | 
 | ||||||
|  | - Obtain the `gpt4all-lora-quantized.bin` model | ||||||
|  | - It is distributed in the old `ggml` format which is not obsoleted. So you have to convert it to the new format using [./convert-gpt4all-to-ggml.py](./convert-gpt4all-to-ggml.py): | ||||||
|  | 
 | ||||||
|  |   ```bash | ||||||
|  |   python3 convert-gpt4all-to-ggml.py models/gpt4all-7B/gpt4all-lora-quantized.bin ./models/tokenizer.model  | ||||||
|  |   ``` | ||||||
|  |    | ||||||
|  | - You can now use the newly generated `gpt4all-lora-quantized.bin` model in exactly the same way as all other models. The original model is stored in the same folder with a suffix `.orig` | ||||||
|  | 
 | ||||||
| ### Obtaining and verifying the Facebook LLaMA original model and Stanford Alpaca model data | ### Obtaining and verifying the Facebook LLaMA original model and Stanford Alpaca model data | ||||||
| 
 | 
 | ||||||
| - **Under no circumstances share IPFS, magnet links, or any other links to model downloads anywhere in this respository, including in issues, discussions or pull requests. They will be immediately deleted.** | - **Under no circumstances share IPFS, magnet links, or any other links to model downloads anywhere in this respository, including in issues, discussions or pull requests. They will be immediately deleted.** | ||||||
|  |  | ||||||
		Loading…
	
		Reference in New Issue