Update README.md
Browse files
README.md
CHANGED
|
@@ -8,4 +8,43 @@ pipeline_tag: text-generation
|
|
| 8 |
library_name: transformers
|
| 9 |
tags:
|
| 10 |
- text-generation-inference
|
| 11 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
library_name: transformers
|
| 9 |
tags:
|
| 10 |
- text-generation-inference
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# Vulpecula-4B-GGUF
|
| 14 |
+
|
| 15 |
+
> **Vulpecula-4B** is fine-tuned based on the traces of **SK1.1**, consisting of the same 1,000 entries of the **DeepSeek thinking trajectory**, along with fine-tuning on **Fine-Tome 100k** and **Open Math Reasoning** datasets. This specialized 4B parameter model is designed for enhanced mathematical reasoning, logical problem-solving, and structured content generation, optimized for precision and step-by-step explanation.
|
| 16 |
+
|
| 17 |
+
## Model Files
|
| 18 |
+
|
| 19 |
+
| File Name | Size | Quantization | Format | Description |
|
| 20 |
+
| -------------------------- | ------- | ------------ | ------ | ----------------------------- |
|
| 21 |
+
| `Vulpecula-4B.F16.gguf` | 8.05 GB | FP16 | GGUF | Float16 precision version |
|
| 22 |
+
| `Vulpecula-4B.Q4_K_M.gguf` | 2.5 GB | Q4\_K\_M | GGUF | 4-bit quantized (K M variant) |
|
| 23 |
+
| `Vulpecula-4B.Q5_K_M.gguf` | 2.89 GB | Q5\_K\_M | GGUF | 5-bit quantized (K M variant) |
|
| 24 |
+
| `Vulpecula-4B.Q8_0.gguf` | 4.28 GB | Q8\_0 | GGUF | 8-bit quantized |
|
| 25 |
+
| `.gitattributes` | 1.8 kB | — | — | Git LFS tracking file |
|
| 26 |
+
| `README.md` | 31 B | — | — | Model documentation |
|
| 27 |
+
|
| 28 |
+
## Quants Usage
|
| 29 |
+
|
| 30 |
+
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
| 31 |
+
|
| 32 |
+
| Link | Type | Size/GB | Notes |
|
| 33 |
+
|:-----|:-----|--------:|:------|
|
| 34 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q2_K.gguf) | Q2_K | 0.4 | |
|
| 35 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q3_K_S.gguf) | Q3_K_S | 0.5 | |
|
| 36 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q3_K_M.gguf) | Q3_K_M | 0.5 | lower quality |
|
| 37 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q3_K_L.gguf) | Q3_K_L | 0.5 | |
|
| 38 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.IQ4_XS.gguf) | IQ4_XS | 0.6 | |
|
| 39 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_S.gguf) | Q4_K_S | 0.6 | fast, recommended |
|
| 40 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf) | Q4_K_M | 0.6 | fast, recommended |
|
| 41 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q5_K_S.gguf) | Q5_K_S | 0.6 | |
|
| 42 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q5_K_M.gguf) | Q5_K_M | 0.7 | |
|
| 43 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q6_K.gguf) | Q6_K | 0.7 | very good quality |
|
| 44 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q8_0.gguf) | Q8_0 | 0.9 | fast, best quality |
|
| 45 |
+
| [GGUF](https://huggingface.co/mradermacher/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.f16.gguf) | f16 | 1.6 | 16 bpw, overkill |
|
| 46 |
+
|
| 47 |
+
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
| 48 |
+
types (lower is better):
|
| 49 |
+
|
| 50 |
+
|