inference-optimization/Llama-3.1-8B-Instruct-QKV-Cache-FP8-Per-Head
8B
•
Updated
•
25
KV Cache Quantization
Collection on FP8 Quantization of Weights, Activations and KV Cache
-
-
inference-optimization/Llama-3.1-8B-Instruct-QKV-Cache-FP8-Per-Tensor
8B • Updated • 9 -
inference-optimization/Llama-3.1-8B-Instruct-FP8-dynamic-QKV-Cache-FP8-Per-Head
8B • Updated • 9 -
inference-optimization/Llama-3.1-8B-Instruct-FP8-dynamic-QKV-Cache-FP8-Per-Tensor
8B • Updated • 4 -
inference-optimization/Llama-3.3-70B-Instruct-QKV-Cache-FP8-Per-Head
71B • Updated • 6 -
inference-optimization/Llama-3.3-70B-Instruct-QKV-Cache-FP8-Per-Tensor
71B • Updated • 6 -
inference-optimization/Llama-3.3-70B-Instruct-FP8-dynamic-QKV-Cache-FP8-Per-Head
71B • Updated • 9 -
inference-optimization/Llama-3.3-70B-Instruct-FP8-dynamic-QKV-Cache-FP8-Per-Tensor
71B • Updated • 9 -
inference-optimization/Qwen3-32B-QKV-Cache-FP8-Per-Head
33B • Updated • 8 -
inference-optimization/Qwen3-32B-QKV-Cache-FP8-Per-Tensor
33B • Updated • 9 -
inference-optimization/Qwen3-32B-FP8-dynamic-QKV-Cache-FP8-Per-Head
33B • Updated • 5 -
inference-optimization/Qwen3-32B-FP8-dynamic-QKV-Cache-FP8-Per-Tensor
33B • Updated • 9