Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

kv-cache-quantization

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

209

Base only

Active filters: kv-cache-quantization

Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV

Text Generation • 13B • Updated 2 days ago • 464 • 1

VAREQON/Nemotron-3-Nano-4B-RotorQuant-GGUF-Q2_K

Text Generation • 4B • Updated about 4 hours ago • 1

kuzyn8/Nemotron-3-Nano-4B-RotorQuant-GGUF-Q2_K

Text Generation • 4B • Updated about 5 hours ago • 2

rungalileo/llama-3.2-3B-instruct-trtllm-ckpt-wq_int4_awq-kv_int8

Text Generation • Updated Feb 8 • 9

rungalileo/mistral-7b-instruct-v0.3-trtllm-ckpt-wq_int4_awq-kv_int8

Text Generation • Updated Feb 8 • 3

rungalileo/llama-3.1-8b-instruct-trtllm-ckpt-wq_fp8-kv_fp8

Text Generation • Updated Mar 16 • 8

rungalileo/mistral-7b-instruct-v0.3-trtllm-ckpt-wq_fp8-kv_fp8

Text Generation • Updated Mar 16 • 10

rungalileo/llama-3.2-3b-instruct-trtllm-ckpt-wq_fp8-kv_fp8

Text Generation • Updated Apr 8 • 10

rungalileo/llama-3.2-3b-instruct-trtllm-ckpt-wq_nvfp4-kv_fp8

Text Generation • Updated Mar 16 • 9

rungalileo/llama-3.1-8b-instruct-trtllm-ckpt-wq_nvfp4-kv_fp8

Text Generation • Updated Mar 16 • 10

rungalileo/mistral-7b-instruct-v0.3-trtllm-ckpt-wq_nvfp4-kv_fp8

Text Generation • Updated Mar 16 • 4

rungalileo/llama-3.1-8b-instruct-trtllm-ckpt-wq_w8a8sq-kv_int8

Text Generation • Updated Apr 2 • 4

rungalileo/mistral-7b-instruct-v0.3-trtllm-ckpt-wq_w8a8sq-kv_int8

Text Generation • Updated Apr 2 • 4

majentik/gemma-4-E4B-turboquant

Image-Text-to-Text • 8B • Updated 10 days ago • 75 • 8

majentik/Qwen3.5-27B-TurboQuant-MLX-4bit

Text Generation • 27B • Updated 10 days ago • 228 • 2

majentik/Qwen3.5-27B-TurboQuant-MLX-8bit

Text Generation • 27B • Updated 10 days ago • 133

majentik/Qwen3.5-27B-RotorQuant-MLX-4bit

Text Generation • 27B • Updated 10 days ago • 195

majentik/Qwen3.5-27B-RotorQuant-MLX-8bit

Text Generation • 27B • Updated 10 days ago • 203

majentik/Mistral-Small-4-119B-TurboQuant-MLX-4bit

Text Generation • 19B • Updated 10 days ago • 379

majentik/Mistral-Small-4-119B-TurboQuant-MLX-2bit

Text Generation • 12B • Updated 10 days ago • 512 • 1

majentik/Mistral-Small-4-119B-RotorQuant-MLX-2bit

Text Generation • 12B • Updated 10 days ago • 316

majentik/Leanstral-TurboQuant-MLX-2bit

Text Generation • 23B • Updated 10 days ago • 403

majentik/Leanstral-RotorQuant-MLX-2bit

Text Generation • 23B • Updated 10 days ago • 378

majentik/Mistral-Small-4-119B-RotorQuant-MLX-4bit

Text Generation • 19B • Updated 10 days ago • 291

majentik/gemma-4-31B-TurboQuant-MLX-4bit

Image-Text-to-Text • 5B • Updated 10 days ago • 374 • 2

majentik/gemma-4-31B-TurboQuant-MLX-2bit

Image-Text-to-Text • 3B • Updated 10 days ago • 346 • 1

majentik/gemma-4-31B-TurboQuant-MLX-8bit

Image-Text-to-Text • 9B • Updated 10 days ago • 433 • 1

majentik/gemma-4-31B-RotorQuant-MLX-4bit

Image-Text-to-Text • 5B • Updated 10 days ago • 125

majentik/gemma-4-31B-RotorQuant-MLX-8bit

Image-Text-to-Text • 9B • Updated 10 days ago • 498 • 1

majentik/gemma-4-31B-RotorQuant-MLX-2bit

Image-Text-to-Text • 3B • Updated 10 days ago • 157