Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

411

Base only

Active filters: llm-compressor

RedHatAI/Kimi-K3-FP8-BLOCK

2.8T • Updated 1 day ago • 26 • 4

RedHatAI/gemma-4-31B-it-FP8-block

Image-Text-to-Text • 31B • Updated 12 days ago • 5.7M • 43

sakamakismile/Agents-A1-4B-NVFP4

3B • Updated 9 days ago • 1.92k • 17

RedHatAI/Apertus-8B-Instruct-2509-FP8-dynamic

Text Generation • 8B • Updated Apr 28 • 1.54k • 5

bullpoint/Qwen3-Coder-Next-AWQ-4bit

Text Generation • 14B • Updated Feb 3 • 82k • 30

RedHatAI/Qwen3-Coder-Next-NVFP4

Text Generation • Updated Apr 28 • 79.4k • 34

apolo13x/Qwen3.5-35B-A3B-NVFP4

Image-Text-to-Text • Updated Feb 27 • 3.16k • 16

RedHatAI/Qwen3.5-9B-quantized.w8a8

Image-Text-to-Text • 9B • Updated Apr 16 • 132k • 2

ocicek/Qwen3.6-27B-NVFP4

Text Generation • 17B • Updated May 2 • 3.97k • 7

RedHatAI/Qwen3.5-4B-FP8-dynamic

Image-Text-to-Text • 5B • Updated May 11 • 37.9k • 3

OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destilled-abliterated-NVFP4

Image-Text-to-Text • 74B • Updated Jun 16 • 14.7k • 14

amd/Llama-3.3-70B-Instruct-w4a16-llmcompressor-v0.11.0

Text Generation • 11B • Updated Jun 17 • 587 • 1

sahilchachra/Unlimited-OCR-AWQ

Image-Text-to-Text • 3B • Updated Jun 23 • 2.35k • 1

xdavxd/gemma-4-31B-it-heretic-NVFP4

Image-Text-to-Text • 18B • Updated 7 days ago • 1.95k • 1

maci0/Ornith-1.0-35B-abliterated-NVFP4

Image-Text-to-Text • 20B • Updated 22 days ago • 429 • 1

sahilchachra/ThinkingCap-Qwen3.6-27B-AWQ

Text Generation • 28B • Updated 21 days ago • 1.21k • 3

sahilchachra/Qwythos-9B-v2-AWQ

Text Generation • 9B • Updated 12 days ago • 300 • 1

systemsofrecord/Qwen3-Coder-30B-A3B-Instruct-W4A16

Text Generation • 31B • Updated 7 days ago • 927 • 1

sahilchachra/KAT-Coder-V2.5-Dev-NVFP4A16

Text Generation • 20B • Updated 5 days ago • 379 • 2

RedHatAI/Qwen2.5-0.5B-quantized.w8a16

Text Generation • 0.6B • Updated Nov 26, 2024 • 2.27k

RedHatAI/Qwen2.5-1.5B-quantized.w8a16

Text Generation • 2B • Updated Nov 26, 2024 • 67

RedHatAI/Qwen2.5-3B-quantized.w8a16

Text Generation • 3B • Updated Nov 26, 2024 • 194

RedHatAI/Qwen2.5-7B-quantized.w8a16

Text Generation • 8B • Updated Nov 26, 2024 • 162 • 1

RedHatAI/Qwen2.5-32B-quantized.w8a16

Text Generation • 33B • Updated Nov 26, 2024 • 63

RedHatAI/Qwen2.5-72B-quantized.w8a16

Text Generation • 73B • Updated Nov 26, 2024 • 18

RedHatAI/Qwen2.5-Coder-14B-Instruct-FP8-dynamic

Text Generation • 15B • Updated Sep 23, 2025 • 39.3k • 1

RedHatAI/Phi-4-mini-instruct-FP8-dynamic

Text Generation • 4B • Updated Apr 28 • 514 • 1

jsbaicenter/r1-1776-distill-llama-70b-FP8-Dynamic

Text Generation • 71B • Updated Mar 18 • 7 • 1

divish/M-Prometheus-3B-FP8-Dynamic

Text Generation • 3B • Updated Apr 16, 2025 • 2

textgeflecht/Devstral-Small-2505-FP8-llmcompressor

Text Generation • 24B • Updated May 25, 2025 • 4