| --- |
| base_model: |
| - nvidia/Cosmos-Reason2-2B |
| tags: |
| - nvidia |
| - cosmos |
| - cosmos-reason2 |
| - multimodal |
| - vlm |
| - quantized |
| - edge |
| - llmcompressor |
| - NVFP4 |
| pipeline_tag: image-text-to-text |
| license: other |
| license_name: embedl-models-community-licence-1.0 |
| license_link: https://github.com/embedl/embedl-models/blob/main/LICENSE |
| extra_gated_prompt: The information you provide will be collected, stored, processed |
| and shared in accordance with the [Embedl Privacy Policy](https://www.embedl.com/privacy-policy). |
| --- |
| <!-- embedl-banner:start --> |
| <style> |
| .embedl-btn-primary { transition: background 160ms ease, box-shadow 160ms ease; } |
| .embedl-btn-primary:hover { background: #4FDCE4 !important; box-shadow: 0 8px 22px rgba(45,212,221,0.45) !important; } |
| .embedl-btn-secondary { transition: background 160ms ease; } |
| .embedl-btn-secondary:hover { background: rgba(45,212,221,0.15) !important; } |
| .embedl-headline { font-size: clamp(11px, 2.15vw, 15px) !important; } |
| .embedl-btn-primary, .embedl-btn-secondary { |
| font-size: clamp(11px, 1.65vw, 13px) !important; |
| padding: clamp(6px, 1.1vw, 9px) clamp(10px, 1.6vw, 14px) !important; |
| } |
| </style> |
| <div style="background:radial-gradient(600px 220px at 0% 50%,rgba(45,212,221,0.22) 0%,rgba(45,212,221,0) 60%),radial-gradient(400px 180px at 100% 100%,rgba(45,212,221,0.10) 0%,rgba(45,212,221,0) 55%),linear-gradient(135deg,#0B1626 0%,#142338 100%);border:1px solid rgba(45,212,221,0.28);border-radius:12px;padding:22px 24px;margin:0 0 24px 0;color:#F2F6FA;box-shadow:0 4px 16px rgba(11,22,38,0.18);overflow:hidden;box-sizing:border-box;max-width:100%;"> |
| <table style="width:100%;border-collapse:collapse;border:0;background:transparent;"> |
| <tr style="background:transparent;"> |
| <td style="vertical-align:middle;border:0;padding:0;background:transparent;"> |
| <div style="display:inline-block;font-size:10px;letter-spacing:0.08em;text-transform:uppercase;font-weight:700;color:#2DD4DD;background:rgba(45,212,221,0.15);border:1px solid rgba(45,212,221,0.35);padding:4px 10px;border-radius:999px;margin-bottom:10px;white-space:nowrap;">Optimized by Embedl</div> |
| <div class="embedl-headline" style="font-size:15px;font-weight:700;line-height:1.35;color:#F2F6FA;margin-bottom:4px;">Need to <span style="color:#2DD4DD;white-space:nowrap;">fine-tune</span>, hit <span style="color:#2DD4DD;white-space:nowrap;">performance targets</span>, or deploy on <span style="color:#2DD4DD;white-space:nowrap;">specific hardware</span>?</div> |
| <div style="font-size:13px;color:#9BA7B5;">We've got you covered.</div> |
| </td> |
| <td width="1%" style="vertical-align:middle;border:0;padding:0 0 0 18px;white-space:nowrap;text-align:right;background:transparent;"> |
| <a href="https://www.embedl.com/models" class="embedl-btn-secondary" style="display:inline-block;font-size:13px;font-weight:600;padding:9px 14px;border-radius:6px;border:1px solid #2DD4DD;color:#2DD4DD;text-decoration:none;margin-right:8px;">Learn more</a> |
| <a href="https://www.embedl.com/contact" class="embedl-btn-primary" style="display:inline-block;font-size:13px;font-weight:600;padding:9px 14px;border-radius:6px;border:1px solid #2DD4DD;background:#2DD4DD;color:#0B1626;text-decoration:none;box-shadow:0 6px 18px rgba(45,212,221,0.28);">Get in touch →</a> |
| </td> |
| </tr> |
| </table> |
| </div> |
| <!-- embedl-banner:end --> |
| |
| # Cosmos-Reason2-2B-NVFP4A16 |
|
|
| **Optimized version of [nvidia/Cosmos-Reason2-2B](https://huggingface.co/nvidia/Cosmos-Reason2-2B) using |
| Quantization.** Optimized for reduced GPU memory usage and improved inference efficiency while |
| maintaining high-quality multimodal reasoning performance. |
|
|
| This model was created by quantizing the base model to **FP4 weights** while keeping activations |
| in **FP16** precision. |
| The model preserves the reasoning capabilities of the original Cosmos-Reason2-2B model while |
| significantly reducing the memory footprint of model weights. |
|
|
| NVFP4 quantization is a Blackwell-generation feature (e.g., supported in Jetson AGX Thor). |
| Earlier architectures (Ampere, Ada, Hopper) do not provide native hardware support for NVFP4 tensor core acceleration. |
|
|
|
|
| > [!NOTE] |
| > For more efficient inference, Embedl’s proprietary optimizations and architectural enhancements require patching vLLM. |
| > These updates will be released at a later date. |
| > For now, the model can be used with vLLM through the NVIDIA Jetson container. |
|
|
| > [!WARNING] |
| > NVFP4 quantization is a Blackwell-generation feature (e.g., supported in Jetson AGX Thor). |
| > Earlier architectures (Ampere, Ada, Hopper) |
| > do not provide native hardware support for NVFP4 tensor core acceleration. |
| > For systems not supporting FP4, we recommend using [embedl/Cosmos-Reason2-2B-W4A16](https://huggingface.co/embedl/Cosmos-Reason2-2B-W4A16). |
| > |
| > Utilization of NVFP4 quantization is backend-dependent. Not all versions of vLLM provide full or optimized support for NVFP4. |
|
|
| --- |
|
|
| ## Output Example |
|
|
| Real output captured from NVIDIA Jetson AGX Thor |
|
|
| <video |
| src="https://nvidia-cosmos.github.io/cosmos-cookbook/gallery/vs_assets/clip_1_short.mp4" |
| controls |
| autoplay |
| loop |
| muted |
| playsinline |
| width="720"> |
| </video> |
|
|
| > system: You are a helpful assistant. |
| > |
| > user: Describe this video in detail: https://nvidia-cosmos.github.io/cosmos-cookbook/gallery/vs_assets/clip_1_short.mp4 |
| |
| > assistant: |
| > The video captures a scene of a highway winding through a wooded area. The road is divided into two lanes, with vehicles traveling in both directions. |
| > Cars of various colors and models are seen moving along the highway, some maintaining a steady pace while others navigate curves with apparent ease. |
| > Drivers appear attentive, adhering to traffic regulations and ensuring safe navigation through the bends. Amidst this routine movement, |
| > a lone individual in a red jacket stands out, seemingly lost in thought or perhaps enjoying the solitude of the surroundings. |
| > |
| > On the right side of the highway, there is a pedestrian path that runs parallel to the road. This path is bordered by leafless trees, |
| > suggesting it might be late autumn or early spring. Sparse vegetation lines the edge of the path, blending seamlessly with the natural landscape. |
| > The trees stand tall and bare, their branches stark against the clear sky. |
| > |
| > Prominent in the foreground on the right side of the frame is a distinctive yellow and black striped pole, likely a warning or safety marker for drivers. |
| > The pole stands out due to its bright colors, contrasting sharply with the muted tones of the surrounding environment. [...] |
| |
| --- |
| |
| |
| ## Model Details |
| | **Field** | **Value** | |
| |--------------------|----------------------------------------------------------------------------------------------------------------------------| |
| | **Base Model** | [nvidia/Cosmos-Reason2-2B](https://huggingface.co/nvidia/Cosmos-Reason2-2B) | |
| | **Input / Output** | Text + Image / Video → Text | |
| | **Release Date** | 2026-02-24 | |
| | **Version** | 1.0 | |
| | **Optimizations** | Quantization (NVFP4A16) | |
| | **Developers** | Embedl | |
| | **Licenses** | Upstream: [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license). <br>Additional Information: [Apache License 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md). <br>Optimized components: Embedl Models Community Licence v1.0 *(no redistribution)*</br> | | |
| | **Intended Use** | Text generation, reasoning, assistant-style interaction, video analytics, planning, and general-purpose NLP on NVIDIA GPUs | |
| |
| <a href="https://hfviewer.com/embedl/Cosmos-Reason2-2B-NVFP4A16" target="_blank" rel="noopener"> |
| <img |
| src="https://hfviewer.com/api/card.svg?source=embedl%2FCosmos-Reason2-2B-NVFP4A16&granularity=auto&source_page=model-card-embed&community_showcase_opt_in=0" |
| alt="Architecture graph for embedl/Cosmos-Reason2-2B-NVFP4A16. Open in hfviewer" |
| width="100%" |
| /> |
| </a> |
| |
| --- |
| |
| |
| ## Optimizations |
| |
| - **Quantization (NVFP4A16)** - large reduction in memory footprint and latency. |
| |
| --- |
| |
| ## Accuracy |
| |
| For comparative evaluation, we present benchmark scores using the [Physical AI Bench Reason Task](https://huggingface.co/spaces/shi-labs/physical-ai-bench-leaderboard) . |
| |
| > [!WARNING] |
| > We have not been able to reproduce the baseline benchmarks reported by [nvidia/Cosmos-Reason2-2B](https://huggingface.co/nvidia/Cosmos-Reason2-2B) |
| > on the [Physical AI Bench Leaderboard](https://huggingface.co/spaces/shi-labs/physical-ai-bench-leaderboard), |
| > see related issue: https://github.com/nvidia-cosmos/cosmos-reason2/issues/52 |
| |
| ### Overall + Category Scores |
| |
| | Model | Overall | Embodied Reasoning | Common Sense | |
| |---------------------------------------------------------------------------------------------------|--------:|-------------------:|-------------:| |
| | [nvidia/Cosmos-Reason2-2B](https://huggingface.co/nvidia/Cosmos-Reason2-2B) | 50.60 | 53.93 | 47.19 | |
| | [**embedl/Cosmos-Reason2-2B-NVFP4A16**](https://huggingface.co/embedl/Cosmos-Reason2-2B-NVFP4A16) | 49.84 | 50.16 | 49.50 | |
| | [embedl/Cosmos-Reason2-2B-W4A16](https://huggingface.co/embedl/Cosmos-Reason2-2B-W4A16) | 48.68 | 50.49 | 46.85 | |
| | [embedl/Cosmos-Reason2-2B-W4A16-Edge2](https://huggingface.co/embedl/Cosmos-Reason2-2B-W4A16-Edge2) | 50.58 | 53.61 | 47.52 | |
| |
| ### Subcategory Scores |
| |
| | Model | AV | Physical World | Time | Space | Agibot | HoloAssist | RoboFail | RoboVQA | BridgeData V2 | |
| |---------------------------------------------------------------------------------------------------|------:|---------------:|------:|------:|-------:|-----------:|---------:|--------:|--------------:| |
| | [nvidia/Cosmos-Reason2-2B](https://huggingface.co/nvidia/Cosmos-Reason2-2B) | 44.00 | 46.90 | 45.30 | 55.00 | 34.00 | 60.00 | 49.00 | 90.91 | 42.00 | |
| | [**embedl/Cosmos-Reason2-2B-NVFP4A16**](https://huggingface.co/embedl/Cosmos-Reason2-2B-NVFP4A16) | 44.00 | 45.13 | 52.01 | 52.50 | 28.00 | 58.00 | 51.00 | 84.55 | 32.00 | |
| | [embedl/Cosmos-Reason2-2B-W4A16](https://huggingface.co/embedl/Cosmos-Reason2-2B-W4A16) | 36.00 | 47.79 | 44.30 | 53.75 | 36.00 | 61.00 | 42.00 | 80.91 | 44.00 | |
| | [embedl/Cosmos-Reason2-2B-W4A16-Edge2](https://huggingface.co/embedl/Cosmos-Reason2-2B-W4A16-Edge2) | 45.00 | 44.25 | 48.66 | 52.50 | 32.00 | 59.00 | 54.00 | 85.45 | 43.00 | |
| |
| --- |
| |
| ## Performance |
| |
| On-device performance benchmarks can be explored on [embedl/Edge-Inference-Benchmarks](https://huggingface.co/spaces/embedl/Edge-Inference-Benchmarks). |
| |
| <img src="https://huggingface.co/datasets/embedl/documentation-images/resolve/main/Cosmos-Reason2-2B-NVFP4A16/screenshot_edge_inference_benchmarks.png" alt="Screenshot Edge Inference Benchmarks" width="75%"> |
| |
| |
| --- |
| |
| |
| ## Usage Examples |
| |
| ### vLLM Video Inference |
| |
| **vLLM image:** [NVIDIA vLLM 26.01](https://docs.nvidia.com/deeplearning/frameworks/vllm-release-notes/rel-26-01.html#rel-26-01) |
| |
| **Test Hardware:** NVIDIA Jetson AGX Thor |
| |
| > [!NOTE] |
| > `--gpu-memory-utilization` and `--max-model-len` should be adapted to system specifications (i.e., available RAM). |
| |
| ```bash |
| docker run --rm -it \ |
| --network host \ |
| --shm-size=8g \ |
| --ulimit memlock=-1 \ |
| --ulimit stack=67108864 \ |
| --runtime=nvidia \ |
| --name=vllm-serve \ |
| -e HF_TOKEN=hf_*** \ |
| -e HF_HOME=/root/.cache/huggingface \ |
| nvcr.io/nvidia/vllm:26.01-py3 \ |
| vllm serve "embedl/Cosmos-Reason2-2B-NVFP4A16" \ |
| --host 0.0.0.0 \ |
| --port 8000 \ |
| --tensor-parallel-size 1 \ |
| --max-model-len 16384 \ |
| --gpu-memory-utilization 0.9 |
| ``` |
| |
| |
| > [!NOTE] |
| > `gpu_memory_utilization` and `max_num_seqs` should be adapted to system specifications (i.e., available RAM). |
| |
| |
| ```python |
| from vllm import LLM, SamplingParams |
| |
| if __name__ == "__main__": |
| |
| model = "embedl/Cosmos-Reason2-2B-NVFP4A16" |
| video_url = "https://nvidia-cosmos.github.io/cosmos-cookbook/gallery/vs_assets/clip_1_short.mp4" |
| |
| messages = [ |
| { |
| "role": "system", |
| "content": [ |
| {"type": "text", "text": "You are a helpful assistant."} |
| ], |
| }, |
| { |
| "role": "user", |
| "content": [ |
| { |
| "type": "video_url", |
| "video_url": {"url": video_url, "fps": 4}, |
| }, |
| { |
| "type": "text", |
| "text": "Describe this video in detail.", |
| }, |
| ], |
| }, |
| ] |
| |
| llm = LLM( |
| model=model, |
| limit_mm_per_prompt={ |
| "video": { |
| "count": 1, |
| "num_frames": 12, |
| "width": 1920, |
| "height": 1080, |
| }, |
| "image": 0, |
| "audio": 0, |
| }, |
| media_io_kwargs={"video": {"num_frames": -1}}, |
| max_model_len=16384, |
| mm_processor_kwargs={"truncation": False}, |
| disable_log_stats=False, |
| gpu_memory_utilization=0.9, |
| ) |
| |
| output = llm.chat( |
| messages, |
| sampling_params=SamplingParams(max_tokens=256), |
| ) |
| print(output[0].outputs[0].text) |
| |
| ``` |
| |
| ### Transformers Inference |
| |
| **Test Hardware:** NVIDIA H200 GPU |
| |
| Adapted from [nvidia/Cosmos-Reason2-2B](https://huggingface.co/nvidia/Cosmos-Reason2-2B). |
| |
| ```python |
| import torch |
| import transformers |
| |
| if __name__ == "__main__": |
| model_name = "embedl/Cosmos-Reason2-2B-NVFP4A16" |
| model = transformers.Qwen3VLForConditionalGeneration.from_pretrained( |
| model_name, |
| device_map="auto", |
| attn_implementation="sdpa", |
| dtype="bfloat16", |
| ) |
| processor: transformers.Qwen3VLProcessor = ( |
| transformers.AutoProcessor.from_pretrained(model_name) |
| ) |
| video_url = "https://nvidia-cosmos.github.io/cosmos-cookbook/gallery/vs_assets/clip_1_short.mp4" |
| |
| video_messages = [ |
| { |
| "role": "system", |
| "content": [ |
| {"type": "text", "text": "You are a helpful assistant."} |
| ], |
| }, |
| { |
| "role": "user", |
| "content": [ |
| {"type": "video", "video": video_url, "fps": 4}, |
| {"type": "text", "text": "Describe this video in detail."}, |
| ], |
| }, |
| ] |
| |
| # Process inputs |
| inputs = processor.apply_chat_template( |
| video_messages, |
| tokenize=True, |
| add_generation_prompt=True, |
| return_dict=True, |
| return_tensors="pt", |
| truncation=False, |
| fps=4, |
| ) |
| inputs = inputs.to(model.device) |
| |
| # Run inference |
| generated_ids = model.generate(**inputs, max_new_tokens=8192) |
| generated_ids_trimmed = [ |
| out_ids[len(in_ids) :] |
| for in_ids, out_ids in zip( |
| inputs.input_ids, generated_ids, strict=False |
| ) |
| ] |
| output_text = processor.batch_decode( |
| generated_ids_trimmed, |
| skip_special_tokens=True, |
| clean_up_tokenization_spaces=False, |
| ) |
| print(output_text[0]) |
| ``` |
| |
| --- |
|
|
| ## License |
|
|
| **Built on NVIDIA Cosmos** |
|
|
| This model is a derivative of **nvidia/Cosmos-Reason2-2B**. |
|
|
| > Licensed by NVIDIA Corporation under the NVIDIA Open Model License |
|
|
| - **Upstream:** [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license) |
| - **Additional Information:** [Apache License 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| - **Optimized Components:** Embedl Models Community Licence v1.0 *(no redistribution)* |
|
|
| --- |
|
|
| ## Contact |
|
|
| **Enterprise & Commercial Inquiries** |
| [contact@embedl.com](mailto:contact@embedl.com) |
|
|
| **Technical Issues & Early Access** |
| [https://github.com/embedl/embedl-models](https://github.com/embedl/embedl-models) |
|
|
| **More Information & Model Releases** |
| [https://embedl.com](https://embedl.com) |
|
|
| --- |
|
|
| ### Partner & Developer Opportunities |
|
|
| If you are evaluating on-device inference, building products on this model, or exploring custom model optimization, reach out for: |
|
|
| - Engineering support for on-prem/edge deployments |
| - Early access & partner co-marketing opportunities |
|
|
| Contact: [contact@embedl.com](mailto:contact@embedl.com) |
|
|
| --- |
|
|
| <!-- embedl-discord-banner:start --> |
| <style> |
| .embedl-discord-btn { transition: background 160ms ease, box-shadow 160ms ease; } |
| .embedl-discord-btn:hover { background: #6C77F5 !important; box-shadow: 0 8px 22px rgba(88,101,242,0.55) !important; } |
| </style> |
| <div style="background:radial-gradient(600px 220px at 0% 50%,rgba(88,101,242,0.22) 0%,rgba(88,101,242,0) 60%),radial-gradient(400px 180px at 100% 100%,rgba(88,101,242,0.10) 0%,rgba(88,101,242,0) 55%),linear-gradient(135deg,#0B1626 0%,#142338 100%);border:1px solid rgba(88,101,242,0.35);border-radius:12px;padding:22px 24px;margin:24px 0 0 0;color:#F2F6FA;box-shadow:0 4px 16px rgba(11,22,38,0.18);overflow:hidden;box-sizing:border-box;max-width:100%;"> |
| <table style="width:100%;border-collapse:collapse;border:0;background:transparent;"> |
| <tr style="background:transparent;"> |
| <td style="vertical-align:middle;border:0;padding:0;background:transparent;"> |
| <div style="display:inline-block;font-size:10px;letter-spacing:0.08em;text-transform:uppercase;font-weight:700;color:#A5B4FC;background:rgba(88,101,242,0.18);border:1px solid rgba(88,101,242,0.45);padding:4px 10px;border-radius:999px;margin-bottom:10px;white-space:nowrap;">Community & support</div> |
| <div style="font-size:15px;font-weight:700;line-height:1.35;color:#F2F6FA;margin-bottom:4px;">Need help with this model? Chat with the Embedl team and other engineers on <span style="color:#A5B4FC;white-space:nowrap;">Discord</span>.</div> |
| <div style="font-size:13px;color:#9BA7B5;">Quantization gotchas, hardware questions, fine-tuning tips — bring them all.</div> |
| </td> |
| <td width="1%" style="vertical-align:middle;border:0;padding:0 0 0 18px;white-space:nowrap;text-align:right;background:transparent;"> |
| <a href="https://discord.gg/MTbMWdKqE" class="embedl-discord-btn" style="display:inline-block;font-size:13px;font-weight:600;padding:9px 14px;border-radius:6px;border:1px solid #5865F2;background:#5865F2;color:#FFFFFF;text-decoration:none;box-shadow:0 6px 18px rgba(88,101,242,0.35);">Join our Discord →</a> |
| </td> |
| </tr> |
| </table> |
| </div> |
| <!-- embedl-discord-banner:end --> |
| |
