StarVLA/Qwen3-VL-4B-Instruct-Action

Qwen3-VL-4B-Instruct with action tokens

This repository is a Qwen3-VL-4B-Instruct initializer whose tokenizer and text vocabulary have been extended with 2,048 tokens named <robot_action_0> through <robot_action_2047>. It is intended as a starting point for StarVLA frameworks that discretize robot actions as tokens; it is not a trained robot policy.

Model and input contract

Item	Value
Base model	Qwen/Qwen3-VL-4B-Instruct
Architecture	Qwen3VLForConditionalGeneration
Added action tokens	2,048
Action-token IDs	151669–153716 in the packaged tokenizer
Inputs	Same text, image, and video inputs as the base Qwen3-VL model
Output	Language-model logits over the extended vocabulary
Robot action head	None
Robot checkpoint	None

The extension was produced with starVLA/model/modules/vlm/tools/add_qwen_special_tokens.

Training and evaluation

No additional VLM or robot-policy fine-tuning is reported for this repository. The pre-existing Qwen3-VL parameters are retained apart from resizing the token embedding/output vocabulary for the new entries. The added action-token embeddings therefore should be treated as initialization, not as learned action semantics.

No robot benchmark result is associated with this repository.

Loading

Use the model and tokenizer from the same repository:

from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

repo_id = "StarVLA/Qwen3-VL-4B-Instruct-Action"
processor = AutoProcessor.from_pretrained(repo_id)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    repo_id,
    torch_dtype="auto",
    device_map="auto",
)

Downstream training must define the action quantization/codebook that maps continuous robot actions to the 2,048 token indices.

Evidence and limitations

• config.json verifies Qwen3VLForConditionalGeneration; added_tokens.json verifies all 2,048 action-token entries.
• This repository does not define normalization, action dimension, action horizon, robot embodiment, dataset, controller, or evaluation protocol.
• Token IDs are repository-specific. Mixing this tokenizer with the unextended base model or another action-token checkpoint can silently corrupt decoding.
• The model inherits the base model license, acceptable-use requirements, biases, and failure modes.