Create README.md

README.md

@@ -0,0 +1,171 @@
+---
+license: apache-2.0
+language:
+  - ar
+  - en
+pipeline_tag: text-generation
+tags:
+  - text-generation
+  - pytorch
+  - transformers
+  - vllm
+  - causal-lm
+  - depth-extension
+  - arabic
+  - english
+  - Karnak-6B-v1.0 
+  - qwen
+base_model: Qwen/Qwen3-4B-Instruct-2507
+model_name: Karnak-6B-v1.0 
+parameters: 6B
+inference: false
+---
+
+# Karnak-6B-v1.0 : Enhanced Arabic–English Large Language Model
+Karnak-6B-v1.0  is a powerful AI model that works in both Arabic and English, with extra improvements that make it especially strong in Arabic and more natural in the way it writes and responds. It was built by taking an existing model and improving it through more training, so it can understand instructions better, handle longer text, and give more reliable answers. This makes it useful for everyday tasks like answering questions, explaining topics, writing content, or helping with work and research. It can also process long pieces of text, which is helpful for documents and extended conversations. A big advantage is that it is not locked to an online service only, since you can download it, run it locally on your own machine or servers, and even fine-tune it for your own specific use case. 
+
+## Model Summary
+
+**Karnak-6B-v1.0 ** is a depth-extended causal language model optimized for **Arabic and English** generation. It is built on top of **Qwen/Qwen3-4B-Instruct-2507**, featuring architectural depth extension and a tokenizer specifically optimized for Arabic to improve fluency and efficiency.
+
+Karnak-6B-v1.0  was trained using **high-quality, filtered data** through a rigorous pipeline to enhance overall instruction-following capabilities, factuality, and robustness.
+
+## Key Features
+
+- **Depth Extension (~6B):** Expanded depth to increase reasoning capacity and improve long-range dependency modeling.
+- **Arabic-Optimized Tokenizer:** Improved Arabic tokenization efficiency, resulting in reduced token fragmentation and higher-quality generation.
+- **Multi-Stage Training:** The model evolved through: Pre-trained weights → Depth Extension → Continued Pre-training → SFT (Supervised Fine-Tuning).
+- **Extended Context Window:** Designed for long-context usage with a **safe context range up to 20K tokens** (recommended to stay within this limit for optimal stability).
+
+## Model Details
+
+- **Model Name:** Karnak-6B-v1.0 
+- **Base Model:** Qwen/Qwen3-4B-Instruct-2507
+- **Parameter Count:** ~6B (Depth-Extended)
+- **Languages:** Arabic, English
+- **Training:** High-quality filtered data + Multi-stage pipeline (Continued pre-training + SFT)
+- **Safe Context Range:** Up to **20,000 tokens**
+
+---
+
+## Usage
+
+### 1) Hugging Face Transformers
+
+To use Karnak-6B-v1.0  with the standard Transformers library, ensure you have the latest version installed.
+
+```bash
+pip install -U "transformers>=4.40.0" torch accelerate
+```
+
+Python Code Example (Chat Template):
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "Applied-Innovation-Center/Karnak-6B-v1.0 "
+
+# Load tokenizer and model
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="auto",
+    torch_dtype=torch.bfloat16,
+    trust_remote_code=True,
+)
+
+# Prepare Input
+prompt = "اشرح لي نظرية النسبية بشكل مبسط."
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": prompt},
+]
+
+# Apply chat template
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+# Generate
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=512,
+    temperature=0.7,
+    top_p=0.9,
+)
+
+# Decode output (removing the prompt tokens)
+generated_ids = generated_ids[:, model_inputs.input_ids.shape[1]:]
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+print(response)
+```
+
+2) vLLM (Recommended for Production)
+Karnak-6B-v1.0  is compatible with vLLM for high-throughput inference.
+
+Installation:
+
+```bash
+
+pip install -U vllm
+```
+
+Offline Inference:
+
+```python
+
+from vllm import LLM, SamplingParams
+
+model_id = "Applied-Innovation-Center/Karnak-6B-v1.0 "
+
+# Initialize the model
+llm = LLM(
+    model=model_id,
+    trust_remote_code=True,
+    max_model_len=20000,  # Safe context range
+    tensor_parallel_size=1,  # Adjust based on available GPUs
+)
+
+# Set sampling parameters
+sampling_params = SamplingParams(
+    temperature=0.7,
+    top_p=0.9,
+    max_tokens=512,
+)
+
+# Generate
+prompts = ["ما هي عاصمة مصر؟"]
+outputs = llm.generate(prompts, sampling_params)
+
+for o in outputs:
+    print(f"Prompt: {o.prompt}")
+    print(f"Generated: {o.outputs[0].text}")
+```
+
+Server Mode (OpenAI-Compatible API):
+
+You can serve the model as an API compatible with OpenAI clients:
+
+```bash
+
+vllm serve "Applied-Innovation-Center/Karnak-6B-v1.0 " \
+  --trust-remote-code \
+  --dtype bfloat16 \
+  --port 8000
+```
+
+Citation
+If you use this model in your research or application, please cite it as follows:
+
+```bibtex
+@misc{Karnak-6B-v1.0 -6B,
+  title={Karnak-6B-v1.0 : A Depth-Extended Arabic-English LLM},
+  year={2026},
+  publisher={Applied Innovation Center},
+  howpublished={\url{[https://huggingface.co/Applied-Innovation-Center/Karnak-6B-v1.0 ](https://huggingface.co/Applied-Innovation-Center/Karnak-6B-v1.0 )}}
+}
+```