Fill-Mask
Transformers
ONNX
Safetensors
Bashkir
bashkir-roberta-preln
bashkir
masked-language-modeling
roberta
sentencepiece
custom-code
onnxruntime
custom_code
Instructions to use failed09/bashkir-roberta with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use failed09/bashkir-roberta with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="failed09/bashkir-roberta", trust_remote_code=True)# Load model directly from transformers import AutoModelForMaskedLM model = AutoModelForMaskedLM.from_pretrained("failed09/bashkir-roberta", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update model card: add ONNX runtime guide and application use cases
Browse files
README.md
CHANGED
|
@@ -10,13 +10,15 @@ tags:
|
|
| 10 |
- roberta
|
| 11 |
- sentencepiece
|
| 12 |
- custom-code
|
|
|
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
# BashkirRoBERTa
|
| 16 |
|
| 17 |
A masked language model for Bashkir. Given a sentence with one `[MASK]` token,
|
| 18 |
it predicts the most probable missing Bashkir token from its context. The model
|
| 19 |
-
is useful for fill-mask experiments and as a
|
| 20 |
fine-tuning.
|
| 21 |
|
| 22 |
## Examples
|
|
@@ -28,7 +30,7 @@ fine-tuning.
|
|
| 28 |
| `Өфө — ҙур [MASK].` | `ҡала` |
|
| 29 |
| `Бөгөн Өфөлә яңы [MASK] асылды.` | `мәктәп` |
|
| 30 |
|
| 31 |
-
## Model
|
| 32 |
|
| 33 |
| Property | Value |
|
| 34 |
| --- | --- |
|
|
@@ -56,7 +58,7 @@ accuracy for masked subword prediction.
|
|
| 56 |
These are diagnostic MLM results, not a general-purpose language-understanding
|
| 57 |
score. A mask may represent a whole word or a SentencePiece subword fragment.
|
| 58 |
|
| 59 |
-
## Loading
|
| 60 |
|
| 61 |
```python
|
| 62 |
from transformers import AutoModelForMaskedLM, AutoTokenizer
|
|
@@ -76,6 +78,44 @@ print(tokenizer.decode([prediction_id])) # яратам
|
|
| 76 |
Pre-LayerNorm architecture rather than using the stock post-LayerNorm RoBERTa
|
| 77 |
implementation.
|
| 78 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
## License and provenance
|
| 80 |
|
| 81 |
The checkpoint is released under custom terms (`other` on the Hub) while the
|
|
|
|
| 10 |
- roberta
|
| 11 |
- sentencepiece
|
| 12 |
- custom-code
|
| 13 |
+
- onnx
|
| 14 |
+
- onnxruntime
|
| 15 |
---
|
| 16 |
|
| 17 |
# BashkirRoBERTa
|
| 18 |
|
| 19 |
A masked language model for Bashkir. Given a sentence with one `[MASK]` token,
|
| 20 |
it predicts the most probable missing Bashkir token from its context. The model
|
| 21 |
+
is useful for fill-mask experiments, spellchecking, and as a foundation for further
|
| 22 |
fine-tuning.
|
| 23 |
|
| 24 |
## Examples
|
|
|
|
| 30 |
| `Өфө — ҙур [MASK].` | `ҡала` |
|
| 31 |
| `Бөгөн Өфөлә яңы [MASK] асылды.` | `мәктәп` |
|
| 32 |
|
| 33 |
+
## Model Architecture
|
| 34 |
|
| 35 |
| Property | Value |
|
| 36 |
| --- | --- |
|
|
|
|
| 58 |
These are diagnostic MLM results, not a general-purpose language-understanding
|
| 59 |
score. A mask may represent a whole word or a SentencePiece subword fragment.
|
| 60 |
|
| 61 |
+
## PyTorch Loading (Transformers)
|
| 62 |
|
| 63 |
```python
|
| 64 |
from transformers import AutoModelForMaskedLM, AutoTokenizer
|
|
|
|
| 78 |
Pre-LayerNorm architecture rather than using the stock post-LayerNorm RoBERTa
|
| 79 |
implementation.
|
| 80 |
|
| 81 |
+
## ⚡ ONNX Runtime (Fast CPU & Edge Deployment)
|
| 82 |
+
|
| 83 |
+
For resource-constrained devices, edge environments, and production without heavy PyTorch dependencies,
|
| 84 |
+
pre-compiled ONNX models are available in the `onnx/` folder:
|
| 85 |
+
|
| 86 |
+
- **`onnx/model_fp16.onnx`** (95.4 MB): Recommended for GPU and DirectML acceleration.
|
| 87 |
+
- **`onnx/model_int8.onnx`** (58.1 MB): Quantized INT8 checkpoint for ultra-fast CPU, server, and mobile (Android/iOS) inference with negligible accuracy difference.
|
| 88 |
+
|
| 89 |
+
```python
|
| 90 |
+
import numpy as np
|
| 91 |
+
import onnxruntime as ort
|
| 92 |
+
import sentencepiece as spm
|
| 93 |
+
from huggingface_hub import hf_hub_download
|
| 94 |
+
|
| 95 |
+
# Download lightweight INT8 model and tokenizer (~58 MB total)
|
| 96 |
+
model_path = hf_hub_download("failed09/bashkir-roberta", "onnx/model_int8.onnx")
|
| 97 |
+
sp_path = hf_hub_download("failed09/bashkir-roberta", "spm_bashkir_bert_16k.model")
|
| 98 |
+
|
| 99 |
+
session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
|
| 100 |
+
sp = spm.SentencePieceProcessor(model_file=sp_path)
|
| 101 |
+
|
| 102 |
+
# Tokenize sentence with [MASK]
|
| 103 |
+
tokens = [2] + sp.encode("Мин башҡорт телен ") + [6] + sp.encode(".") + [3]
|
| 104 |
+
mask_idx = tokens.index(6)
|
| 105 |
+
inputs = {"input_ids": np.array([tokens], dtype=np.int64)}
|
| 106 |
+
|
| 107 |
+
logits = session.run(None, inputs)[0][0, mask_idx]
|
| 108 |
+
top_tokens = np.argsort(logits)[::-1][:5]
|
| 109 |
+
print([sp.decode([int(t)]) for t in top_tokens]) # ['яратам', 'беләм', 'өйрәнә', ...]
|
| 110 |
+
```
|
| 111 |
+
|
| 112 |
+
## 🎯 Target Applications
|
| 113 |
+
|
| 114 |
+
- **Grammar & Spellchecking:** Contextual candidate ranking, detection of morphological errors and vowel harmony violations.
|
| 115 |
+
- **Cloze & Multiple-Choice Testing:** Automated solving and candidate evaluation for Bashkir educational tests.
|
| 116 |
+
- **OCR Post-Correction:** Resolving noisy characters and ambiguous glyphs in digitized historical print.
|
| 117 |
+
- **Feature Extraction & Fine-Tuning:** Backbone representations for Bashkir text classification, sentiment analysis, and NER.
|
| 118 |
+
|
| 119 |
## License and provenance
|
| 120 |
|
| 121 |
The checkpoint is released under custom terms (`other` on the Hub) while the
|