---
license: cc-by-4.0
datasets:
- DSL-13-SRMAP/Telugu-Dataset
language:
- te
tags:
- sentiment-analysis
- text-classification
- telugu
- multilingual
- xlm-roberta
- rationale-supervision
- explainable-ai
base_model: xlm-roberta-base
pipeline_tag: text-classification
metrics:
- accuracy
- f1
- auroc
---

# XLM-R_WR

## Model Description

**XLM-R_WR** is a Telugu sentiment classification model built on **XLM-RoBERTa (XLM-R)**, a general-purpose multilingual Transformer model developed by Facebook AI. XLM-R is designed to improve cross-lingual understanding through large-scale pretraining on a diverse multilingual corpus.

The base model is pretrained on approximately **2.5 TB of filtered Common Crawl data** spanning **100+ languages**, including Telugu. Unlike mBERT, XLM-R is trained **exclusively using the Masked Language Modeling (MLM) objective**, without the Next Sentence Prediction (NSP) task, enabling stronger contextual representations.

The suffix **WR** denotes **With Rationale supervision**. This model is fine-tuned using both **sentiment labels and human-annotated rationales**, allowing the model to align predictions and explanations with human-identified evidence spans.

---

## Pretraining Details

- **Pretraining corpus:** Filtered Common Crawl (≈2.5 TB, 100+ languages)
- **Training objective:**
  - Masked Language Modeling (MLM)
- **Next Sentence Prediction:** Not used
- **Language coverage:** Telugu included, but not exclusively targeted

---

## Training Data

- **Fine-tuning dataset:** Telugu-Dataset
- **Task:** Sentiment classification
- **Supervision type:** Label + rationale supervision
- **Rationales:** Token-level human-annotated evidence spans

---

## Rationale Supervision

During fine-tuning, **human-provided rationales** are used to guide model learning. In addition to the standard classification loss, an **auxiliary rationale loss** encourages the model’s attention or explanation scores to align with annotated rationale tokens.

This supervision improves:

- Alignment between model explanations and human judgment
- Plausibility of generated explanations
- Interpretability of sentiment predictions

---

## Intended Use

This model is intended for:

- Explainable Telugu sentiment classification
- Rationale-supervised learning experiments
- Cross-lingual explainability research
- Comparative studies against label-only (WOR) baselines

The model is suitable for scenarios where both **predictive performance and explanation quality** are important.

---

## Performance Characteristics

Compared to label-only training, rationale supervision typically improves **explanation plausibility**, while maintaining competitive sentiment classification performance.

### Strengths

- Human-aligned explanations through rationale supervision
- Strong cross-lingual representations from large-scale pretraining
- Suitable for explainable AI benchmarking

### Limitations

- Requires human-annotated rationales, increasing annotation cost
- Classification performance gains may be limited relative to WOR models
- Not explicitly optimized for Telugu morphology or syntax

---

## Use in Explainability Evaluation

**XLM-R_WR** is designed for evaluation with explanation frameworks such as FERRET, enabling:

- **Faithfulness evaluation:** How well explanations support model predictions
- **Plausibility evaluation:** How closely explanations align with human rationales

This makes the model well-suited for rigorous explainability analysis in low-resource Telugu NLP.

---

## References

- Conneau et al., 2019  
- Hedderich et al., 2021  
- Kulkarni et al., 2021  
- Joshi, 2022  
- Das et al., 2022  
- Rajalakshmi et al., 2023