Train Smarter, Not Harder: Switching Signal-Guided Training in Active Learning
Abstract
HybridAL adaptively switches from retraining to fine-tuning during active learning based on online stabilization signals, reducing training time while preserving accuracy and improving calibration.
Training strategy, namely whether to retrain from scratch or fine-tune from the previous checkpoint, is an overlooked decision variable in active learning. We show that this choice has exploitable structure: retraining is most useful in early rounds, when each batch can substantially reshape the labeled distribution, while fine-tuning becomes safer once the model trajectory stabilizes. We propose HybridAL, an adaptive training schedule that monitors an online stabilization signal and switches from retraining to fine-tuning after sustained stabilization. Two complementary signals, spectral exponent change Δα (weight-based) and accuracy change ΔAcc (validation-based), span different points on the time-calibration trade-off. Across three encoder backbones and six text-classification tasks (five seeds each), HybridAL keeps endpoint macro-F1 non-inferior to retraining and fine-tuning at a 0.010 margin, saves up to 49% of retraining time, and recovers a substantial fraction of retraining's calibration advantage as measured by negative log-likelihood (NLL). Compared with schedules that switch at a pre-committed round, HybridAL obtains lower NLL at moderate additional cost, showing that trajectory-dependent switching provides a stronger time-calibration trade-off than fixed early switching.
Community
HybridAL treats the training strategy in active learning (retrain from scratch each round, or fine-tune from the previous checkpoint) as a decision variable rather than a fixed implementation detail.
The structure we found: retraining is most useful in early rounds, when each new batch can substantially reshape the labeled distribution, while fine-tuning becomes safe once the model trajectory stabilizes. HybridAL monitors an online stabilization signal and switches once, after sustained stabilization. We study two signals, spectral exponent change (Δα, weight-based) and accuracy change (ΔAcc, validation-based), which sit at different points on the time–calibration trade-off.
Across three encoder backbones and six text classification tasks, five seeds each: endpoint macro-F1 non-inferior to both retraining and fine-tuning at a 0.010 margin, up to 49% of retraining time saved, and a substantial fraction of retraining's calibration advantage recovered in NLL. Against schedules that switch at a pre-committed round, HybridAL reaches lower NLL at moderate additional cost.
Accepted to EMNLP 2026 Main Conference.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Post-Training Science for Supervised Fine-Tuning (2026)
- Logit-Origin Centering for Singleton Test-Time Adaptation (2026)
- CODS: Iterative Bellman-Residual Data Selection for Reusable Offline Reinforcement Learning (2026)
- AOS: Adaptive Optimizer Switching via Training-State Signals for Faster Convergence and Better Generalization (2026)
- TailSFT: Filtered Fine-Tuning Improves Post-Training Performance (2026)
- Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation (2026)
- Beyond Solvability: Task Learnability as a Static Prior for LLM RL Post-Training (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2609.06806 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper