Paper
Text-to-LoRA: Instant Transformer Adaption
First arXiv submission: · ICML 2025
Uses a hypernetwork to generate LoRA adapters from task descriptions.
What it does
Trains hypernetworks to reconstruct existing adapters or learn from supervised task data. Tests adapter compression and generation for unseen tasks.
- Conditioning
- A natural-language task description.
- Program form
- Generated LoRA matrices.
- Execution
- A base transformer performs the task with the generated LoRA.
Related work
PAW cites Text-to-LoRA as prior work on generating adapters. PAW develops fuzzy-function programming, FuzzyBench, and an interface for reusable neural programs.
Limitations
Reconstruction-trained T2L underperforms the supervised variant on unseen tasks.
Related terms: Text-to-LoRA, T2L, task description, adapter generation.
Related methods
From generated adapters to reusable functions
Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.
Cite this paper
Published version
Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange. Text-to-LoRA: Instant Transformer Adaption. Proceedings of the 42nd International Conference on Machine Learning, 2025.
Publication BibTeX · Publication CSL-JSON
ArXiv preprint
arXiv:2506.06105, initial submission 2025-06-06.