# Text-to-LoRA: Instant Transformer Adaption

Authors: Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange.

Uses a hypernetwork to generate LoRA adapters from task descriptions.

ArXiv initial submission: 2025-06-06.

[Original paper](https://arxiv.org/abs/2506.06105) · [Reading-list entry](https://programasweights.com/reading-list/papers/2506.06105/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2506.06105/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2506.06105/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2506.06105/record.json)

**Contribution:** Trains hypernetworks to reconstruct existing adapters or learn from supervised task data. Tests adapter compression and generation for unseen tasks.

**Program form:** Generated LoRA matrices.

**Conditioning:** A natural-language task description.

**Execution:** A base transformer performs the task with the generated LoRA.

**Relationship to PAW:** PAW cites Text-to-LoRA as prior work on generating adapters. PAW develops fuzzy-function programming, FuzzyBench, and an interface for reusable neural programs.

**Limitations:** Reconstruction-trained T2L underperforms the supervised variant on unseen tasks.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: Text-to-LoRA, T2L, task description, adapter generation.

## Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

Text-to-LoRA: Instant Transformer Adaption. Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange. Proceedings of the 42nd International Conference on Machine Learning, 2025.
[Publication](https://proceedings.mlr.press/v267/charakorn25a.html) · [BibTeX](https://programasweights.com/reading-list/papers/2506.06105/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2506.06105/publication.csl.json)
[Publication record](https://proceedings.mlr.press/v267/charakorn25a.html).

## Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.2.0; updated 2026-10-19.