Paper
Learning to Generate Task-Specific Adapters from Task Description
First arXiv submission: · ACL-IJCNLP 2021
Introduces Hypter, which generates adapters from natural-language task descriptions.
What it does
Combines a description encoder and adapter generator with BART. Tests generalization to unseen tasks on ZEST and synthetic SQuAD.
- Conditioning
- A natural-language task description.
- Program form
- Bottleneck adapters inserted into a text-to-text model.
- Execution
- BART performs the task with the generated adapters.
Related work
An early description-to-adapter method cited by PAW. PAW develops the programming interface for saving and calling the resulting neural functions.
Limitations
Training requires a collection of tasks with descriptions and labeled examples.
Related terms: Hypter, task descriptions, generated adapters, zero-shot learning.
Related methods
From generated adapters to reusable functions
Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.
Cite this paper
Published version
Qinyuan Ye, Xiang Ren. Learning to Generate Task-Specific Adapters from Task Description. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), 2021.
Publication BibTeX · Publication CSL-JSON
ArXiv preprint
arXiv:2101.00420, initial submission 2021-01-02.