# HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation

Authors: Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew Peters.

Turns instructions and examples into reusable prefixes, adapters, and encoded instruction representations.

ArXiv initial submission: 2022-12-20.

[Original paper](https://arxiv.org/abs/2212.10315) · [Reading-list entry](https://programasweights.com/reading-list/papers/2212.10315/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2212.10315/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2212.10315/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2212.10315/record.json)

**Contribution:** Combines a hypernetwork with instruction fusion so repeated calls can reuse the instruction computation. Evaluates zero-shot and few-shot generalization.

**Program form:** Prefixes, adapters, and encoded instruction representations.

**Conditioning:** Task instructions and optional examples.

**Execution:** An encoder-decoder model uses the generated modules and attends to cached instruction representations.

**Relationship to PAW:** HINT reuses generated modules and an encoded instruction. PAW reuses a program containing text and adaptation parameters.

**Limitations:** The generated modules and cached instruction representations are tied to the trained encoder-decoder model.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: HINT, instruction tuning, hypernetwork, instruction fusion.

## Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

HINT: Hypernetwork Instruction Tuning for Efficient Zero- and Few-Shot Generalisation. Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew E. Peters. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023.
[Publication](https://aclanthology.org/2023.acl-long.631/) · [BibTeX](https://programasweights.com/reading-list/papers/2212.10315/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2212.10315/publication.csl.json)
[Publication record](https://aclanthology.org/2023.acl-long.631/).

## Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.2.0; updated 2026-10-19.