# HyperNetworks

Authors: David Ha, Andrew Dai, Quoc V. Le.

Uses one neural network to generate the weights of another.

ArXiv initial submission: 2016-09-27.

[Original paper](https://arxiv.org/abs/1609.09106) · [Reading-list entry](https://programasweights.com/reading-list/papers/1609.09106/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/1609.09106/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/1609.09106/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/1609.09106/record.json)

**Contribution:** Applies weight generation to convolutional and recurrent networks, relaxing weight sharing across layers and generating dynamic weights for sequence models.

**Program form:** Weights for the target network.

**Conditioning:** Layer embeddings or the current recurrent state.

**Execution:** The target network runs with the generated weights.

**Relationship to PAW:** Provides the weight-generation mechanism used by later description-to-adapter methods and PAW.

**Limitations:** The hypernetwork is trained end to end for the target architecture.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: hypernetwork, weight generation, fast weights, conditional parameters.

## Published version

HyperNetworks. David Ha, Andrew M. Dai, Quoc V. Le. International Conference on Learning Representations, 2017.
[Publication](https://openreview.net/forum?id=rkpACe1lx) · [BibTeX](https://programasweights.com/reading-list/papers/1609.09106/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/1609.09106/publication.csl.json)
[Publication record](https://openreview.net/pdf?id=rkpACe1lx).

## Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.2.0; updated 2026-10-19.