Paper
HyperNetworks
First arXiv submission: · ICLR 2017
Uses one neural network to generate the weights of another.
What it does
Applies weight generation to convolutional and recurrent networks, relaxing weight sharing across layers and generating dynamic weights for sequence models.
- Conditioning
- Layer embeddings or the current recurrent state.
- Program form
- Weights for the target network.
- Execution
- The target network runs with the generated weights.
Related work
Provides the weight-generation mechanism used by later description-to-adapter methods and PAW.
Limitations
The hypernetwork is trained end to end for the target architecture.
Related terms: hypernetwork, weight generation, fast weights, conditional parameters.
Cite this paper
Published version
David Ha, Andrew M. Dai, Quoc V. Le. HyperNetworks. International Conference on Learning Representations, 2017.
Publication BibTeX · Publication CSL-JSON
ArXiv preprint
arXiv:1609.09106, initial submission 2016-09-27.