# Generated weights and adapters

Generate model weights from a task description, examples, or context.

Papers are ordered by first arXiv submission date.

For example: A task description produces LoRA matrices for a fixed language model.

Version 1.2.0; updated 2026-10-19.

[Topic guide](https://programasweights.com/reading-list/guides/compilation/#generated-adapters) · [JSON](https://programasweights.com/reading-list/data/topics/generated-adapters.json)

## Method comparison

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)

## Papers — oldest first

## HyperNetworks

Authors: David Ha, Andrew Dai, Quoc V. Le.

Uses one neural network to generate the weights of another.

ArXiv initial submission: 2016-09-27.

[Original paper](https://arxiv.org/abs/1609.09106) · [Reading-list entry](https://programasweights.com/reading-list/papers/1609.09106/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/1609.09106/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/1609.09106/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/1609.09106/record.json)

**Contribution:** Applies weight generation to convolutional and recurrent networks, relaxing weight sharing across layers and generating dynamic weights for sequence models.

**Program form:** Weights for the target network.

**Conditioning:** Layer embeddings or the current recurrent state.

**Execution:** The target network runs with the generated weights.

**Relationship to PAW:** Provides the weight-generation mechanism used by later description-to-adapter methods and PAW.

**Limitations:** The hypernetwork is trained end to end for the target architecture.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: hypernetwork, weight generation, fast weights, conditional parameters.

### Published version

HyperNetworks. David Ha, Andrew M. Dai, Quoc V. Le. International Conference on Learning Representations, 2017.
[Publication](https://openreview.net/forum?id=rkpACe1lx) · [BibTeX](https://programasweights.com/reading-list/papers/1609.09106/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/1609.09106/publication.csl.json)
[Publication record](https://openreview.net/pdf?id=rkpACe1lx).

### Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.2.0; updated 2026-10-19.


## Learning to Generate Task-Specific Adapters from Task Description

Authors: Qinyuan Ye, Xiang Ren.

Introduces Hypter, which generates adapters from natural-language task descriptions.

ArXiv initial submission: 2021-01-02.

[Original paper](https://arxiv.org/abs/2101.00420) · [Reading-list entry](https://programasweights.com/reading-list/papers/2101.00420/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2101.00420/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2101.00420/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2101.00420/record.json)

**Contribution:** Combines a description encoder and adapter generator with BART. Tests generalization to unseen tasks on ZEST and synthetic SQuAD.

**Program form:** Bottleneck adapters inserted into a text-to-text model.

**Conditioning:** A natural-language task description.

**Execution:** BART performs the task with the generated adapters.

**Relationship to PAW:** An early description-to-adapter method cited by PAW. PAW develops the programming interface for saving and calling the resulting neural functions.

**Limitations:** Training requires a collection of tasks with descriptions and labeled examples.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: Hypter, task descriptions, generated adapters, zero-shot learning.

### Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


### Published version

Learning to Generate Task-Specific Adapters from Task Description. Qinyuan Ye, Xiang Ren. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), 2021.
[Publication](https://aclanthology.org/2021.acl-short.82/) · [BibTeX](https://programasweights.com/reading-list/papers/2101.00420/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2101.00420/publication.csl.json)
[Publication record](https://aclanthology.org/2021.acl-short.82/).

### Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.2.0; updated 2026-10-19.


## HyperTuning: Toward Adapting Large Language Models without Back-propagation

Authors: Jason Phang, Yi Mao, Pengcheng He, Weizhu Chen.

Produces task-specific parameters for a frozen language model from a few demonstrations.

ArXiv initial submission: 2022-11-22.

[Original paper](https://arxiv.org/abs/2211.12485) · [Reading-list entry](https://programasweights.com/reading-list/papers/2211.12485/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2211.12485/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2211.12485/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2211.12485/record.json)

**Contribution:** Introduces HyperT5 models that generate prefixes or LoRA parameters. Hyperpretraining and multitask training prepare the generator for new tasks.

**Program form:** Generated prefix or LoRA parameters.

**Conditioning:** A few examples of the task.

**Execution:** A frozen T5 model runs with the generated parameters.

**Relationship to PAW:** PAW cites HyperTuning as prior work on generating adaptation parameters from examples.

**Limitations:** The hypermodel accepts a limited number of demonstrations. The paper also studies further task-specific tuning.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: HyperTuning, HyperT5, few-shot adaptation, generated LoRA.

### Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


### Published version

HyperTuning: Toward Adapting Large Language Models without Back-propagation. Jason Phang, Yi Mao, Pengcheng He, Weizhu Chen. Proceedings of the 40th International Conference on Machine Learning, 2023.
[Publication](https://proceedings.mlr.press/v202/phang23a.html) · [BibTeX](https://programasweights.com/reading-list/papers/2211.12485/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2211.12485/publication.csl.json)
[Publication record](https://proceedings.mlr.press/v202/phang23a.html).

### Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.2.0; updated 2026-10-19.


## HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation

Authors: Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew Peters.

Turns instructions and examples into reusable prefixes, adapters, and encoded instruction representations.

ArXiv initial submission: 2022-12-20.

[Original paper](https://arxiv.org/abs/2212.10315) · [Reading-list entry](https://programasweights.com/reading-list/papers/2212.10315/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2212.10315/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2212.10315/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2212.10315/record.json)

**Contribution:** Combines a hypernetwork with instruction fusion so repeated calls can reuse the instruction computation. Evaluates zero-shot and few-shot generalization.

**Program form:** Prefixes, adapters, and encoded instruction representations.

**Conditioning:** Task instructions and optional examples.

**Execution:** An encoder-decoder model uses the generated modules and attends to cached instruction representations.

**Relationship to PAW:** HINT reuses generated modules and an encoded instruction. PAW reuses a program containing text and adaptation parameters.

**Limitations:** The generated modules and cached instruction representations are tied to the trained encoder-decoder model.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: HINT, instruction tuning, hypernetwork, instruction fusion.

### Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


### Published version

HINT: Hypernetwork Instruction Tuning for Efficient Zero- and Few-Shot Generalisation. Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew E. Peters. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023.
[Publication](https://aclanthology.org/2023.acl-long.631/) · [BibTeX](https://programasweights.com/reading-list/papers/2212.10315/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2212.10315/publication.csl.json)
[Publication record](https://aclanthology.org/2023.acl-long.631/).

### Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.2.0; updated 2026-10-19.


## Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass

Authors: Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Benjamin Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng.

Converts new context into low-rank model updates in one forward pass.

ArXiv initial submission: 2024-11-08.

[Original paper](https://arxiv.org/abs/2411.05877) · [Reading-list entry](https://programasweights.com/reading-list/papers/2411.05877/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2411.05877/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2411.05877/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2411.05877/record.json)

**Contribution:** Trains an adapter generator with self-supervision. Tests learning from documents, demonstrations, and conversation history.

**Program form:** Additive low-rank adapters.

**Conditioning:** Documents, demonstrations, or conversation history, including successive chunks of context.

**Execution:** The adapted model answers new inputs using context stored in its parameter updates.

**Relationship to PAW:** PAW cites this context-to-adapter method. Generative Adapter stores information from context; PAW compiles the behavior of a function.

**Limitations:** The adapter generator is tied to its frozen base model.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: Generative Adapter, contextualization, fast weights, streaming context.

### Method comparisons

**What gets reused?**

Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module.

Sources: [Gist paper](https://arxiv.org/abs/2304.08467) · [Generative Adapter paper](https://arxiv.org/abs/2411.05877) · [PAW program definition](https://arxiv.org/html/2607.02512v1#S2)

Paper details: [Learning to Compress Prompts with Gist Tokens](https://programasweights.com/reading-list/papers/2304.08467/) · [Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass](https://programasweights.com/reading-list/papers/2411.05877/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


An AFM 2024 OpenReview version appeared before the first arXiv submission. [Source](https://arxiv.org/abs/2411.05877).

### Published version

Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass. Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Benjamin Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng. International Conference on Learning Representations, 2025.
[Publication](https://openreview.net/forum?id=bc3sUsS6ck) · [BibTeX](https://programasweights.com/reading-list/papers/2411.05877/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2411.05877/publication.csl.json)
[Publication record](https://openreview.net/pdf?id=bc3sUsS6ck).

### Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.
- [Context compression](https://programasweights.com/reading-list/guides/compilation/#context-compression): Compress an instruction or context once and reuse the result across calls.

Reading-list version 1.2.0; updated 2026-10-19.


## Text-to-LoRA: Instant Transformer Adaption

Authors: Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange.

Uses a hypernetwork to generate LoRA adapters from task descriptions.

ArXiv initial submission: 2025-06-06.

[Original paper](https://arxiv.org/abs/2506.06105) · [Reading-list entry](https://programasweights.com/reading-list/papers/2506.06105/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2506.06105/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2506.06105/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2506.06105/record.json)

**Contribution:** Trains hypernetworks to reconstruct existing adapters or learn from supervised task data. Tests adapter compression and generation for unseen tasks.

**Program form:** Generated LoRA matrices.

**Conditioning:** A natural-language task description.

**Execution:** A base transformer performs the task with the generated LoRA.

**Relationship to PAW:** PAW cites Text-to-LoRA as prior work on generating adapters. PAW develops fuzzy-function programming, FuzzyBench, and an interface for reusable neural programs.

**Limitations:** Reconstruction-trained T2L underperforms the supervised variant on unseen tasks.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: Text-to-LoRA, T2L, task description, adapter generation.

### Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


### Published version

Text-to-LoRA: Instant Transformer Adaption. Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange. Proceedings of the 42nd International Conference on Machine Learning, 2025.
[Publication](https://proceedings.mlr.press/v267/charakorn25a.html) · [BibTeX](https://programasweights.com/reading-list/papers/2506.06105/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2506.06105/publication.csl.json)
[Publication record](https://proceedings.mlr.press/v267/charakorn25a.html).

### Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.2.0; updated 2026-10-19.


## Program-as-Weights: A Programming Paradigm for Fuzzy Functions

Authors: Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng.

Proposes fuzzy-function programming: compile a natural-language function specification into a neural program, then reuse it locally.

ArXiv initial submission: 2026-07-02.

[Original paper](https://arxiv.org/abs/2607.02512) · [Reading-list entry](https://programasweights.com/reading-list/papers/2607.02512/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2607.02512/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2607.02512/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2607.02512/record.json)

**Contribution:** Develops the compiler–interpreter abstraction and programs that combine text with learned parameters. Introduces the 10M-example FuzzyBench dataset, local execution interfaces, and image-conditioned compilation. Prefix tuning and LoRA are two implementations.

**Program form:** A discrete pseudo-program and a compiler-generated parameter-efficient module.

**Conditioning:** A function specification, optionally with examples. The image extension also accepts visual information.

**Execution:** A frozen lightweight interpreter loads the program and runs new inputs locally.

**Relationship to PAW:** Compile by Training builds on this programming abstraction. It keeps the same interface and refines the fast compiler’s prediction using synthesized examples.

**Limitations:** Each compiler produces programs for a specific interpreter. Compiled parameters are opaque, and FuzzyBench uses synthetic data.

[Discussion in the paper](https://arxiv.org/html/2609.04199v1#S1)
[Code](https://github.com/programasweights)

Related terms: Program-as-Weights, ProgramAsWeights, PAW, fuzzy-function programming, neural compiler, neural interpreter, FuzzyBench.

### Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)

**Code and neural programs**

ALCHEmist generates symbolic labeling programs. PAW compiles fuzzy functions into text and learned parameters. Both build an artifact once and reuse it; one runs as code, the other on a neural interpreter.

Sources: [ALCHEmist paper](https://proceedings.neurips.cc/paper_files/paper/2024/hash/72802bef5cf1a3449e909b20c2ae18d5-Abstract-Conference.html) · [PAW evaluation](https://arxiv.org/html/2607.02512v1#S6)

Paper details: [The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators](https://programasweights.com/reading-list/papers/2407.11004/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)

**What gets reused?**

Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module.

Sources: [Gist paper](https://arxiv.org/abs/2304.08467) · [Generative Adapter paper](https://arxiv.org/abs/2411.05877) · [PAW program definition](https://arxiv.org/html/2607.02512v1#S2)

Paper details: [Learning to Compress Prompts with Gist Tokens](https://programasweights.com/reading-list/papers/2304.08467/) · [Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass](https://programasweights.com/reading-list/papers/2411.05877/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)

**Program form and compiler**

LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications.

Sources: [LoRA paper](https://arxiv.org/abs/2106.09685) · [Prefix-tuning paper](https://aclanthology.org/2021.acl-long.353/) · [PAW implementations](https://arxiv.org/html/2607.02512v1#S3)

Paper details: [LoRA: Low-Rank Adaptation of Large Language Models](https://programasweights.com/reading-list/papers/2106.09685/) · [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://programasweights.com/reading-list/papers/2101.00190/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)

**Fast prediction and training**

PAW’s fast compiler predicts a program. Compile by Training starts from that prediction, generates examples, and trains an adapter. Both use the same runtime interface; the extra training improves semantic accuracy on FuzzyBench-Hard at a higher compilation cost.

Sources: [PAW compiler implementations](https://arxiv.org/html/2607.02512v1#S3) · [Compile by Training method](https://arxiv.org/html/2609.04199v1#S3) · [Compile by Training evaluation](https://arxiv.org/html/2609.04199v1#S5)

Paper details: [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) · [Compile by Training: Turning Natural-Language Specifications into Local Neural Functions](https://programasweights.com/reading-list/papers/2609.04199/)


### Approaches

- [Programs and reusable artifacts](https://programasweights.com/reading-list/guides/fuzzy-functions/): Learn programs that can be saved and run on new inputs.
- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.2.0; updated 2026-10-19.