# Parameter-efficient building blocks Adapt a frozen model with a small set of learned parameters. Papers are ordered by first arXiv submission date. For example: Add low-rank weight updates or prefix activations to a frozen transformer. Version 1.2.0; updated 2026-10-19. [Topic guide](https://programasweights.com/reading-list/guides/compilation/#peft) · [JSON](https://programasweights.com/reading-list/data/topics/peft.json) ## Method comparison **Program form and compiler** LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications. Sources: [LoRA paper](https://arxiv.org/abs/2106.09685) · [Prefix-tuning paper](https://aclanthology.org/2021.acl-long.353/) · [PAW implementations](https://arxiv.org/html/2607.02512v1#S3) Paper details: [LoRA: Low-Rank Adaptation of Large Language Models](https://programasweights.com/reading-list/papers/2106.09685/) · [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://programasweights.com/reading-list/papers/2101.00190/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) ## Papers — oldest first ## Prefix-Tuning: Optimizing Continuous Prompts for Generation Authors: Xiang Lisa Li, Percy Liang. Adapts a frozen language model by learning a short sequence of continuous prefix vectors. ArXiv initial submission: 2021-01-01. [Original paper](https://arxiv.org/abs/2101.00190) · [Reading-list entry](https://programasweights.com/reading-list/papers/2101.00190/) [Preprint BibTeX](https://programasweights.com/reading-list/papers/2101.00190/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2101.00190/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2101.00190/record.json) **Contribution:** Trains prefixes for table-to-text generation and summarization while updating far fewer parameters than full fine-tuning. **Program form:** Continuous prefix activations. **Conditioning:** Training examples for the target task. **Execution:** The frozen language model attends to the learned prefix. **Relationship to PAW:** PAW evaluates compiler-generated prefixes alongside LoRA adapters. **Limitations:** A separate prefix is optimized for each task. [Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10) Related terms: prefix-tuning, continuous prompts, virtual tokens, parameter-efficient fine-tuning. ### Method comparisons **Program form and compiler** LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications. Sources: [LoRA paper](https://arxiv.org/abs/2106.09685) · [Prefix-tuning paper](https://aclanthology.org/2021.acl-long.353/) · [PAW implementations](https://arxiv.org/html/2607.02512v1#S3) Paper details: [LoRA: Low-Rank Adaptation of Large Language Models](https://programasweights.com/reading-list/papers/2106.09685/) · [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://programasweights.com/reading-list/papers/2101.00190/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) ### Published version Prefix-Tuning: Optimizing Continuous Prompts for Generation. Xiang Lisa Li, Percy Liang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021. [Publication](https://aclanthology.org/2021.acl-long.353/) · [BibTeX](https://programasweights.com/reading-list/papers/2101.00190/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2101.00190/publication.csl.json) [Publication record](https://aclanthology.org/2021.acl-long.353/). ### Approaches - [Parameter-efficient building blocks](https://programasweights.com/reading-list/guides/compilation/#peft): Adapt a frozen model with a small set of learned parameters. Reading-list version 1.2.0; updated 2026-10-19. ## LoRA: Low-Rank Adaptation of Large Language Models Authors: Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen. Adapts a pretrained model with low-rank weight updates while keeping the original weights fixed. ArXiv initial submission: 2021-06-17. [Original paper](https://arxiv.org/abs/2106.09685) · [Reading-list entry](https://programasweights.com/reading-list/papers/2106.09685/) [Preprint BibTeX](https://programasweights.com/reading-list/papers/2106.09685/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2106.09685/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2106.09685/record.json) **Contribution:** Reduces the parameters trained and stored for each task. Evaluates adaptation across several language-model families. **Program form:** Low-rank matrices that update selected model weights. **Conditioning:** Training examples for the target task. **Execution:** The language model applies the low-rank updates during inference. **Relationship to PAW:** PAW uses LoRA as a program representation and generates the adapter with a trained compiler. **Limitations:** A separate adapter is trained for each task. [Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10) Related terms: LoRA, low-rank adaptation, parameter-efficient fine-tuning, adapters. ### Method comparisons **Program form and compiler** LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications. Sources: [LoRA paper](https://arxiv.org/abs/2106.09685) · [Prefix-tuning paper](https://aclanthology.org/2021.acl-long.353/) · [PAW implementations](https://arxiv.org/html/2607.02512v1#S3) Paper details: [LoRA: Low-Rank Adaptation of Large Language Models](https://programasweights.com/reading-list/papers/2106.09685/) · [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://programasweights.com/reading-list/papers/2101.00190/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) ### Published version LoRA: Low-Rank Adaptation of Large Language Models. Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen. International Conference on Learning Representations, 2022. [Publication](https://openreview.net/forum?id=nZeVKeeFYf9) · [BibTeX](https://programasweights.com/reading-list/papers/2106.09685/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2106.09685/publication.csl.json) [Publication record](https://openreview.net/pdf?id=nZeVKeeFYf9). ### Approaches - [Parameter-efficient building blocks](https://programasweights.com/reading-list/guides/compilation/#peft): Adapt a frozen model with a small set of learned parameters. Reading-list version 1.2.0; updated 2026-10-19.