# Prefix-Tuning: Optimizing Continuous Prompts for Generation

Authors: Xiang Lisa Li, Percy Liang.

Adapts a frozen language model by learning a short sequence of continuous prefix vectors.

ArXiv initial submission: 2021-01-01.

[Original paper](https://arxiv.org/abs/2101.00190) · [Reading-list entry](https://programasweights.com/reading-list/papers/2101.00190/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2101.00190/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2101.00190/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2101.00190/record.json)

**Contribution:** Trains prefixes for table-to-text generation and summarization while updating far fewer parameters than full fine-tuning.

**Program form:** Continuous prefix activations.

**Conditioning:** Training examples for the target task.

**Execution:** The frozen language model attends to the learned prefix.

**Relationship to PAW:** PAW evaluates compiler-generated prefixes alongside LoRA adapters.

**Limitations:** A separate prefix is optimized for each task.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: prefix-tuning, continuous prompts, virtual tokens, parameter-efficient fine-tuning.

## Method comparisons

**Program form and compiler**

LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications.

Sources: [LoRA paper](https://arxiv.org/abs/2106.09685) · [Prefix-tuning paper](https://aclanthology.org/2021.acl-long.353/) · [PAW implementations](https://arxiv.org/html/2607.02512v1#S3)

Paper details: [LoRA: Low-Rank Adaptation of Large Language Models](https://programasweights.com/reading-list/papers/2106.09685/) · [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://programasweights.com/reading-list/papers/2101.00190/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

Prefix-Tuning: Optimizing Continuous Prompts for Generation. Xiang Lisa Li, Percy Liang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021.
[Publication](https://aclanthology.org/2021.acl-long.353/) · [BibTeX](https://programasweights.com/reading-list/papers/2101.00190/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2101.00190/publication.csl.json)
[Publication record](https://aclanthology.org/2021.acl-long.353/).

## Approaches

- [Parameter-efficient building blocks](https://programasweights.com/reading-list/guides/compilation/#peft): Adapt a frozen model with a small set of learned parameters.

Reading-list version 1.2.0; updated 2026-10-19.