# Context compression Compress an instruction or context once and reuse the result across calls. Papers are ordered by first arXiv submission date. For example: Cache a gist representation instead of encoding the same instruction again. Version 1.2.0; updated 2026-10-19. [Topic guide](https://programasweights.com/reading-list/guides/compilation/#context-compression) · [JSON](https://programasweights.com/reading-list/data/topics/context-compression.json) ## Method comparison **What gets reused?** Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module. Sources: [Gist paper](https://arxiv.org/abs/2304.08467) · [Generative Adapter paper](https://arxiv.org/abs/2411.05877) · [PAW program definition](https://arxiv.org/html/2607.02512v1#S2) Paper details: [Learning to Compress Prompts with Gist Tokens](https://programasweights.com/reading-list/papers/2304.08467/) · [Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass](https://programasweights.com/reading-list/papers/2411.05877/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) ## Papers — oldest first ## Learning to Compress Prompts with Gist Tokens Authors: Jesse Mu, Xiang Lisa Li, Noah Goodman. Compresses an instruction into a few reusable token activations. ArXiv initial submission: 2023-04-17. [Original paper](https://arxiv.org/abs/2304.08467) · [Reading-list entry](https://programasweights.com/reading-list/papers/2304.08467/) [Preprint BibTeX](https://programasweights.com/reading-list/papers/2304.08467/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2304.08467/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2304.08467/record.json) **Contribution:** Trains with an attention mask that forces later tokens to use gist activations in place of the original prompt. **Program form:** Cached activations at a few gist-token positions. **Conditioning:** The instruction or prompt. **Execution:** The language model generates outputs while attending to the cached gist. **Relationship to PAW:** PAW cites gisting as another way to reuse instruction computation. Gisting stores activations; PAW stores a program with text and adaptation parameters. **Limitations:** Compression can reduce output quality. The cached activations are tied to the trained model. [Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10) Related terms: gist tokens, prompt compression, activation caching, amortized inference. ### Method comparisons **What gets reused?** Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module. Sources: [Gist paper](https://arxiv.org/abs/2304.08467) · [Generative Adapter paper](https://arxiv.org/abs/2411.05877) · [PAW program definition](https://arxiv.org/html/2607.02512v1#S2) Paper details: [Learning to Compress Prompts with Gist Tokens](https://programasweights.com/reading-list/papers/2304.08467/) · [Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass](https://programasweights.com/reading-list/papers/2411.05877/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) ### Published version Learning to Compress Prompts with Gist Tokens. Jesse Mu, Xiang Li, Noah Goodman. Advances in Neural Information Processing Systems, 2023. [Publication](https://proceedings.neurips.cc/paper_files/paper/2023/hash/3d77c6dcc7f143aa2154e7f4d5e22d68-Abstract-Conference.html) · [BibTeX](https://programasweights.com/reading-list/papers/2304.08467/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2304.08467/publication.csl.json) [Publication record](https://proceedings.neurips.cc/paper_files/paper/2023/hash/3d77c6dcc7f143aa2154e7f4d5e22d68-Abstract-Conference.html). ### Approaches - [Context compression](https://programasweights.com/reading-list/guides/compilation/#context-compression): Compress an instruction or context once and reuse the result across calls. Reading-list version 1.2.0; updated 2026-10-19. ## Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass Authors: Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Benjamin Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng. Converts new context into low-rank model updates in one forward pass. ArXiv initial submission: 2024-11-08. [Original paper](https://arxiv.org/abs/2411.05877) · [Reading-list entry](https://programasweights.com/reading-list/papers/2411.05877/) [Preprint BibTeX](https://programasweights.com/reading-list/papers/2411.05877/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2411.05877/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2411.05877/record.json) **Contribution:** Trains an adapter generator with self-supervision. Tests learning from documents, demonstrations, and conversation history. **Program form:** Additive low-rank adapters. **Conditioning:** Documents, demonstrations, or conversation history, including successive chunks of context. **Execution:** The adapted model answers new inputs using context stored in its parameter updates. **Relationship to PAW:** PAW cites this context-to-adapter method. Generative Adapter stores information from context; PAW compiles the behavior of a function. **Limitations:** The adapter generator is tied to its frozen base model. [Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10) Related terms: Generative Adapter, contextualization, fast weights, streaming context. ### Method comparisons **What gets reused?** Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module. Sources: [Gist paper](https://arxiv.org/abs/2304.08467) · [Generative Adapter paper](https://arxiv.org/abs/2411.05877) · [PAW program definition](https://arxiv.org/html/2607.02512v1#S2) Paper details: [Learning to Compress Prompts with Gist Tokens](https://programasweights.com/reading-list/papers/2304.08467/) · [Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass](https://programasweights.com/reading-list/papers/2411.05877/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) An AFM 2024 OpenReview version appeared before the first arXiv submission. [Source](https://arxiv.org/abs/2411.05877). ### Published version Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass. Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Benjamin Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng. International Conference on Learning Representations, 2025. [Publication](https://openreview.net/forum?id=bc3sUsS6ck) · [BibTeX](https://programasweights.com/reading-list/papers/2411.05877/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2411.05877/publication.csl.json) [Publication record](https://openreview.net/pdf?id=bc3sUsS6ck). ### Approaches - [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context. - [Context compression](https://programasweights.com/reading-list/guides/compilation/#context-compression): Compress an instruction or context once and reuse the result across calls. Reading-list version 1.2.0; updated 2026-10-19.