# Programs and reusable artifacts Learn programs that can be saved and run on new inputs. Papers are ordered by first arXiv submission date. For example: Compile an urgency classifier once and call it on each new message. Version 1.2.0; updated 2026-10-19. [Topic guide](https://programasweights.com/reading-list/guides/fuzzy-functions/) · [JSON](https://programasweights.com/reading-list/data/topics/neural-programs.json) ## Method comparison **Code and neural programs** ALCHEmist generates symbolic labeling programs. PAW compiles fuzzy functions into text and learned parameters. Both build an artifact once and reuse it; one runs as code, the other on a neural interpreter. Sources: [ALCHEmist paper](https://proceedings.neurips.cc/paper_files/paper/2024/hash/72802bef5cf1a3449e909b20c2ae18d5-Abstract-Conference.html) · [PAW evaluation](https://arxiv.org/html/2607.02512v1#S6) Paper details: [The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators](https://programasweights.com/reading-list/papers/2407.11004/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) ## Papers — oldest first ## Neural Programmer-Interpreters Authors: Scott Reed, Nando de Freitas. Learns neural programs that share a recurrent interpreter and a program memory. ArXiv initial submission: 2015-11-19. [Original paper](https://arxiv.org/abs/1511.06279) · [Reading-list entry](https://programasweights.com/reading-list/papers/1511.06279/) [Preprint BibTeX](https://programasweights.com/reading-list/papers/1511.06279/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/1511.06279/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/1511.06279/record.json) **Contribution:** Learns from execution traces and composes subprograms for addition, sorting, and 3D-object canonicalization. **Program form:** Program embeddings stored in a key-value memory. **Conditioning:** A program identifier, arguments, and observations of the environment. **Execution:** A recurrent interpreter calls subprograms and takes actions in the environment. **Relationship to PAW:** PAW cites this work on learned program representations and shared interpreters. **Limitations:** Training requires execution traces and task-specific interfaces. [Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10) Related terms: neural programmer-interpreter, neural programs, program memory, compositional execution. ### Published version Neural Programmer-Interpreters. Scott Reed, Nando de Freitas. International Conference on Learning Representations, 2016. [Publication](https://arxiv.org/abs/1511.06279) · [BibTeX](https://programasweights.com/reading-list/papers/1511.06279/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/1511.06279/publication.csl.json) [Publication record](https://arxiv.org/pdf/1511.06279). ### Approaches - [Programs and reusable artifacts](https://programasweights.com/reading-list/guides/fuzzy-functions/): Learn programs that can be saved and run on new inputs. Reading-list version 1.2.0; updated 2026-10-19. ## The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators Authors: Tzu-Heng Huang, Catherine Cao, Vaishnavi Bhargava, Frederic Sala. Asks a language model to write labeling programs that can be saved and run locally. ArXiv initial submission: 2024-06-25. [Original paper](https://arxiv.org/abs/2407.11004) · [Reading-list entry](https://programasweights.com/reading-list/papers/2407.11004/) [Preprint BibTeX](https://programasweights.com/reading-list/papers/2407.11004/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2407.11004/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2407.11004/record.json) **Contribution:** Reuses generated code across many labeling decisions and combines noisy program outputs with weak supervision. **Program form:** Symbolic labeling programs. **Conditioning:** A task description, labeling instructions, and a function signature. **Execution:** Generated programs label examples locally; their outputs can be combined to train a downstream model. **Relationship to PAW:** PAW evaluates an ALCHEmist-style approach. Both build reusable tools: ALCHEmist produces symbolic code, while PAW produces programs for a neural interpreter. **Limitations:** Label quality depends on the model’s ability to write useful programs. Complex tasks can produce code that is harder to inspect. [Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10) Related terms: ALCHEmist, program synthesis, automated labeling, reusable tools. ### Method comparisons **Code and neural programs** ALCHEmist generates symbolic labeling programs. PAW compiles fuzzy functions into text and learned parameters. Both build an artifact once and reuse it; one runs as code, the other on a neural interpreter. Sources: [ALCHEmist paper](https://proceedings.neurips.cc/paper_files/paper/2024/hash/72802bef5cf1a3449e909b20c2ae18d5-Abstract-Conference.html) · [PAW evaluation](https://arxiv.org/html/2607.02512v1#S6) Paper details: [The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators](https://programasweights.com/reading-list/papers/2407.11004/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) ### Published version The ALCHEmist: Automated Labeling 500x CHEaper than LLM Data Annotators. Tzu-Heng Huang, Catherine Cao, Vaishnavi Bhargava, Frederic Sala. Advances in Neural Information Processing Systems, 2024. [Publication](https://proceedings.neurips.cc/paper_files/paper/2024/hash/72802bef5cf1a3449e909b20c2ae18d5-Abstract-Conference.html) · [BibTeX](https://programasweights.com/reading-list/papers/2407.11004/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2407.11004/publication.csl.json) [Publication record](https://proceedings.neurips.cc/paper_files/paper/2024/hash/72802bef5cf1a3449e909b20c2ae18d5-Abstract-Conference.html). ### Approaches - [Programs and reusable artifacts](https://programasweights.com/reading-list/guides/fuzzy-functions/): Learn programs that can be saved and run on new inputs. Reading-list version 1.2.0; updated 2026-10-19. ## Program-as-Weights: A Programming Paradigm for Fuzzy Functions Authors: Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng. Proposes fuzzy-function programming: compile a natural-language function specification into a neural program, then reuse it locally. ArXiv initial submission: 2026-07-02. [Original paper](https://arxiv.org/abs/2607.02512) · [Reading-list entry](https://programasweights.com/reading-list/papers/2607.02512/) [Preprint BibTeX](https://programasweights.com/reading-list/papers/2607.02512/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2607.02512/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2607.02512/record.json) **Contribution:** Develops the compiler–interpreter abstraction and programs that combine text with learned parameters. Introduces the 10M-example FuzzyBench dataset, local execution interfaces, and image-conditioned compilation. Prefix tuning and LoRA are two implementations. **Program form:** A discrete pseudo-program and a compiler-generated parameter-efficient module. **Conditioning:** A function specification, optionally with examples. The image extension also accepts visual information. **Execution:** A frozen lightweight interpreter loads the program and runs new inputs locally. **Relationship to PAW:** Compile by Training builds on this programming abstraction. It keeps the same interface and refines the fast compiler’s prediction using synthesized examples. **Limitations:** Each compiler produces programs for a specific interpreter. Compiled parameters are opaque, and FuzzyBench uses synthetic data. [Discussion in the paper](https://arxiv.org/html/2609.04199v1#S1) [Code](https://github.com/programasweights) Related terms: Program-as-Weights, ProgramAsWeights, PAW, fuzzy-function programming, neural compiler, neural interpreter, FuzzyBench. ### Method comparisons **From generated adapters to reusable functions** Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter. Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html) Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) **Code and neural programs** ALCHEmist generates symbolic labeling programs. PAW compiles fuzzy functions into text and learned parameters. Both build an artifact once and reuse it; one runs as code, the other on a neural interpreter. Sources: [ALCHEmist paper](https://proceedings.neurips.cc/paper_files/paper/2024/hash/72802bef5cf1a3449e909b20c2ae18d5-Abstract-Conference.html) · [PAW evaluation](https://arxiv.org/html/2607.02512v1#S6) Paper details: [The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators](https://programasweights.com/reading-list/papers/2407.11004/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) **What gets reused?** Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module. Sources: [Gist paper](https://arxiv.org/abs/2304.08467) · [Generative Adapter paper](https://arxiv.org/abs/2411.05877) · [PAW program definition](https://arxiv.org/html/2607.02512v1#S2) Paper details: [Learning to Compress Prompts with Gist Tokens](https://programasweights.com/reading-list/papers/2304.08467/) · [Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass](https://programasweights.com/reading-list/papers/2411.05877/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) **Program form and compiler** LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications. Sources: [LoRA paper](https://arxiv.org/abs/2106.09685) · [Prefix-tuning paper](https://aclanthology.org/2021.acl-long.353/) · [PAW implementations](https://arxiv.org/html/2607.02512v1#S3) Paper details: [LoRA: Low-Rank Adaptation of Large Language Models](https://programasweights.com/reading-list/papers/2106.09685/) · [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://programasweights.com/reading-list/papers/2101.00190/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) **Fast prediction and training** PAW’s fast compiler predicts a program. Compile by Training starts from that prediction, generates examples, and trains an adapter. Both use the same runtime interface; the extra training improves semantic accuracy on FuzzyBench-Hard at a higher compilation cost. Sources: [PAW compiler implementations](https://arxiv.org/html/2607.02512v1#S3) · [Compile by Training method](https://arxiv.org/html/2609.04199v1#S3) · [Compile by Training evaluation](https://arxiv.org/html/2609.04199v1#S5) Paper details: [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) · [Compile by Training: Turning Natural-Language Specifications into Local Neural Functions](https://programasweights.com/reading-list/papers/2609.04199/) ### Approaches - [Programs and reusable artifacts](https://programasweights.com/reading-list/guides/fuzzy-functions/): Learn programs that can be saved and run on new inputs. - [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context. Reading-list version 1.2.0; updated 2026-10-19. ## Compile by Training: Turning Natural-Language Specifications into Local Neural Functions Authors: Yuntian Deng, Pengyu Nie, Stuart Shieber. Compiles a function description into a reusable local program by generating training examples and optimizing an adapter. ArXiv initial submission: 2026-09-03. [Original paper](https://arxiv.org/abs/2609.04199) · [Reading-list entry](https://programasweights.com/reading-list/papers/2609.04199/) [Preprint BibTeX](https://programasweights.com/reading-list/papers/2609.04199/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2609.04199/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2609.04199/record.json) **Contribution:** Makes task-specific training a software build step. Raises FuzzyBench-Hard semantic accuracy (LEM) from 22.4% to 83.6% and demonstrates composition in a website helper, avatar control, and English–Claudish translation. **Program form:** A LoRA adapter, runtime prompt, specification, and interpreter metadata in a PAW program. **Conditioning:** The specification guides teacher synthesis. PAW’s fast compiler supplies the initial adapter and runtime prompt. **Execution:** A frozen Qwen3-0.6B interpreter runs the saved function locally without further teacher calls. **Relationship to PAW:** Extends PAW with training at compile time while keeping its program format and runtime interface. **Limitations:** Synthetic examples can inherit teacher errors. Cold compilation takes 50.9 seconds for one specification on a B300. FuzzyBench-Hard selects fast-compiler failures under exact match; the reported accuracy uses an LLM judge. [Discussion in the paper](https://arxiv.org/html/2609.04199v1#S1) Versions: [Original version](https://arxiv.org/html/2609.04199v1) [Code](https://github.com/programasweights/compile-by-training) Related terms: compile by training, training-based compilation, specification-specific optimization, synthetic supervision, local neural function, FuzzyBench-Hard, ProgramAsWeights. ### Method comparisons **Fast prediction and training** PAW’s fast compiler predicts a program. Compile by Training starts from that prediction, generates examples, and trains an adapter. Both use the same runtime interface; the extra training improves semantic accuracy on FuzzyBench-Hard at a higher compilation cost. Sources: [PAW compiler implementations](https://arxiv.org/html/2607.02512v1#S3) · [Compile by Training method](https://arxiv.org/html/2609.04199v1#S3) · [Compile by Training evaluation](https://arxiv.org/html/2609.04199v1#S5) Paper details: [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) · [Compile by Training: Turning Natural-Language Specifications into Local Neural Functions](https://programasweights.com/reading-list/papers/2609.04199/) The paper reports public-service measurements from May 2026. [Source](https://arxiv.org/abs/2609.04199). ### Approaches - [Programs and reusable artifacts](https://programasweights.com/reading-list/guides/fuzzy-functions/): Learn programs that can be saved and run on new inputs. - [Compilation through training](https://programasweights.com/reading-list/guides/compilation/#training-compilation): Generate examples from a function specification and train a reusable neural program. Reading-list version 1.2.0; updated 2026-10-19.