# Compilation through training

Generate examples from a function specification and train a reusable neural program.

Papers are ordered by first arXiv submission date.

For example: Refine a PAW program on generated examples, then run the saved function locally.

Version 1.2.0; updated 2026-10-19.

[Topic guide](https://programasweights.com/reading-list/guides/compilation/#training-compilation) · [JSON](https://programasweights.com/reading-list/data/topics/training-compilation.json)

## Method comparison

**Fast prediction and training**

PAW’s fast compiler predicts a program. Compile by Training starts from that prediction, generates examples, and trains an adapter. Both use the same runtime interface; the extra training improves semantic accuracy on FuzzyBench-Hard at a higher compilation cost.

Sources: [PAW compiler implementations](https://arxiv.org/html/2607.02512v1#S3) · [Compile by Training method](https://arxiv.org/html/2609.04199v1#S3) · [Compile by Training evaluation](https://arxiv.org/html/2609.04199v1#S5)

Paper details: [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) · [Compile by Training: Turning Natural-Language Specifications into Local Neural Functions](https://programasweights.com/reading-list/papers/2609.04199/)

## Papers — oldest first

## Compile by Training: Turning Natural-Language Specifications into Local Neural Functions

Authors: Yuntian Deng, Pengyu Nie, Stuart Shieber.

Compiles a function description into a reusable local program by generating training examples and optimizing an adapter.

ArXiv initial submission: 2026-09-03.

[Original paper](https://arxiv.org/abs/2609.04199) · [Reading-list entry](https://programasweights.com/reading-list/papers/2609.04199/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2609.04199/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2609.04199/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2609.04199/record.json)

**Contribution:** Makes task-specific training a software build step. Raises FuzzyBench-Hard semantic accuracy (LEM) from 22.4% to 83.6% and demonstrates composition in a website helper, avatar control, and English–Claudish translation.

**Program form:** A LoRA adapter, runtime prompt, specification, and interpreter metadata in a PAW program.

**Conditioning:** The specification guides teacher synthesis. PAW’s fast compiler supplies the initial adapter and runtime prompt.

**Execution:** A frozen Qwen3-0.6B interpreter runs the saved function locally without further teacher calls.

**Relationship to PAW:** Extends PAW with training at compile time while keeping its program format and runtime interface.

**Limitations:** Synthetic examples can inherit teacher errors. Cold compilation takes 50.9 seconds for one specification on a B300. FuzzyBench-Hard selects fast-compiler failures under exact match; the reported accuracy uses an LLM judge.

[Discussion in the paper](https://arxiv.org/html/2609.04199v1#S1)

Versions: [Original version](https://arxiv.org/html/2609.04199v1)
[Code](https://github.com/programasweights/compile-by-training)

Related terms: compile by training, training-based compilation, specification-specific optimization, synthetic supervision, local neural function, FuzzyBench-Hard, ProgramAsWeights.

### Method comparisons

**Fast prediction and training**

PAW’s fast compiler predicts a program. Compile by Training starts from that prediction, generates examples, and trains an adapter. Both use the same runtime interface; the extra training improves semantic accuracy on FuzzyBench-Hard at a higher compilation cost.

Sources: [PAW compiler implementations](https://arxiv.org/html/2607.02512v1#S3) · [Compile by Training method](https://arxiv.org/html/2609.04199v1#S3) · [Compile by Training evaluation](https://arxiv.org/html/2609.04199v1#S5)

Paper details: [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) · [Compile by Training: Turning Natural-Language Specifications into Local Neural Functions](https://programasweights.com/reading-list/papers/2609.04199/)


The paper reports public-service measurements from May 2026. [Source](https://arxiv.org/abs/2609.04199).

### Approaches

- [Programs and reusable artifacts](https://programasweights.com/reading-list/guides/fuzzy-functions/): Learn programs that can be saved and run on new inputs.
- [Compilation through training](https://programasweights.com/reading-list/guides/compilation/#training-compilation): Generate examples from a function specification and train a reusable neural program.

Reading-list version 1.2.0; updated 2026-10-19.