What does a neural compiler produce?
A task description can be encoded as input context, generated weights, or saved activations. The representation determines what can be reused and what must run for every new input.
Generated weights and adapters
HyperNetworks uses one network to generate weights for another. Hypter generates task-specific adapters from descriptions. HyperTuning generates adaptation parameters from few-shot examples. HINT encodes instructions through a hypernetwork.
Text-to-LoRA maps task descriptions to LoRA adaptations. PAW uses text-to-LoRA compilation within a compiler–interpreter system for reusable fuzzy functions. The compiler creates a program from a description; a lightweight interpreter executes it on new inputs. PAW, §10.
From generated adapters to reusable functions
Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.
Compilation through training
Compile by Training, by Yuntian Deng, Pengyu Nie, and Stuart Shieber, makes adaptation part of the build process. Teacher models synthesize examples from a natural-language function specification. The compiler uses those examples to refine an adapter initialized by PAW’s fast compiler, then packages it with the runtime scaffold for repeated local execution. The shared Qwen3-0.6B interpreter stays frozen during this training procedure.
The fast compiler predicts program parameters. Compile by Training adds optimization for the specification. Both produce functions with the same interface, and the saved function runs without the teachers. The evaluation guide compares compilation cost and accuracy.
Fast prediction and training
PAW’s fast compiler predicts a program. Compile by Training starts from that prediction, generates examples, and trains an adapter. Both use the same runtime interface; the extra training improves semantic accuracy on FuzzyBench-Hard at a higher compilation cost.
PAW compiler implementations · Compile by Training method · Compile by Training evaluation
Context compression
Processing the same instruction for every input repeats work. Gisting compresses prompts into a small number of gist-token activations. Generative Adapter produces contextual parameter updates.
To compare their cost, count the work needed to create the representation and the computation that remains for each input. The conditioning information also matters: an instruction, a demonstration, and a document provide different information about the task.
What gets reused?
Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module.
Gist paper · Generative Adapter paper · PAW program definition
Prefixes and LoRA adapters
Prefix-Tuning learns continuous prefix activations for a frozen language model. LoRA represents adaptation through low-rank updates to model weights. Both original methods learn task adaptations from examples.
PAW generates prefixes or LoRA adapters from a specification. Compile by Training refines an initialized LoRA using synthesized examples. Prefixes and LoRA define the program’s representation; the compiler determines how it is produced; the programming interface determines how software saves and calls it.
Program form and compiler
LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications.
Comparing compilers
For each method, compare its inputs, its output, whether a new task requires optimization, what stays frozen, and what is reused across calls. The paper pages describe these components. The evaluation guide covers accuracy, robustness, and execution cost.