ProgramAsWeights: compile English function descriptions into neural programs that run locally [R]
Our take
![ProgramAsWeights: compile English function descriptions into neural programs that run locally [R]](https://preview.redd.it/la7iukj3vjqh1.png?width=320&crop=smart&auto=webp&s=50c5bf5b2b8cccfb76d05dc3c253298776dca717)
The recent surge of interest in locally-run AI tools, exemplified by projects like Jev, continues to reshape the landscape of data processing and application development. ProgramAsWeights (PAW), a fascinating research project emerging from the University of Waterloo, represents a compelling step forward in this direction. It elegantly separates the compilation of function descriptions – essentially, defining *what* a task should do – from the execution of those functions, enabling a degree of modularity and efficiency previously difficult to achieve. As we’ve explored in articles like 5 Prompt Optimization Strategies That Actually Improve LLM Output, the ability to precisely define and control AI behavior is critical to unlocking its full potential, and PAW's approach offers a novel avenue for accomplishing this. The project's architecture, which leverages a larger model to generate specialized LoRA adapters for a smaller, frozen interpreter, is particularly intriguing, demonstrating a clever trade-off between compilation complexity and runtime efficiency. The success of PAW on the FuzzyBench dataset, outperforming direct prompting of significantly larger models, underscores the power of this decoupled approach.
The core innovation of PAW lies in its ability to transform natural language descriptions into executable neural programs that can run locally, even on CPUs. This removes the dependency on external APIs and associated latency, opening up possibilities for offline processing and greater data privacy. The “Compile by Training” extension, which further refines the generated adapters through a short training loop, highlights the project’s iterative development and potential for continuous improvement. This approach resonates with the broader trend of resource-efficient AI, as demonstrated by projects like [I trained a 44M parameter quantized LLM from scratch on 45B tokens. It ships in 19.8 MB and runs at ~1,900 tok/s on CPU. [P]](/post/i-trained-a-44m-parameter-quantized-llm-from-scratch-on-45b-cmu2zdxjz0fjbrgedx4250r36), where minimizing model size and computational requirements is paramount. The ability to create and distribute these specialized neural programs as reusable components, composing them with standard code, promises to significantly streamline development workflows and empower users with greater control over their AI-powered applications. The researchers' emphasis on open-sourcing the code and model weights, along with the availability of a demo playground, further encourages community engagement and experimentation.
Beyond the immediate technical merits, PAW’s vision of “large models becoming tool builders” is particularly compelling. The prospect of describing a desired function and receiving a small, executable neural program in return represents a paradigm shift in how we interact with and leverage AI. This moves beyond the current model of prompting and fine-tuning towards a more declarative approach, where users define *what* they want, and the AI handles the *how*. While still in its early stages, the project’s results, especially the semantic accuracy gains achieved on the FuzzyBench-Hard dataset, suggest a significant step towards this goal. The ability to quickly iterate on function specifications, inspect errors, and refine the program through techniques like Finetune, points toward a future where AI assists in building customized, highly specialized tools tailored to specific needs. This is further highlighted by advancements in clean tech, as seen in Clean tech startup Fluxnium found a way to tap 50,000 years’ worth of nuclear fuel, where innovation is driven by the need for efficiency and resource optimization.
The ongoing development of PAW and similar projects raises a fundamental question: Will we see a move away from monolithic, general-purpose AI models towards a future dominated by smaller, specialized neural programs, each designed for a specific task and easily composable into larger systems? The success of PAW hinges on continued improvements in compiler efficiency, the robustness of the generated programs, and the ease with which users can create and manage these specialized components. However, if these challenges can be addressed, PAW could represent a pivotal moment in the evolution of AI, enabling a more modular, accessible, and ultimately more powerful approach to data management and application development.
| | Given the recent interest in tools like Jev, I wanted to share ProgramAsWeights (PAW), an open-source research project I'm working on at the University of Waterloo. You describe a text function in English, compile it into a reusable neural program, and run it locally, including on a CPU. For example: This example uses our hosted compiler for compilation (you can host your own compiler if you have a GPU using our released model weights). Once the program and local runtime are downloaded, subsequent calls run on your machine without an external API. The idea: separate compilation from inferenceIn many applications, the task stays fixed while the inputs keep changing. You define what makes an email urgent once, then apply that definition to thousands of emails. There are two jobs here: understanding the function you want, and executing it repeatedly. Our idea is to train a larger model to generate the task-specific weights that let a smaller model do the second job. The resulting function can be saved, distributed, and composed with ordinary code. How it worksOur standard compiler uses a finetuned Qwen3-4B model to generate a LoRA adapter for a frozen Qwen3-0.6B model, which we call the interpreter. Different functions use different adapters with the same base interpreter. A neural program contains two components:
The adapter-generation mechanism is similar to text-to-LoRA (Charakorn et al., 2025). To train the compiler, we use triples of Gradients flow through the frozen interpreter into the compiler and its adapter-generating layers. The interpreter's base weights remain unchanged. This is one of the parts I still find surprising: the same frozen 0.6B model can become much more useful just by changing the program we load onto it. We're training the compiler to discover how to specialize it. After training, the standard compiler predicts adapter weights for a new specification directly. Compilation takes seconds, and the larger compiler is no longer needed when processing new inputs. ResultsWe built FuzzyBench, a synthetic dataset of specification/input/output examples covering classification, extraction, parsing, format conversion, and other text functions. The train/validation/test split is by specification, so test specifications are unseen during compiler training. On FuzzyBench, PAW with the 0.6B interpreter reaches 73.4% exact-match accuracy, compared with 68.7% for direct prompting of Qwen3-32B. Our first paper includes the other benchmarks, baselines, and ablations. A higher-accuracy compilation modeBecause the generated program contains a LoRA adapter, we can also use it as an initialization for further training. Our follow-up, Compile by Training, does this automatically: teacher models synthesize task-specific examples, then we finetune the generated adapter for 100 steps. This takes roughly a minute in our deployment and produces the same reusable program format for the same local interpreter. On FuzzyBench-Hard, a subset selected from specifications where the original PAW evaluation produced no exact matches, this reaches 83.6% semantic accuracy. The two compilers offer different compile-time/accuracy tradeoffs. Both produce functions that run locally without teacher calls afterward. Trying it and building on itMy recommended workflow is to handwrite a small validation set, compile a specification, and inspect the errors. Then revise the specification or try the Finetune compiler. My longer-term hope is that large models can become tool builders: describe the function you need, get back a small neural program, and use it as part of your own software. This is what makes me excited about PAW beyond any individual benchmark. I'd love to see people build on this, whether through better compilers, different interpreters, or applications we haven't considered. Happy to answer questions about the architecture, training details, and failure cases. Papers
Code and models [link] [comments] |
![[link]](https://i.redd.it/la7iukj3vjqh1.png){kind=link}
Read on the original site
Open the publisher's page for the full experience