Small Model That Writes Bank Payment Files
What it does
A Qwen3-4B fine-tuned with LoRA (trained on Modal) that turns a plain-language payment spec into a NACHA ACH file. That’s the strict 94-character fixed-width format US banks use for ACH payments. One wrong character fails the file. A code validator checks every output for exact correctness, and a Claude Haiku judge scores spec adherence.
- Code-generated specs6,000 examples
- Exact validatorkeeps only valid files
- LoRA fine-tuneQwen3-4B on one GPU
- Validator + judgeexact match, spec adherence
- Runs locally18 of 20 hardest specs
Results (exact-match files)
| Model | Hardest 20 specs |
|---|---|
| This 4B model, running locally | 18 |
| Claude Opus, best prompt | 17 |
| GPT-5.5 | 3 |
| Claude Sonnet | 1 |
The current model scores 45/50 on the eval set and 99/100 on the holdout set, against 0/50 for the untuned base. It now handles originator names it never saw in training (9 of 10, up from 0 of 10). The remaining gap is larger files with unseen names (11 of 50), which is the focus of current work. A self-training round didn’t beat it, so I dropped it. Training takes about five hours on one GPU, using 6,000 code-generated files.
How it was built
- Find a real gap first. I tested candidate tasks like financial statement spreading and chest X-ray reading and dropped them because frontier models were already good enough. Exact fixed-width banking files were not.
- Generate data by code, filter with an exact checker. Every training example is generated by code and kept only if the validator passes it.
- One command reproduces the results table comparing the base and tuned models.
Tools
Python, Qwen3-4B, LoRA, Modal, vLLM, Hugging Face, Claude Haiku as judge