Wren

Wren is a small attention-only language model designed for tool calls. It started out as an experiment to teach myself about how language models work at the very foundation. Inspired by needle, the goal of this model is to call predefined tools (targets) given an input prompt. I.e. "Turn on the kitchen light" should cause the model with high confidence to set the corresponding vector entry to true.

The github repo provides a pre-trained model and setup intructions. Training your own model can easily be done on CPU and is explained in the documentation.

Being a very small model, we can achieve ultra-fast inference with just a tiny memory footprint:

Measurement	Value
Parameters	1.12M
Pack size	535 kB
Median request latency	~52 ms, single thread (BLAS)
End-to-end training	~40 m (20-core CPU)
Saved confidence threshold	~0.973