Turn text input into actions with Needle

Cactus Compute’s Needle 2 model runs entirely offline on a Raspberry Pi 5’s CPU. Turn text input into actions with Needle, a 14MB function-calling LLM. In this post, our friends from Cactus Compute show how Needle 2, a 14MB function-calling model, turns plain English into local actions on a Raspberry Pi 5, using the CPU alone.

Type “Turn the LED on” and 78 milliseconds later, the LED on our Raspberry Pi 5 comes on. That is Needle 2 running on the CPU alone – without a dedicated AI HAT. Instead, the team at Cactus Compute trained it for one task only: reliable, structured on-device actions.

“Needle 2 is rather excellent” – Eben Upton, CEO, Raspberry Pi The setup is simple – you declare Python functions, then Needle picks which one to call and fills in the arguments. We used it to write local notes, query vcgencmd, interact with the LEDs, and more on a CPU-only Raspberry Pi 5. Python3 -m venv needle-env source needle-env/bin/activate python -m pip install cactus-needle After the first download, Needle runs locally without a cloud API or network connection.

Import needle from pathlib import Path import subprocess notes_path = Path("needle-notes.txt") The Needle class accepts functions decorated with @needle.tool.