A native C++ harness for local AI.
LlamaBoss runs GGUF models on your own GPU through llama.cpp, in a native Windows and Mac app written in C++. No Electron, no Docker. Its agent shows its work in the chat, checks native file changes against shared folders, and asks you to review risky actions.
<, it stops and reports that version.json returned a web page, not JSON. A bad deploy shows a clear message instead of a parse error.A desktop app, not a black box.
Everything the model does shows up in the chat as it happens: what it searched, what it read, what it wants to run. You stay in charge of anything that changes your machine.
Review risky actions
Each chat gets its own workspace folder. Inside it, and inside any Project you attach, the agent can read, search and write freely, and every step shows up in the chat.
With normal approval settings, deleting files, PowerShell beyond simple read-only commands, arbitrary Python code, running scripts and package installs ask first. Native writes outside shared folders need a folder grant. Allow once, trust the chat for this app session, or deny. Approving a new script also covers its first run; review imported Skills before using them.
Big files don't flood the conversation
When a file or tool result is too large to paste into the prompt, LlamaBoss saves it to disk and hands the model a short card instead: size, line count, structure, and a preview.
The model then searches inside it and reads only the lines it needs. Small local models can work through files many times larger than their context window.
Many windows, one model in memory
Open as many chat windows as you like. They share a single local model service, so your GPU loads the model once instead of once per window.
Each window can also talk to a different remote model at the same time without getting in each other's way.
Your GPU first, the cloud when you want it
LlamaBoss ships with llama.cpp for CPU and NVIDIA CUDA on Windows and Metal on Mac, so local chat needs no Ollama or Docker. The Mac version also includes its own Python for the document tools. Download a curated model or point it at your own GGUF folder.
When you want a bigger model, add OpenRouter, OpenAI or any OpenAI-compatible server. Local and remote models sit in the same picker.
Projects, Workflows and Skills
A Project keeps sources and workflows for a line of work and can be attached to any chat. Skills are reusable instructions you can import, export and share as a zip. Project Workflows keep repeatable plans close to the files they use.
Projects and Skills live in ordinary Windows folders you can open, edit and back up.
What touches the network, and when.
Local chat never leaves your computer. Downloads, providers and tools can use the network. Code and scripts you approve can also connect to services.
- Chatting with a local modelnever
- Remote modelswhen you pick one
- Model downloadswhen you start one
- Updateswhen you check or install an update
- Fetching a web pagewhen the agent asks for one
- Installing Python packagesafter you approve
Download LlamaBoss.
On Windows, run the installer. On a Mac, open the disk image and drag LlamaBoss to Applications. Then pick a model and start chatting. Local chat needs no account; provider connections and ChatGPT sign-in are optional.
Prefer to build it yourself? The full C++ source is on GitHub.
- VersionBeta 0.1.25 (Windows), 0.1.25 (Mac)
- Windows10 or 11, x64; NVIDIA CUDA or CPU only
- MacApple silicon, macOS 13.3 or later; Metal
- LicenseMIT