Remote endpoints
Remote endpoints let LlamaBoss use hosted or LAN inference services alongside local GGUF models. They are optional and explicitly configured under Settings.
Endpoint and Connection are separate
An Endpoint stores the base URL, chat path, authentication type, tool protocol, and model list. A Connection stores—or references—the API key used by that endpoint.
LlamaBoss ships with a default OpenRouter endpoint definition. Add the corresponding Connection secret before selecting one of its models.
Configure a Connection
Open Settings → Connections → Manage. Provider/key names become an injected environment-variable name for Skill scripts, such as OPENROUTER_API_KEY or GMAIL_API_KEY.
You can choose:
- Direct value: LlamaBoss stores the value in
%LOCALAPPDATA%\LlamaBoss\secrets.json. - Environment-variable reference: LlamaBoss stores the variable name and resolves the actual value from Windows at runtime.
Direct secret values are stored as plaintext JSON with a user-only file ACL. They are not currently protected with DPAPI or Windows Credential Manager. Prefer an environment-variable reference when that is appropriate for your setup.
Add or edit an endpoint
Open Settings → Remote Endpoints → Manage. Configure:
- Endpoint ID and display name
- Base URL
- Chat path, normally
/v1/chat/completions - Bearer token or
X-API-Keyauthentication - Connection provider and key name
- Tool protocol
- One or more model IDs
Model-list syntax
Enter one model per line:
provider/model-id = Friendly display name
google/gemini-image-model = Friendly image model [image]The display name is optional. Appending [image] marks a model as an image-output model. LlamaBoss then requests image output and presents returned image artifacts instead of treating the model as ordinary text-only chat.
Tool protocols
| Mode | Use |
|---|---|
| Native function calling | For providers/models that support OpenAI-style tool schemas and tool-call responses. |
| XML text protocol | For models that follow LlamaBoss's text-based <tool_call> contract more reliably than native function calling. |
Tool support still depends on the selected model. A remote endpoint can accept chat requests while its model fails to call tools reliably.
What is sent
When a remote model is selected, the request can include your message, recent conversation context, Project/Skill instructions, tool schemas, tool results, and attached images or document-derived content needed for the task. The endpoint provider receives that data under its own terms.
Switch back to a local GGUF model for offline inference. See Privacy & data.