A local LLM API client for macOS

Calling an LLM API is simple until it is not: streaming chunks arrive half-formed, Anthropic wants a different message shape than OpenAI, Gemini’s base URL looks nothing like either, and your “quick curl” has grown a twenty-line header block you no longer trust.

A local LLM API client is a desktop workbench for that job. It is not a chatbot with a key pasted into settings. It is a request composer that speaks provider APIs, shows the raw wire, and keeps credentials on your machine.

What you need from the client

Direct transport. Requests should leave your Mac for api.openai.com, api.anthropic.com, Gemini, OpenRouter, or your own compatible host — not through a product backend that re-implements auth.

Provider templates plus escape hatches. Built-ins for the big hosts save time. Editable base URL, auth type, custom headers, chat endpoint, and model list keep you honest when a gateway or self-hosted server is almost OpenAI-compatible but not quite.

Streaming as a first-class view. Token timers and “assistant said…” bubbles are nice; the chunk stream and final body are what you debug when JSON mode fails or a tool call truncates.

Multi-turn and extras. Role-based messages, images, sampling parameters, and per-request headers should be editable without leaving the composer.

Export. Once a request works, you want Node, Python, or cURL that matches what you validated — not a reinvented SDK sample.

Local-first vs browser-hosted testers

Hosted API consoles and shared cloud workspaces are convenient for demos. They are a poor default for production keys and proprietary prompts:

Concern	Local client	Hosted tester
API keys	On your Mac	On their server (or in a shared vault)
Prompt / response retention	Your disk	Their logs/policies
Offline drafts	Available	Needs the tab
Custom / internal endpoints	Point at your URL	Often blocked or awkward
Multi-provider compare	Same app session	Usually one vendor console

If your threat model treats provider keys like production secrets — and it should — keep the client local.

A short setup checklist

1. Install a signed macOS build (Developer ID + notarization, or Mac App Store).
2. Add a provider template and paste the key you already use in apps or CI.
3. Send a minimal chat completion and confirm streaming + headers look right.
4. Save the request into a collection or environment so tomorrow’s debug starts from a known-good baseline.
5. When you need to pick a model, fan the same request out — see multi-model comparison.

How this differs from Postman

Postman is excellent general HTTP. LLM APIs add streaming UX, multi-turn message editing, provider-specific auth, and “run this across three models” that a generic collection only approximates. The trade-offs are spelled out in Roshi vs Postman for LLM APIs.

Try Roshi

Roshi is a local-first macOS LLM API workbench: compose requests, stream responses, inspect headers and latency, generate code, and keep keys on disk. Free via GitHub Releases; optional Mac App Store build for automatic updates.