Hybrid AI Setup: Local Server, MacBook Pro, and the Things That Matter

• A private local inference server
• A MacBook Pro with M5 Pro and 48GB of unified memory
• Hosted models through OpenRouter and Z.ai

The local server handles inference. The MacBook is where I work: agents, editor, terminal, and smaller offline models.

Hosted models handle most of my interactive coding.

The MacBook gives me mobility and control. The local server gives me private capacity. Hosted models give me speed when latency matters.

Why I use a dedicated local server

The reason is simple: enough fast shared memory for larger local models.

I plan around roughly 96GB for inference, leaving the rest for the operating system, runtime, cache, and other processes.

That is enough for comfortable 27B and 35B models, useful 70B quantizations, and some larger sparse models.

But fitting a model and wanting to use it interactively are different things.

A strong local model running at 20–30 TPS is impressive. Inside an agentic coding loop, it can still feel slow. The agent reads files, reasons, edits code, runs tests, inspects failures, and starts again.

Latency compounds.

For unattended work, that speed is fine.

For something I use all day, I want faster.

Keep the server simple

The first thing I install is Tailscale.

curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up --ssh

After that, I manage the local server remotely from the MacBook.

For inference, I start with a current llama.cpp build and establish a stable baseline before optimizing backends or quantizations.

For an always-on server, I prefer llama-server because it gives me direct GGUF support and an OpenAI-compatible API.

Start with 32K context and increase it when the workload actually needs more.

The local models

I do not want many models sitting around.

I want a few models with clear jobs.

Qwen3.6-35B-A3B is the fast local worker. It makes sense for periodic agent jobs, repository analysis, research, document processing, and tool-heavy automation.

Qwen3.8-27B is the stronger local option. It is capable enough for serious reasoning and coding, but I still would not make it my primary interactive coding model.

I also keep an uncensored model around for isolated refusal research and red-team experiments. It stays isolated and does not get production access.

Hosted models are my coding layer

For daily coding, I mostly use OpenRouter.

Agentic coding is extremely sensitive to latency. A single task can involve multiple rounds of reading, reasoning, editing, testing, and correcting.

The difference between a local model and fast hosted inference gets multiplied across that entire loop.

I choose the runtime around the workload. Privacy-sensitive, autonomous, research, personal jobs stay local. Interactive work that benefits from lower latency or stronger models goes to hosted inference.

Where the local becomes useful

The local makes much more sense for autonomous work.

Hermes can point at the local endpoint and run periodic jobs such as repository analysis, research, document processing, monitoring, or other workflows that do not need an immediate answer or frontier model.

If a job takes ten minutes instead of thirty seconds, I do not care.

What the MacBook is for

The MacBook is the control plane.

I use it to:

• Run OpenCode, Hermes, and other agent clients
• Edit code and documents
• Connect to the local AI server over tailscale
• Route interactive coding to OpenRouter or Z.ai
• Run smaller local models through MLX or llama.cpp
• Inspect and control autonomous jobs running on the local server

The 48GB of unified memory also gives me a useful offline fallback without turning the laptop into a dedicated inference machine.

The laptop exists to work on.

I do not want every available gigabyte occupied by model weights.

The final setup

There are really two main paths.

For daily development:

MacBook → OpenCode → OpenRouter / Z.ai → hosted model

For autonomous local work that can tolerate latency:

**Hermes → local endpoint **

I put each workload where it makes sense. Daily coding uses hosted inference for speed and access to frontier models. Periodic agents run locally for privacy and no API cost.

Personal financial records, payment details, private documents, and other sensitive data stay on infrastructure I control. Account credentials remain outside the model context and are exposed only through restricted tools when a task needs them.

This setup gives me fast models for daily work and a private environment for work that should not leave my network.