Local AI Setup — GTR9 Pro, MacBook Pro, and the Things That Matter

• A Beelink GTR9 Pro with an AMD Ryzen AI Max+ 395, Radeon 8060S, 128GB of LPDDR5X memory, and a 2TB SSD
• A MacBook Pro with M5 Pro and 48GB of unified memory

The GTR9 Pro is the inference server. The MacBook runs my agents, editor, terminal, and smaller offline models.

One machine provides capacity. The other provides mobility and capacity.

Why the GTR9 Pro is the server

The GTR9 Pro works as an inference server because it has 128GB of fast shared memory.

The operating system, runtime, and context cache also need memory. I plan around roughly 96GB for inference.

That comfortably handles 27B and 35B models, useful 70B quantizations, and some larger sparse models.

Install Tailscale first

First, give the GTR9 an internet connection, install Tailscale, and enable SSH.

curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up --ssh

Then switch back to the MacBook and finish everything remotely. That is it.

Establish a Vulkan baseline first

On Strix Halo, I would begin with a current llama.cpp build using the Vulkan backend.

ROCm can improve prompt processing and concurrency, but Radeon 8060S support still depends on exact kernel, Mesa, and ROCm versions. Current Strix Halo testing finds Vulkan/RADV easier to keep stable for single-user inference.

Get a correct Vulkan baseline first. Optimize with ROCm only after you have a workload worth optimizing.

Ollama or LM Studio is easier for beginners. For an always-on server, I prefer llama-server: direct GGUF support and an OpenAI-compatible API.

A minimal server looks like this:

llama-server \
  --model /models/qwen3.8-27b-q6_k.gguf \
  --n-gpu-layers 999 \
  --ctx-size 32768 \
  --host 100.x.x.x \
  --port 8080

Replace 100.x.x.x with the GTR9's Tailscale IP. Start with 32K context. Larger windows consume more memory and make prompts slower.

The models I would actually keep

I keep a small set of models with clear jobs.

Qwen3.6-35B-A3B: the fast default

Qwen3.6-35B-A3B has 35B total parameters but activates about 3B per token. It is my fast default for terminal work, tools, drafts, and everyday coding.

All 35B parameters must still fit in memory. I use Q6 or Q8 on the GTR9 and a smaller quant on the MacBook.

Qwen3.8-27B: the quality default

Qwen3.8-27B is my quality default. It is a dense 27B model with reasoning, vision, tool use, and a 262K native context window.

On the GTR9, I use ROCmFP4-STRIX-MTP. It targets the Ryzen AI Max+ 395 and reaches roughly 30 TPS with tuned MTP at 8K context. A standard Q4_K_M route reaches about 20 TPS in separate GTR9 testing.

It needs the matching runtime and MTP settings. I keep Q4_K_M as the stable fallback.

On the MacBook, I use OptiQ-4bit. It is MLX-native and occupies about 19GB. Most layers use 4-bit precision; sensitive layers remain at 8-bit.

I would rather run a strong 27B model with a hardware-specific quantization than force a 70B model into Q2 and call it an upgrade.

An uncensored option for isolated research

For refusal research or red-team experiments, I keep Qwen3.8-27B-Uncensored-GGUF. Its publisher provides Q4 through Q8 GGUFs for research use.

An uncensored model gives fewer refusals and potentially worse judgment. I isolate it, sometimes disable powerful tools, and never give it production access.

GLM-5.2: use the API

GLM-5.2 is strong at long-horizon coding and agent work. It has a 1M context window and a 744B-A40B MoE architecture. That scale requires hosted or multi-node infrastructure.

Only 40B parameters activate per token, but all 744B weights must be stored. Even 2-bit weights start around 186GB before overhead.

I use it through Z.ai or another trusted provider when its capability justifies leaving the local boundary.

This is why my setup is hybrid:

local by default, hosted when capability clearly wins.

What the MacBook is for

The M5 Pro MacBook has 48GB of unified memory and 307GB/s bandwidth according to Apple. I keep enough capacity free for the editor, agents, and the rest of macOS.

Its primary job is to be the control plane:

• Run OpenCode, Hermes Agent, or another client
• Edit code and documents
• Connect to the GTR9 over Tailscale
• Use MLX or llama.cpp with Metal for an offline fallback
• Route exceptional tasks to GLM-5.2 or another hosted frontier model

For travel, Qwen3.6 is comfortable. Qwen3.8 OptiQ is the power-user option. Compressed 70B models consume too much of the machine for my daily workflow.

Storage and operations

Two terabytes sounds large until every model has five quantizations.

I keep one or two quantizations per model and delete the rest. The GTR9 has a second M.2 slot, so storage can grow later. Active weights stay on local NVMe.

The server runs under systemd, restarts on failure, and binds only to Tailscale. I keep one health check:

curl http://100.x.x.x:8080/health

If that works, the network and server are alive. Any remaining failure is in the model or runtime.

The final setup

The complete architecture is intentionally boring:

MacBook → Tailscale → GTR9 Pro → llama-server → local model

The MacBook can fall back to its own model. Selected tasks can move to GLM-5.2 through an API. Nothing public points at the GTR9.

I want a fast default, a stronger local option, an isolated uncensored model, and a hosted frontier fallback.

That is the setup I want: local enough to control, powerful enough to use, and simple enough to maintain.