Running GLM-4.7-Flash Locally with OpenCode

On this page
GLM-4.7-Flash from Zhipu AI is a 30B Mixture-of-Experts coding model that
runs on a single 16GB GPU with Q4 quantization. This guide shows how to
build llama.cpp for NVIDIA or AMD GPUs, serve GLM-4.7-Flash with
llama-server, and connect it to OpenCode for agentic coding on your own
hardware. GLM-4.7-Flash is a lightweight model optimized for coding tasks,
based on the GLM-4.7 architecture.
Why GLM-4.7-Flash
Zhipu AI's GLM-4.7-Flash is a 30B parameter model with a Mixture-of-Experts (MoE) architecture. Only about 3B parameters are active per forward pass, so you get the reasoning capacity of a much larger model on hardware you might already own.
For coding work, the benchmark results are what make it interesting. On SWE-bench Verified, the standard test for whether a model can actually fix real GitHub issues, GLM-4.7-Flash scores 59.2%. That's nearly triple what Qwen3-30B-A3B-Thinking-2507 scores (22.0%) and well ahead of GPT-OSS-20B (34.0%). The τ²-Bench results tell a similar story: 79.5% versus 49.0% and 47.7% for the other two.
GLM-4.7-Flash benchmark comparison
These aren't cherry-picked numbers. Across six benchmarks (SWE-bench Verified, τ²-Bench, BrowseComp, AIME 25, GPQA and HLE), GLM-4.7-Flash either leads or stays competitive. The model was clearly tuned for agentic workloads: tool calling and multi-step reasoning, the tasks that matter when you use it as a coding assistant.
The practical upside: with Q4 quantization, GLM-4.7-Flash runs on a 16GB GPU. The weights are open and MIT-licensed. There are no API costs and no rate limits, and you control the whole inference stack. (For how far quantization can go, see our post on BitNet and 1.58-bit LLMs.)
Prerequisites
- NVIDIA GPU with CUDA 12+ OR AMD GPU with ROCm 6+
- 16GB+ VRAM (for the Q4 quantized version)
- About 20GB disk space for the model
Building llama.cpp
Pre-built llama.cpp binaries exist, but building from source gets you the latest optimizations and proper GPU support. The build differs between NVIDIA and AMD.
NVIDIA (CUDA)
Install dependencies and build with CUDA enabled:
AMD (ROCm)
AMD requires ROCm to be installed first. On Ubuntu (Noble):
On Arch Linux:
Then build llama.cpp with HIP support. You need to specify your GPU
architecture in AMDGPU_TARGETS. Find it by running rocminfo and
looking for the Name field:
*******
Agent 2
*******
Name: gfx1151 <-- this is your target
Marketing Name: Radeon 8060S Graphics
Vendor Name: AMD
Build with your target:
Run the GLM-4.7-Flash server
llama.cpp can download models directly from Hugging Face. First, set up a Hugging Face token, because some model repos require authentication. Create one at huggingface.co/settings/tokens and export it:
Then start the server:
The -hf flag pulls the model automatically on first run. UD-Q4_K_XL is
a good balance between quality and VRAM usage. Expect around 12-14GB with
this quant. If you have 24GB+ VRAM, swap it for a Q8 variant.
The flags that matter most:
--alias glm-4.7-flash: the model name OpenCode will use--ctx-size 32768: context window (adjust based on your VRAM)--jinja: enables chat template support--sleep-idle-seconds 300: unloads the model after 5 minutes idle to free VRAM
Verify it's running:
You should get {"status":"ok"}.
Test the model
Send a request to check that the model responds:
You should see streamed JSON chunks with a Python hello world snippet. If that works, the server is ready for OpenCode.
Configure OpenCode for the local model
Add the provider to your opencode.json (or
~/.config/opencode/opencode.json for global config):
The model ID (glm-4.7-flash) must match the --alias you passed to
llama-server. Run /models in OpenCode and select GLM-4.7-Flash (local).
OpenCode answering a test prompt with GLM-4.7-Flash
Conclusion
That's it. You now have a local coding assistant that rivals closed-source models on agentic benchmarks and runs entirely on your own hardware. No API keys to manage, no usage limits, no data leaving your machine.
The MoE architecture makes GLM-4.7-Flash practical where other 30B+ models aren't. If you've got a 16GB GPU sitting around, this is one of the better uses for it. If you'd rather serve a larger reasoning model on two GPUs, our guide to running DeepSeek R1 with SGLang covers that setup, and our vLLM and Caddy guide shows how to put a self-hosted model behind HTTPS.
Interested in building high-quality AI agent systems?
We prepared a comprehensive guide based on cutting-edge research for how to build robust, reliable AI agent systems that actually work in production. This guide covers:
- Understanding the 14 systematic failure modes in multi-agent systems
- Evidence-based best practices for agent design
- Structured communication protocols and verification mechanisms
Further reading
- DeepSeek R1 On-Prem Setup: Run Advanced AI Models on Your Hardware with SGLang
- How to Set Up a Secure, Self-Hosted Large Language Model with vLLM & Caddy
- How to Save on LLM Costs
- Saving costs with LLM routing: send simple prompts to a cheaper model
- Open WebUI: a self-hosted chat interface for local models



