Guides

Step-by-step tutorials for running AI locally. Contribute a guide on GitHub.

Getting Started

• Installing Ollama on Linux beginner 10 min

  Get Ollama up and running on any Linux distribution in under ten minutes.

Hardware

• How Much Context Actually Fits in Your VRAM intermediate 20 min

  The KV cache arithmetic for any model, read straight from its config.json — plus the four architectures that break the standard formula, one of them by 57x.

• What Actually Fits on Dual RTX 3090s: Qwen 27B and the KV Cache Math advanced 25 min

  Why a 27B model holds 262K context on 48GB — 48 of its 64 layers have no KV cache at all — and what decode speed you should honestly expect from two 3090s.

Deployment

• Choosing a Qwen3.8-27B Quantization and Backend: What Actually Fits intermediate 20 min

  No Q4_K_M build of Qwen3.8-27B fits in 16GB from any repository, the quant that does fit has never been quality-tested, and llama.cpp silently stops generating at ~98K context. The measured file sizes and the open bugs behind each decision.

• Controlling Reasoning Token Budgets in llama.cpp intermediate 20 min

  Cap how many tokens a reasoning model spends thinking — with server flags, undocumented per-request fields, and a mid-stream interrupt. Includes what it costs you in throughput.

• Running Prime Agent on a Local Model intermediate 30 min

  Point prime-agent at Ollama or vLLM with no Prime Intellect account: the models.json schema, which compat flags matter for which backend, why to disable auto-refine on small models, and the sandbox and telemetry defaults you should change.