Tagged "large-model-inference"
13 articles tagged large-model-inference, 11 February 2026 to 28 September 2026. Newest first.
-
Perplexity Brings On-Device AI to AMD Ryzen AI Max PCs, Running 27B Models Locally
Perplexity has enabled on-device AI inference on AMD Ryzen AI Max processors, allowing users to run 27-billion parameter models entirely locally. This demonstrates practical large-model deployment on consumer-grade hardware without cloud dependencies.
-
Running 104GB Qwen3.8-Flash-Next on 48GB Mac at ~12 tok/s
A developer demonstrates running a 104GB model on a 48GB Mac using innovative slot streaming techniques, achieving practical inference speeds of ~12 tokens/second and expanding the possibilities for large model deployment on consumer hardware.
-
M5 Max MacBook Runs Local Large Language Models Efficiently
Testing demonstrates that Apple's M5 Max processor effectively handles local large language model inference with strong performance characteristics. The MacBook's unified memory architecture proves particularly well-suited for efficient LLM execution without dedicated accelerators.
-
AMD's New Ryzen AI Max Pro 400 with 192GB LPDDR5X Memory
AMD reveals the Ryzen AI Max Pro 400 series processors featuring 192GB of LPDDR5X memory, significantly expanding on-device LLM deployment capabilities for enterprise and professional workloads.
-
Dynamic Expert Cache in llama.cpp Achieves 27% Faster Inference on Large MoE Models
A new optimization technique for llama.cpp improves CPU+GPU token generation speed by 27% on Qwen3.5-122B through dynamic expert caching, raising practical inference rates from 15 to 23 tokens per second.
-
Qwen 3.5 122B Achieves 198 Tokens/sec on Dual RTX PRO 6000 Blackwell GPUs
A detailed optimization case study demonstrates running Qwen 3.5 122B at impressive inference speeds on a budget dual-GPU Blackwell setup. The community shares verified benchmarks with full methodology and reproducible results for large-scale local deployment.
-
Ollama Gets Blazing Fast on Macs with Full MLX Support and 2× Speedups
Ollama has integrated full MLX support for macOS, delivering up to 2× performance improvements and NVIDIA-quality 4-bit quantisation inference on Apple silicon. This major update significantly accelerates local LLM inference for Mac users.
-
TinyGPU Adds Mac Support for External Nvidia GPU Acceleration
TinyGPU framework now enables Mac users to leverage external Nvidia GPUs for local LLM inference, expanding deployment options for Apple silicon users.
-
Intel's $949 GPU has 32GB of VRAM for local AI, but the software is why Nvidia keeps winning
Intel's new discrete GPU offers compelling hardware specs for local AI workloads at competitive pricing, but software ecosystem and driver maturity remain critical challenges compared to Nvidia's dominance.
-
Qwen3.5-397B Achieves 282 tok/s on 4x RTX PRO 6000 Blackwell Through Custom CUTLASS Kernel
A developer achieved a 5x performance improvement on the massive Qwen3.5-397B model by building a custom CUTLASS kernel to fix SM120's broken MoE GEMM tiles, reaching 282 tokens/second on Blackwell GPUs. This breakthrough demonstrates significant optimization potential for running large models locally with multi-GPU setups.
-
Apple M5 Max 128GB Benchmark Results for Local LLM Inference
Community member benchmarks the new Apple M5 Max 128GB laptop for local LLM inference, providing real-world performance data for Apple Silicon's latest generation. Results demonstrate viability of premium consumer hardware for serious local deployment.
-
M5 Max and M5 Ultra Chipsets Demonstrate Significant Bandwidth Improvements for Local LLM Inference
Apple's newest M5 silicon generations offer substantially improved memory bandwidth compared to prior generations, enabling practical deployment of larger models on MacBook hardware with competitive inference throughput.
-
Community Member Builds 144GB VRAM Local LLM Powerhouse
A LocalLLaMA community member showcases a custom-built system with 6x RTX 3090 GPUs providing 144GB of VRAM, featuring modified drivers with P2P support for high-performance local LLM inference.