Tagged "throughput-optimization"

8 articles tagged throughput-optimization, 26 February 2026 to 11 October 2026. Newest first.

• Local LLM Inference at Scale with vLLM 11 October 2026

  Deep dive into using vLLM for high-throughput local LLM inference, covering optimization techniques and scaling strategies for self-hosted deployments.

• vLLM Architecture, Memory and Benchmarks Deep Dive 23 September 2026

  An in-depth technical analysis of vLLM's architecture, memory management, and throughput characteristics, providing concrete benchmarks and optimization strategies for local LLM inference.

• Self-hosted Inference Orchestrators Compared: LocalAI, exo, GPUStack, vLLM 21 September 2026

  Comprehensive comparison of leading self-hosted LLM inference orchestration platforms, evaluating LocalAI, exo, GPUStack, and vLLM for on-device and distributed inference deployments.

• Squeezing Silicon Limits: Effective Strategies to Eliminate GPU Idle Time and Maximize GPU Utilization 2 August 2026

  Practical techniques for maximising GPU utilisation during local LLM inference, addressing idle time and throughput bottlenecks that waste expensive compute resources.

• Prefill Is Compute-Bound, Decode Is Memory-Bound: Optimizing GPU Utilization for LLM Inference 16 April 2026

  A deep dive into why GPUs shouldn't handle both prefill and decode phases equally, and how understanding this fundamental bottleneck can dramatically improve local LLM inference performance.

• Qwen 3.5 27B Achieves 1.1M Tokens/Second on B200 GPUs with Optimized vLLM Config 27 March 2026

  A developer optimized Qwen 3.5 27B to reach 1.1 million tokens per second on 96 B200 GPUs using vLLM, with detailed configurations and all settings published on GitHub. Key optimizations included distributed parallelism, reduced context windows, FP8 KV cache, and speculative decoding.

• P-EAGLE: Faster LLM Inference with Parallel Speculative Decoding in vLLM 14 March 2026

  AWS introduces P-EAGLE, a parallel speculative decoding technique integrated into vLLM that significantly accelerates LLM inference speed. This advancement is crucial for practitioners deploying local LLMs who need to optimize throughput and reduce latency.

• DeepSeek Releases DualPath: Addressing Storage Bandwidth Bottlenecks in Agentic Inference 26 February 2026

  A new paper from DeepSeek, Peking University, and Tsinghua University presents DualPath, a technique for breaking storage bandwidth limitations in agent-based LLM inference. The research tackles a fundamental performance constraint affecting local deployment at scale.