Tagged "kv-cache-management"

5 articles tagged kv-cache-management, 14 February 2026 to 10 September 2026. Newest first.

• vLLM 0.29.0 Makes Model Runner V2 the Default for All Models 10 September 2026

  vLLM 0.29.0 marks a major milestone with Model Runner V2 becoming the default inference engine across all model types, bringing CUDA graph memory profiling and batch-shard optimizations to self-hosted LLM deployments.

• Sorting 1M u64 KV-Pairs in 20ms on i9-13980HX Using Branchless Rust Implementation 18 April 2026

  A deep dive into extreme performance optimisation for in-memory operations using branchless Rust code, achieving sub-20ms throughput for million-element datasets. Directly applicable to KV-cache and token management in local LLM inference.

• GPU Memory for LLM Inference (Part 1) 6 April 2026

  A detailed technical guide exploring GPU memory optimization strategies for running large language models efficiently during inference, critical knowledge for anyone deploying LLMs locally with limited VRAM.

• Intel Updates LLM-Scaler-vLLM With Support For More Qwen3/3.5 Models 13 March 2026

  Intel has expanded LLM-Scaler-vLLM compatibility to include additional Qwen3 and Qwen3.5 models, improving inference optimization for self-hosted deployments on Intel hardware.

• NVIDIA's Dynamic Memory Sparsification Cuts LLM Inference Costs by 8x 14 February 2026

  NVIDIA introduces Dynamic Memory Sparsification technique that reduces LLM reasoning costs by 8x through intelligent KV cache management without accuracy loss.