Tagged "gguf-format"

6 articles tagged gguf-format, 1 April 2026 to 15 September 2026. Newest first.

• Qwen3.8-Flash-Next achieves efficient inference on dual RTX 3090s via non-uniform quantization 15 September 2026

  A community-optimized GGUF quantization of Qwen3.8-Flash-Next demonstrates that large instruction-tuned models can now run efficiently on accessible consumer hardware through advanced quantization techniques.

• Qwen3.8-Flash-Next Added to llama.cpp with GGUF Support 28 August 2026

  llama.cpp now supports Qwen3.8-Flash-Next with full GGUF architecture implementation, including low-rank hyper-connections and n-gram hash embeddings for optimized local inference.

• Qwen 3.8 27B Successfully Runs on 16GB RAM Using LM Studio 16 August 2026

  Community testing confirms Qwen 3.8 27B operates efficiently on 16GB systems with LM Studio, making a capable 27-billion parameter model accessible to users with modest hardware. Quantised GGUF weights enable practical local deployment without expensive GPUs.

• What else is included in the 'GGUF' file format used by llama.cpp for AI language models, besides weights? 21 June 2026

  An in-depth technical analysis of the GGUF format ecosystem, exploring the metadata, configuration, and structural components beyond model weights. Understanding GGUF is essential for practitioners working with llama.cpp and quantized model deployment.

• MiniMax M2.7 Released: New Model Available for Local Deployment 12 April 2026

  MiniMax has released the M2.7 model, generating significant interest in the LocalLLaMA community with rapid quantization support from Unsloth and other contributors. However, the model comes with restrictive licensing that prohibits commercial use without prior written permission.

• Llama.cpp Merging TurboQuant Lite (attn-rot) with Major Performance Gains 1 April 2026

  ggerganov's TurboQuant lite (attn-rot) quantisation method is on the verge of being merged into llama.cpp, showing significant improvements in KL-divergence and inference quality. Benchmarks on Qwen3.5-35B demonstrate superior performance across multiple quantisation levels, promising faster and more accurate local inference.