Tagged "gpu-acceleration"
22 articles tagged gpu-acceleration, 12 February 2026 to 10 October 2026. Newest first.
-
ML Drift: Google's Open-Source On-Device GPU Inference Engine
Google has released ML Drift, a new open-source GPU inference engine specifically optimized for edge and on-device deployment scenarios. The framework focuses on GPU efficiency for local inference workloads.
-
AMD ROCm vs. Vulkan Performance Benchmarking for Local AI Inference on Lemonade Server
Phoronix benchmarks AMD's ROCm and Vulkan backends for llama.cpp, comparing GPU acceleration performance for local inference on AMD hardware. The results provide practical insights for practitioners choosing between different GPU acceleration strategies.
-
Janus: New Go Binary Runs GGUF Models via Vulkan on AMD, Intel, and NVIDIA
Janus is a newly released Go binary that enables GGUF model inference through Vulkan, providing cross-platform GPU acceleration for AMD, Intel, and NVIDIA hardware without vendor-specific dependencies.
-
Llama.cpp Achieves 2.2x Faster Inference on Intel Arc GPUs
A developer reports significant performance improvements running llama.cpp on Intel Arc graphics cards, achieving 2.2x more tokens per second through optimizations. This breakthrough demonstrates Intel's viability as a cost-effective alternative to Nvidia for local LLM inference.
-
Hugging Face Releases 200+ WebGPU Kernels for Local AI Inference
Hugging Face launches a comprehensive collection of WebGPU kernels enabling efficient local AI inference directly in browsers and on-device. This represents a major step toward browser-native LLM deployment without server backends.
-
llama.cpp Adds CUDA Pool Operations Support
llama.cpp release b10589 introduces 1D pooling support for CUDA, expanding the inference runtime's capability to handle more complex model architectures on NVIDIA hardware.
-
Llama.cpp B10327 Fixes CUDA Quantized Copy Kernel Performance
The latest llama.cpp release addresses critical thread and block count issues in CUDA quantized copy kernels, improving inference performance on NVIDIA GPUs. This fix ensures more efficient parallel execution for quantized model operations.
-
llama.cpp Release b10257 – Vulkan LLVMpipe Fixes
Latest llama.cpp release fixes critical Vulkan LLVMpipe CI runs, continuing the project's focus on cross-platform GPU inference stability.
-
You Can Now Run Max AI Models on Apple Silicon
Modular's Max platform now supports running AI models directly on Apple Silicon GPUs, expanding local deployment options for macOS users and M-series chip owners.
-
WSL 3 Brings Near-Native GPU and NPU Passthrough for Local AI on Windows
Microsoft's WSL 3 at Build 2026 enables near-native GPU and NPU passthrough, making it significantly easier to run local LLMs on Windows with direct hardware acceleration. This development removes a major bottleneck for Windows-based local inference deployments.
-
NVIDIA RTX Spark Superchip Delivers 6,144 CUDA Cores for Consumer Local AI Inference
NVIDIA's new RTX Spark superchip combines 6,144 CUDA cores with a 20-core Grace CPU, targeting consumer and creator machines with unprecedented local AI performance. The chip architecture mirrors smartphone efficiency approaches while delivering desktop-class compute for on-device inference.
-
AMD's Lemonade SDK Advances macOS Support for Local AI Inference with ROCm 7.13
AMD promotes macOS to general availability status in its Lemonade SDK for AI, integrating ROCm 7.13 to enable GPU-accelerated local LLM inference on Apple Silicon and AMD-powered Macs.
-
llama.cpp Delivers Sharp Performance Gains for AMD RDNA3 Users
llama.cpp continues to expand GPU acceleration support with optimizations for AMD's RDNA3 architecture, enabling faster local inference on consumer graphics cards. This development significantly improves the accessibility of local LLM deployment for AMD GPU owners.
-
GPU Passthrough to LXCs in Proxmox Outperforms VMs and Simplifies Local AI Infrastructure
Advanced virtualization techniques enable efficient GPU passthrough to LXC containers in Proxmox, providing superior performance over traditional virtual machines for local LLM inference. This approach simplifies complex deployment scenarios.
-
Intel's Arc GPU Offers 32GB VRAM for Local AI, But Software Ecosystem Lags Behind
Intel's $949 Arc GPU provides impressive specifications for local inference with 32GB of VRAM, yet software maturity and framework support remain significant barriers compared to NVIDIA's ecosystem. Hardware capability alone insufficient without robust software integration.
-
Pluggable's TBT5-AI: First Thunderbolt Dock Explicitly Targeting Local LLM Workstations
Pluggable announces the TBT5-AI, a Thunderbolt 5 dock designed specifically for local LLM inference and GPU-accelerated workloads, addressing connectivity bottlenecks for distributed local inference setups.
-
Llamafile 0.10 Released with GPU Support and Rebuilt Core
Mozilla's Llamafile, the portable single-file LLM runner, reaches version 0.10 with enhanced GPU acceleration and a completely rebuilt inference core. This update makes it easier than ever to run large language models locally without complex dependencies.
-
This External GPU Enclosure Tries to Break Cloud Dependence for Local AI Inference
New external GPU enclosure hardware aims to democratize local AI inference by enabling retrofit GPU acceleration for standard PCs. The solution targets users looking to reduce cloud costs and latency for LLM workloads.
-
Startup Transforms Mac Mini Into Full-Powered AI Inference System With External GPU
A new approach enables Mac Mini systems to leverage external NVIDIA and AMD GPUs for dramatically enhanced local LLM inference performance.
-
AMD Launches Agent System Optimized for Local AI Inference With Ryzen and Radeon
AMD announces a new integrated system designed specifically for local AI workloads, combining Ryzen CPUs with Radeon GPU acceleration for efficient inference.
-
GPU-Accelerated DataFrame Library for Local Inference Workloads
A new DataFrame library that runs on GPUs, accelerators, and alternative hardware, enabling efficient data processing for local AI inference pipelines.
-
OpenClaw with vLLM Running for Free on AMD Developer Cloud
AMD launches free cloud access to run OpenClaw and vLLM inference workloads, providing developers with no-cost GPU resources for local LLM development.