Tagged "custom-asic"
43 articles tagged custom-asic, 17 February 2026 to 20 October 2026. Newest first.
-
vLLM Adds PyTorch-Native TPU Backend for Distributed Inference
vLLM introduces native PyTorch backend support for Google TPUs, enabling efficient distributed LLM inference on TPU clusters. This expansion broadens vLLM's hardware compatibility for local and on-premises deployments.
-
Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin
vLLM now supports Tenstorrent hardware through a dedicated plugin, enabling efficient LLM inference on alternative accelerators beyond NVIDIA and AMD. This expands deployment options for self-hosted inference with optimized performance on specialized silicon.
-
Google Cloud finds Gemma 3 12B outscales 27B on TPU
Google's Gemma 3 12B model delivers superior performance to the 27B variant when running on TPU infrastructure. This finding highlights the importance of hardware-model co-optimization for efficient local and edge inference.
-
Amazon Developing Custom On-Device AI Chips for Echo and Fire TV Lineups
Amazon is engineering proprietary AI accelerators specifically designed for on-device inference in Echo speakers and Fire TV devices, signaling major hardware investments in local AI deployment.
-
NVIDIA and Microsoft Team Up to Bring Secure On-Device AI Agents to Windows PCs
NVIDIA and Microsoft have announced RTX Spark, a new AI superchip designed to power autonomous AI agents directly on consumer Windows PCs with improved security and privacy. The collaboration marks a significant step toward making local LLM inference mainstream on desktop hardware.
-
NVIDIA Launches N1X/N1 CPU-GPU SoC for PC Market, Targeting Heavy On-Device AI Users
NVIDIA introduces its first PC-targeted System-on-Chip (N1X/N1) designed for on-device AI workloads. The chip combines CPU and GPU capabilities for local LLM inference, though adoption depends on Windows ecosystem maturity.
-
Zoho-Backed Netrasemi Launches 12nm AI Chip, Mass Production Begins This Year
India's Netrasemi, backed by Zoho, is launching a 12nm AI processor with mass production starting in 2026, offering a homegrown option for local LLM inference with implications for edge deployment and hardware accessibility.
-
Why AI Hardware Is a Chip Layer Problem
On-device AI deployment requires fundamental hardware redesigns at the chip level, with implications for how local LLM inference will be optimized across consumer devices.
-
Hardware LLM Taalas Reaches >14,000 TPS on Llama 3.1 8B
Taalas demonstrates breakthrough throughput of over 14,000 tokens per second on Llama 3.1 8B, showcasing specialized hardware acceleration for local and edge LLM deployment.
-
Anker's Thus Chip Puts AI On-Device, Promising Faster Responses And Better Privacy
Anker introduces the Thus chip, a dedicated hardware accelerator designed to run AI models entirely on-device with improvements in response latency and privacy preservation.
-
Anker's New 'Thus' Chip Brings 150x AI Power to Earbuds
Anker has announced a specialized AI chip for earbuds that dramatically increases on-device processing capability, enabling local inference on ultra-constrained hardware.
-
Building Real-World On-Device AI with LiteRT and NPU
Google details LiteRT framework for deploying optimized LLMs on edge devices using Neural Processing Units, enabling efficient on-device inference without cloud dependency.
-
AI Agent Designs a RISC-V CPU Core from Scratch
An AI agent has successfully designed a complete RISC-V CPU core autonomously, demonstrating advanced reasoning capabilities and opening new possibilities for hardware optimization tailored to local LLM inference.
-
Anker Unveils 'Thus' Chip to Bring On-Device AI Across Product Line
Anker has announced a custom AI processor chip called 'Thus' designed to enable on-device LLM inference in consumer electronics, launching first in Soundcore earphones with plans for broader product integration.
-
DeepX and Hyundai Motor Group Robotics LAB Partner to Develop Next-Generation Physical AI Compute Platform
DeepX and Hyundai's Robotics LAB are collaborating on an on-device AI compute platform optimized for robotic systems, demonstrating how local inference is enabling physical AI applications at scale.
-
Community Computer: Collaborative Autoresearch on a Peer-to-Peer Network
A decentralized platform enabling distributed AI research and computation through peer-to-peer networks, allowing researchers to contribute local compute resources for collaborative model training and experimentation.
-
Apple Brings Enhanced On-Device AI Features to iPhone
Apple continues expanding on-device AI capabilities in iOS, integrating machine learning features directly on iPhones. The company's focus on local processing improves privacy and reduces latency for consumer AI features.
-
Lotte Innovate and DeepX Collaborate on Mass Production of Domestic AI Semiconductors
A strategic partnership between Lotte Innovate and DeepX aims to mass-produce AI semiconductors optimized for edge inference, positioning NPUs as alternatives to GPUs for local LLM deployment and reducing dependency on traditional GPU infrastructure.
-
Chinese Chipmakers Claim Nearly Half of Local Market as Nvidia's Lead Shrinks
Chinese semiconductor manufacturers are rapidly gaining market share in their domestic AI chip market, now commanding nearly 50% of the segment as Nvidia's dominance faces competitive pressure. This shift has significant implications for local LLM inference costs and accessibility in Asia.
-
Select the Right Hardware for Your Local LLM Deployment with This Online Guide
An authoritative guide for choosing appropriate hardware for local LLM inference, helping practitioners match their deployment needs to cost-effective hardware solutions.
-
CERN Embeds Tiny AI Models in Silicon Chips for Real-Time LHC Data Filtering
CERN is deploying custom AI models burned directly into silicon to filter the Large Hadron Collider's 40,000 exabytes of annual data in real-time, demonstrating the inverse trend to the industry's pursuit of ever-larger models. This represents a compelling use case for edge inference at scientific scale.
-
Hold on to Your Hardware: Implications for Local LLM Deployment
An article examining hardware longevity and sustainability raises important considerations for practitioners investing in local inference infrastructure.
-
Nota AI and SiMa.ai Partner on Physical AI Technology for Local Deployment
Strategic partnership between Nota AI and SiMa.ai aims to advance physical AI and on-device inference, combining model compression with hardware optimization.
-
Korea to Deploy Domestic AI Chips in Smart Cities as NPU Trials Scale Up
South Korea is scaling trials of domestically-developed AI chips optimized for neural processing in smart city infrastructure, marking a significant shift toward regional edge computing independence.
-
ASUS ExpertCenter PN55 Mini PC Combines AMD AI CPU and 55 TOPS NPU
ASUS launches a ruggedized industrial mini PC featuring AMD's latest AI-optimized CPU and a dedicated 55 TOPS NPU, purpose-built for on-device inference deployments in demanding environments.
-
Dell Pro Max 16 Plus Launches With Enterprise-Grade Discrete NPU for On-Device AI
Dell's new Pro Max 16 Plus laptop features a dedicated Neural Processing Unit (NPU) designed for efficient on-device AI inference. The hardware advancement enables faster, more power-efficient local LLM deployment on enterprise devices.
-
Snapdragon 8 Elite Gen 5 Hands the Galaxy S26 the AI Upgrade We've Been Waiting For
Qualcomm's Snapdragon 8 Elite Gen 5 delivers significant improvements to on-device AI performance through enhanced neural processing units, enabling more sophisticated local LLM inference on flagship smartphones. This hardware evolution supports increasingly capable models running natively on mobile devices.
-
KAIST Develops World's First Hyper-Personalized On-Device AI Chip
Researchers at KAIST have created a specialized AI chip optimized for personalized inference on mobile and edge devices, enabling efficient model adaptation without cloud synchronization.
-
Show HN: VmExit – An Experiment in AI-Native Computing
VmExit explores fundamental reimagining of computing infrastructure optimized specifically for AI workloads, challenging conventional approaches to local model deployment.
-
Building PyTorch-Native Support for IBM Spyre Accelerator
IBM Research announces new PyTorch-native support for the IBM Spyre accelerator, enabling better integration of custom hardware with popular deep learning frameworks. This development simplifies local LLM deployment on specialized accelerators.
-
Building PyTorch-Native Support for IBM Spyre Accelerator
IBM Research has developed native PyTorch support for the IBM Spyre Accelerator, enabling optimised local inference on specialised hardware.
-
HyperExcel Seeks 150 Billion Won Series B to Scale LPU and Verda in Korea
Korean startup HyperExcel is raising Series B funding to scale production of LPU (Language Processing Unit) accelerators and Verda inference optimisation technology for local deployment.
-
The Emerging Role of SRAM-Centric Chips in AI Inference
Hardware architectures optimized around SRAM are reshaping AI inference capabilities for edge and local deployments. This emerging trend addresses critical bottlenecks in memory bandwidth and latency for on-device LLM execution.
-
AMD Expands Ryzen AI 400 Series Portfolio for Consumer and Enterprise AI PC Options
AMD announced an expanded lineup of Ryzen AI 400 Series processors, bringing more hardware options for local AI inference across consumer laptops and business workstations. The expansion increases accessibility of dedicated NPU hardware for on-device LLM deployment.
-
The ML.energy Leaderboard
ML.energy launches a comprehensive leaderboard benchmarking model efficiency metrics including inference latency, memory consumption, and energy usage across diverse hardware platforms, providing crucial data for local deployment decisions.
-
South Korea to Launch $687 Million Project to Develop On-Device AI Semiconductors
South Korea announces a major government investment in developing specialized semiconductors for on-device AI inference. This signals growing infrastructure support for local LLM deployment at the hardware level.
-
Nvidia Could Launch Its First Laptops With Its Own Processors
Nvidia is reportedly developing its own laptop processors, which could significantly impact the hardware landscape for local LLM deployment. Custom silicon optimised for AI inference could offer better performance and efficiency than traditional CPUs.
-
AI Is Stress Testing Processor Architectures and RISC-V Fits the Moment
RISC-V architecture emerges as a compelling alternative for AI workloads as traditional processor designs face thermal and efficiency challenges under LLM inference loads, opening new possibilities for local deployment on custom silicon.
-
Google Open-Sources NPU IP, Synaptics Implements It for Hardware Acceleration
Google has open-sourced its Neural Processing Unit IP architecture, with Synaptics already implementing it, potentially enabling more efficient hardware accelerators for local LLM inference across edge devices.
-
Taalas Etches AI Models onto Transistors to Rocket Boost Inference
Taalas introduces a novel approach to hardware-level AI optimization by etching neural network models directly onto transistors, achieving dramatic inference speed improvements for local deployment. This breakthrough hardware innovation enables faster, more efficient on-device LLM execution.
-
Google Is Exploring Ways to Use Its Financial Might to Take on Nvidia
Google explores strategic investments and partnerships to compete with Nvidia's dominance in AI accelerator chips, potentially enabling more accessible hardware options for local LLM deployment. This shift could significantly impact the economics of on-device inference infrastructure.
-
Free ASIC-Accelerated Llama 3.1 8B Inference at 16,000 Tokens/Second
Taalas, a fast inference hardware startup, has released a free chatbot interface and API endpoint running Llama 3.1 8B on custom ASICs, achieving 16,000 tokens/second throughput. This demonstrates the viability of specialized hardware for cost-effective local-style inference.
-
Chinese AI Chipmaker Axera Semiconductor Plans $379 Million Hong Kong IPO for Edge Inference Hardware
Axera Semiconductor, a Chinese AI chipmaker focused on edge inference, is raising $379 million through a Hong Kong IPO. The funding round signals strong investor confidence in the edge AI hardware market and accelerates development of specialized silicon for local LLM deployment.