Tagged "batch-processing"
9 articles tagged batch-processing, 17 February 2026 to 10 October 2026. Newest first.
-
Local LLM Inference at Scale with vLLM
An in-depth technical exploration of vLLM's architecture for achieving high-throughput local inference, covering optimization techniques that enable efficient batch processing and token generation at scale.
-
Phison and Intel Roll Out aiDAPTIV to Boost Local AI on Intel AI PC Platforms
Phison and Intel have launched aiDAPTIV, a collaborative optimization framework designed to accelerate local AI inference on Intel AI PC platforms. The initiative bridges storage and compute to improve overall system efficiency for on-device model deployment.
-
What Type of AI Usage? Deployment Patterns and Implementation Considerations
A framework for categorizing different AI implementation patterns, helping developers choose appropriate architectures for local versus cloud deployment.
-
Automating Read-It-Later Workflows with Local LLMs for Overnight Summarization
A practical guide demonstrating how to build an automated article summarization pipeline using self-hosted LLMs, eliminating the need for cloud-based services while maintaining privacy and reducing costs.
-
ik_llama.cpp Fork Delivers 26x Faster Prompt Processing on Qwen 3.5 27B
A fork of llama.cpp called ik_llama.cpp is delivering dramatic 26x speed improvements for prompt processing on Qwen 3.5 27B models. Real-world benchmarks on Blackwell RTX PRO GPUs show tangible performance gains for production agentic workloads.
-
Achieving 2000 Tokens Per Second with QWEN 3.5 27B on RTX-5090
A practitioner shares real-world performance benchmarks achieving 2000 TPS with QWEN 3.5 27B optimized for document classification workloads on consumer-grade RTX-5090 hardware.
-
Nemotron 9B Powers Large-Scale Local Inference: Patent Classification and Real-Time Applications
Practitioners are leveraging Nemotron 9B for production workloads, from classifying 3.5M patents on a single RTX 5090 to powering real-time Minecraft agent control, demonstrating the model's efficiency and practical viability.
-
Breaking the Speed Limit: Strategies for 17k Tokens/Sec Local Inference
Practical strategies and techniques for achieving ultra-high token throughput in local LLM inference, reaching 17,000 tokens per second. Essential performance optimization guide for practitioners running models on-device.
-
Show HN: PgCortex – AI enrichment per Postgres row, zero transaction blocking
Novel tool integrating local AI inference directly into PostgreSQL for per-row data enrichment without blocking transactions, enabling efficient batch processing of LLM operations.