Tag
AI inference
AI inference is the runtime phase where trained models generate outputs in production, so latency, memory footprint, and compute cost matter most. Topics here include home-based inference nodes, KV-cache compression, and how long contexts keep DRAM demand high.
8 articles
AMD+Helios lets Cerebras widen inference throughput
I break down how Cerebras and AMD are pairing Helios with WSE, and give you a copy-ready template for workload-optimized inference.
Microsoft adds AMD chips to Azure AI and HPC
Microsoft is adding AMD Helios and next-gen EPYC chips to Azure, powering new VMs for AI inference, data prep, and chip design.
GPT-5.6 Sol, Terra, Luna arrive on DigitalOcean
OpenAI’s GPT-5.6 Sol, Terra, and Luna are now available on DigitalOcean Serverless Inference, with new reasoning modes and per-token pricing.
Anthropic’s chip move is a necessary break from GPU dependence
Anthropic should pursue a custom AI chip because control over compute now matters more than vendor convenience.
Why Zyphra Cloud on AMD Matters More Than Another Model Launch
Zyphra Cloud matters because inference, not training, is now the real AI platform battle.
Span, Nvidia, Pulte: Mini AI Data Centers in Homes
Span is testing home-based AI inference nodes with 1.25 MW across 100 homes, cutting build time from years to months.
TurboQuant cuts memory use 6x without accuracy loss
Google Research’s TurboQuant claims 6x less memory and 8x faster inference with no accuracy loss, jolting AI inference economics.
TurboQuant Won’t Fix the Memory Crunch
Google’s TurboQuant can cut KV-cache memory use 6x, but longer contexts may keep DRAM and NAND demand climbing.