Network World

Tether is shipping TurboQuant KV-cache quantization with Vulkan support into its QVAC SDK

Tether successfully integrated Google’s TurboQuant into the inference engine of its local AI framework, QVAC. It is the ...
VentureBeat

How attention offloading reduces the costs of LLM inference at scale

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Rearranging the computations and hardware used to serve large language ...
Business Wire

Penguin Solutions Introduces Industry's First Production-Ready CXL-Based KV Cache Server

FREMONT, Calif.--(BUSINESS WIRE)--Penguin Solutions, Inc. (Nasdaq: PENG), the AI factory platform company, today announced the industry's first production-ready KV cache server that utilizes CXL ...

Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit

LCLMs compress LLM context before decode — 8.8x faster at 16x compression, beating every KV cache method tested. Open-sourced by NYU and Columbia.