Sean Michael Kerner (@techjournalist.bsky.social)

FriendliAI's founders wrote the research inside vLLM. Now they want neocloud GPUs earning more money between workloads "The cloud vendors, neoclouds, have GPUs but they can't really use GPUs all the time at full capacity," Byung-Gon Chun told VentureBeat. venturebeat.com/infrastructu...

loading . . .

The team behind continuous batching says your idle GPUs should be running inference, not sitting dark Meta description (SEO/AEO) FriendliAI — founded by the researcher behind continuous batching, the technique at the core of vLLM — is launching InferenceSense, a platform that fills idle neocloud GPU capacity with paid AI inference workloads and splits the token revenue with operators. The company claims 2–3x the token throughput of a standard vLLM deployment. https://venturebeat.com/infrastructure/the-team-behind-continuous-batching-says-your-idle-gpus-should-be-running