All posts tagged: LPUs

AI chip race: Groq CEO takes on Nvidia, claims most startups will use speedy LPUs by end of 2024

AI chip race: Groq CEO takes on Nvidia, claims most startups will use speedy LPUs by end of 2024

Everyone is talking about Nvidia’s jaw-dropping earnings results — up a whopping 265% from a year ago. But don’t sleep on Groq, the Silicon Valley-based company creating new AI chips for large language model (LLM) inference (making decisions or predictions on existing models, as opposed to training). Last weekend, Groq suddenly enjoyed a viral moment most startups just dream of.  Sure, it wasn’t as big a social media splash as even one of Elon Musk’s posts about the totally unrelated large language model Grok. But I’m certain the folks at Nvidia took notice after Matt Shumer, CEO of HyperWrite, posted on X about Groq’s “wild tech” that is “serving Mixtral at nearly 500 tok/s” with answers that are “pretty much instantaneous.”  Shumer followed up on X with a public demo of a “lightning-fast answers engine” showing “factual, cited answers with hundreds of words in less than a second” —and suddenly it seemed like everyone in AI was talking about and trying out Groq’s chat app on its website, where users can choose from output served up …