📜 Post with us on Reddit
📝 Create a new article
🏭 Create a company page
🇬🇧 - in English
🇫🇷 - en Français (FR)
🇵🇹 - em Português (PT)
🇩🇪 - in Deutsch (DE)
🇯🇵 - 日本語で (JA)
Sponsored articles

You can edit almost every page by Creating an account. Otherwise, see the FAQ.

Groq Inc.

From EverybodyWiki Bios & Wiki

Groq Inc.
Private
ISIN🆔
Industry
Founded 📆2016; 8 years ago (2016)
Founders 👔
  • Jonathan Ross
Headquarters 🏙️,
Mountain View, California
,
US
Number of locations
Toronto, Canada
Area served 🗺️
Key people
Jonathan Ross (CEO), Andrew S. Rappaport (Board Member), Chamath Palihapitiya (Investor)
Products 📟 Language Processing Unit (LPU)
Members
Number of employees
250 (2023)
🌐 Websitegroq.com
📇 Address
📞 telephone

Groq is an American artificial intelligence (AI) company that builds an AI accelerator application-specific integrated circuit (ASIC) that they call the Language Processing Unit (LPU) and related hardware to accelerate the inference performance of AI workloads.

Examples of the types AI workloads the run on Groq's LPU are: large language models[1][2], image classification[3], anomaly detection[4][5], and predictive analysis.[6][7]

Groq is headquartered in Mountain View, CA, and has offices in San Jose, CA, Liberty Lake, WA, Toronto, Canada, London, U.K. and remote employees throughout North America and Europe.

History[edit]

Groq was founded in 2016 by a group of former Google engineers, led by Jonathan Ross, one of the designers of the Tensor Processing Unit (TPU), an AI accelerator ASIC, and Douglas Wightman, an entrepreneur and former engineer at Google X (known as X Development).[8]

Groq received seed funding from Social Capital’s Chamath Palihapitiya, with a $10 million investment in 2017[9] and soon after secured additional funding.

In April of 2021, Groq raised $300 million in a series C round led by Tiger Global Management and D1 Capital Partners.[10] Current investors include: The Spruce House Partnership, Addition, GCM Grosvenor, Xⁿ, Firebolt Ventures, General Global Capital, and Tru Arrow Partners, as well as follow-on investments from TDK Ventures, XTX Ventures, Boardman Bay Capital Management, and Infinitum Partners.[11][12] After Groq’s series C funding round, it was valued at over 1 billion dollars, making the startup a unicorn.[13]

On March 1st, 2022, Groq acquired Maxeler Technologies, a company known for its dataflow systems technologies.[14]

On August 16th, 2023, Groq selected Samsung Electronics foundry in Taylor, Texas to manufacture its next generation chips, on Samsung's 4-nanometer (nm) process node. This was the first order this new Samsung chip factory.[15]

On February 19th, 2024, Groq soft launched its GroqCloud to attract developers into using the Groq API.[16] On March 1st, 2024 Groq acquired Definitive Intelligence, a startup known for offering a range of business-oriented AI solutions.[17] Groq acquired Definitive Intelligence to help groq the newly formed business unit GroqCloud, Groq’s cloud platform that provides self-serve API access to popular large language models running on the company’s cloud-hosted accelerators.[16][17]

Technology[edit]

A die photo of Groq’s LPU V1

Groq's inital name for their ASIC was the Tensor Streaming Processor (TSP), but after the evident efficieny of the TSP with the rise of Generative AI, Groq rebranded the TSP as the Language Processing Unit (LPU).[1][18]

The LPU features a functionally sliced microarchitecture, where memory units are interleaved with vector and matrix computation units.[19][20] This design facilitates the exploitation of dataflow locality in AI compute graphs, improving execution performance and efficiency. The LPU was designed off of two key observations:

  1. AI workloads exhibit substantial data parallelism, which can be mapped onto purpose built hardware, leading to performance gains.[19][20]
  2. A deterministic processor design, coupled with a producer-consumer programming model, allows for precise control and reasoning over hardware components, allowing for optimized performance and energy efficiency.[19][20]

In addition to its functionally sliced microarchitecture, the LPU can also be characterized by its single core, deterministic architecture.[19][21] The LPU is able to achieve deterministic execution by avoiding the use of traditional reactive hardware components (branch predictors, arbiters, reordering buffers, caches)[19] and by having all execution explicitly controlled by the compiler thereby guaranteeing determinism in execution of an LPU program.[20]

The first generation of the LPU (LPU v1) yields a computational density of more than 1TeraOp/s per square mm of silicon for its 25×29 mm 14nm chip operating at a nominal clock frequency of 900 MHz.[19] The second generation of the LPU (LPU v2) will be manufacturing on Samsung's 4nnm process node.[15]

Performance[edit]

Groq emerged as the first API provider to break the 100 tokens per second generation rate while running Meta’s Llama2-70B parameter model.[22]

Groq currently hosts a variety of open-source large language models running on its LPUs for public access.[23] Access to these demos are available through Groq's website. The LPU's performance while running these open source LLMs has been independently benchmarked by ArtificialAnalysis.ai, in comparison with other LLM providers.[24] The LPU's measured performance is shown in the table below:

Language Processing Unit LLM Performance
Model Name Tokens/second (T/s) Latency (seconds)
Llama2-70B[25][26][27] 253 T/s 0.3s
Mixtral[28] 473 T/s 0.3s
Gemma[29] 826 T/s 0.3s

See also[edit]

References[edit]

  1. ↑ 1.0 1.1 Williams, Wayne. "'Feels like magic!': Groq's ultrafast LPU could well be the first LLM-native processor — and its latest demo may well convince Nvidia and AMD to get out their checkbooks". TechRadar Pro. TechRadar. Retrieved 19 April 2024.
  2. ↑ Ward-Foxton, Sally. "Groq Demonstrates Fast LLMs on 4-Year-Old Silicon". EETimes. Retrieved 19 April 2024.
  3. ↑ Ward-Foxton, Sally. "Groq's AI Chip Debuts in the Cloud". EETimes. EETimes. Retrieved 19 April 2024.
  4. ↑ Moorhead, Patrick. "US Army Analytics Group – Cybersecurity Anomaly Detection 1000X Faster With Less False Positives". Forbes. Forbes. Retrieved 19 April 2024.
  5. ↑ Herman, Arthur. "Cybersecurity Is Entering The High-Tech Era". Forbes. Forbes. Retrieved 19 April 2024.
  6. ↑ Heinonen, Nils. "Researchers accelerate fusion research with Argonne's Groq AI platform". Argonne Leadership Computing Facility. Retrieved 19 April 2024.
  7. ↑ Larwood, Mariah; Cerny, Beth. "Argonne deploys new Groq system to ALCF AI Testbed, providing AI accelerator access to researchers globally". Argonne Leadership Computing Facility. Retrieved 19 April 2024.
  8. ↑ Levy, Ari. "Several Google engineers have left one of its most secretive AI projects to form a stealth start-up". CNBC. CNBC. Retrieved 19 April 2024.
  9. ↑ Clark, Kate. "Secretive semiconductor startup Groq raises $52M from Social Capital". TechCrunch. Retrieved 19 April 2024.
  10. ↑ King, Ian. "Tiger Global, D1 Lead $300 Million Round in AI Chip Startup Groq". Bloomberg. Retrieved 19 April 2024.
  11. ↑ Wheatly, Mike. "AI chipmaker Groq raises $300M in Series C round". Silicon Angle. Retrieved 19 April 2024.
  12. ↑ McFarland, Alex. "AI Chip Startup Groq Closes $300 Million in Series C Fundraising". Unite.AI. Retrieved 19 April 2024.
  13. ↑ Andonov, Kaloyan; Lavine, Rob. "Analysis: Groq computes a $300m series C". Global Venturing. Retrieved 19 April 2024.
  14. ↑ Prickett Morgan, Timothy. "GROQ BUYS MAXELER FOR ITS HPC AND AI DATAFLOW EXPERTISE". The Next Platform. Retrieved 19 April 2024.
  15. ↑ 15.0 15.1 Hwang, Jeong-Soo. "Samsung's new US chip fab wins first foundry order from Groq". The Korea Economic Daily. Retrieved 19 April 2024.
  16. ↑ 16.0 16.1 Franzen, Carl. "Groq launches developer playground GroqCloud with newly acquired Definitive Intelligence". Venture Beat. Retrieved 19 April 2024.
  17. ↑ 17.0 17.1 Wiggers, Kyle. "AI chip startup Groq forms new business unit, acquires Definitive Intelligence". TechCrunch. Retrieved 19 April 2024.
  18. ↑ Mellor, Chris. "Grokking Groq's Groqness". Blocks & Files. Retrieved 19 April 2024.
  19. ↑ 19.0 19.1 19.2 19.3 19.4 19.5 Abts, Dennis; Kimmell, Garrin; Ling, Andrew; Kim, John; Boyd, Matt; Bitar, Andrew; Parmar, Sahil; Ahmed, Ibrahim; Dicecco, Roberto; Han, David; Thompson, John; Bye, Michael; Hwang, Jennifer; Fowers, Jeremy; Lillian, Peter; Murthy, Ashwin; Mehtabuddin, Elyas; Tekur, Chetan; Sohmers, Thomas; Kang, Kris; Maresh, Stephen; Ross, Jonathan (2022-06-11). "A software-defined tensor streaming multiprocessor for large-scale machine learning". Proceedings of the 49th Annual International Symposium on Computer Architecture. pp. 567–580. doi:10.1145/3470496.3527405. ISBN 978-1-4503-8610-4. Search this book on
  20. ↑ 20.0 20.1 20.2 20.3 Abts, Dennis; Kimmell, Garrin; Ling, Andrew; Kim, John; Boyd, Matt; Bitar, Andrew; Parmar, Sahil; Ahmed, Ibrahim; Dicecco, Roberto; Han, David; Thompson, John; Bye, Michael; Hwang, Jennifer; Fowers, Jeremy; Lillian, Peter; Murthy, Ashwin; Mehtabuddin, Elyas; Tekur, Chetan; Sohmers, Thomas; Kang, Kris; Maresh, Stephen; Ross, Jonathan (June 11, 2022). "[A software-defined tensor streaming multiprocessor for large-scale machine learning]". ACM Digital Library: 567–580. doi:10.1145/3470496.3527405. ISBN 978-1-4503-8610-4. Retrieved 2024-03-18.
  21. ↑ Singh, Satnam (February 11, 2022). "[The Virtuous Cycles of Determinism: Programming Groq's Tensor Streaming Processor]". ACM Digital Library: 153. doi:10.1145/3490422.3510453. ISBN 978-1-4503-9149-8. Retrieved 2024-03-18.
  22. ↑ Smith-Goodson, Paul. "Groq's Record-Breaking Language Processor Hits 100 Tokens Per Second On A Massive AI Model". Forbes. Retrieved 19 April 2024.
  23. ↑ Morrison, Ryan. "Meet Groq — the chip designed to run AI models really, really fast". Tom’s Guide. Retrieved 19 April 2024.
  24. ↑ "Groq Shows Promising Results in New LLM Benchmark, Surpassing Industry Averages". HPCwire. 2024-02-13. Retrieved 2024-03-18.
  25. ↑ "Llama-2 Chat 70B Providers". artificialanalysis.ai. Retrieved 2024-03-18.
  26. ↑ "Groq Shows Promising Results in New LLM Benchmark, Surpassing Industry Averages". Datanami. 2024-02-13. Retrieved 2024-03-18.
  27. ↑ "Groq Demos Fast LLMs on 4-Year-Old Silicon". EE Times. 2023-09-12. Retrieved 2024-03-18.
  28. ↑ "Mixtral 8x7B Instruct Providers". artificialanalysis.ai. Retrieved 2024-03-18.
  29. ↑ "Gemma-7B Models Providers". artificialanalysis.ai. Retrieved 2024-03-18.

External links[edit]


This article "Groq" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Groq. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.


Retrieved from "https://en.everybodywiki.com/index.php?title=Groq&oldid=4048267"
License CC BY-SA 3.0
Powered by MediaWiki
Powered by Semantic MediaWiki