SambaNova Systems, developers of the only full-stack AI platform, introduces the SN40L, a groundbreaking chip. With its groundbreaking new design, the SN40L will power SambaNova Suite, a full stack large language model (LLM) platform, with dense and sparse computing and huge and fast memory. It is truly an “intelligent chip”.
The TSMC-made SambaNova SN40L can serve a 5 trillion parameter model with 256k+ sequence length on a single system node. This is only achievable with an integrated stack, which is much better than previous state-of-the-art processors and enables higher-quality models, faster inference and training, and cheaper total cost of ownership.
“Today, SambaNova offers the only purpose-built full stack LLM platform — the SambaNova Suite — now with an intelligent AI chip; it’s a game changer for the Global 2000,” said co-founder and CEO Rodrigo Liang. “We can now address more memory with the smartest compute core on one chip, enabling organizations to capitalize on pervasive AI with their own LLMs to rival GPT4 and beyond.”
The new chip is part of SambaNova’s full-stack LLM platform, which answers enterprises’ greatest generative AI challenges: “We’ve started to see a trend towards smaller models, but bigger is still better and bigger models will start to become more modular,” said SambaNova Systems co-founder Kunle Olukotun. Customers desire an LLM with the strength of a trillion-parameter model like GPT-4 and the benefits of a data-tuned model. Our most powerful AI chip, the SN40L, coupled into a full stack LLM platform, lets users run the largest LLMs with improved training and inference performance without losing model correctness.
The SN40L powers the SambaNova Suite:
On one system node, SambaNova’s SN40L can service a 5 trillion parameter model with 256k+ sequence length. This allows higher-quality models, faster inference and training, and cheaper TCO.
Companies can quickly search, analyze, and generate data in multimodal LLMs with larger memory.
Due to LLM inference efficiency, AI models have lower TCOs.
The LLM platform is fully modular and extensible because SambaNova’s team considered customers’ AI implementation challenges, such as training and inference costs, sequence length limitations, and LLM speed (or latency), after six years of engineering and customer feedback. Customers can incrementally add modalities, knowledge, and model parameters up to 5T without affecting inference performance.
The SN40L chip from SambaNova is unique. It accesses HBM (High Bandwidth Memory) and DRAM from a single chip, allowing AI algorithms to choose the best memory for the task and access much more than otherwise. The chips also use SambaNova’s RDU (Reconfigurable Data Unit) architecture to run sparse models more efficiently using smarter computation, said IDC Research Vice-President, Performance-Intensive Computing Peter Rutten.
New SambaNova Suite models and features:
Llama2 variations (7B, 70B): state-of-the-art open-source language models that let users adapt, expand, and run the best LLM models while retaining ownership.
BLOOM 176B: the most accurate open-source, multilingual foundation model, allowing users to solve more problems with various languages and adapt the model to include additional, low-resource languages.
A novel vector-based retrieval embedding methodology allows clients to embed their documents into vector embeddings that may be retrieved during Q&A without hallucinations.
LLMs evaluate, extract, and summarize results.
The world’s best automatic speech recognition model for voice data transcription and analysis.
Additional multimodal and lengthy sequence duration capabilities.
Inference-optimized 3-tier Dataflow memory systems for unmatched bandwidth and capacity.
“Recent breakthroughs show that AI can rapidly improve our lives and businesses. GV General Partner and SambaNova Series A Board Director Dave Munichiello stated the Fortune 1000 wants predictability, dependability, and enterprise-grade availability, not just a new capability. “SambaNova’s announcement today underscores the company’s commitment to providing reliable access to best-in-class open-source models on top of world-class systems, putting customers in control of their models, data, and compute resources.”
About SambaNova Suite
SambaNova Suite is the first enterprise and government-optimized full-stack, generative AI platform from chip to model. The clever SN40L chip powers the SambaNova Suite, a fully integrated platform supplied on-premises or in the cloud with state-of-the-art open-source models that can be quickly and securely fine-tuned using client data for improved accuracy. Customers own generative AI models forever after adapting them with customer data, making it one of their most valuable assets.
SambaNova Systems
Customers use SambaNova to deploy cutting-edge generative AI in the enterprise quickly. Our purpose-built enterprise-scale AI platform powers next-generation AI computing.
Industry leaders, including Sun/Oracle and Stanford University hardware and software design professionals, launched SambaNova Systems in Palo Alto, California, in 2017. SoftBank Vision Fund 2, BlackRock, Intel Capital, GV, Walden International, Temasek, GIC, Redline Capital, Atlantic Bridge Ventures, Celesta, and others are investors.
