On Monday, Cerebras Techniques unveiled its 13.5 million core Andromeda AI supercomputer for deep studying, reviews Reuters. In accordance Cerebras, Andromeda delivers over one 1 exaflop (1 quintillion operations per second) of AI computational energy at 16-bit half precision.
The Andromeda is itself a cluster of 16 Cerebras C-2 computer systems linked collectively. Every CS-2 incorporates one Wafer Scale Engine chip (typically known as “WSE-2”), which is at the moment the most important silicon chip ever made, at about 8.5-inches sq. and full of 2.6 trillion transistors organized into 850,000 cores.
Cerebras constructed Andromeda at an information heart in Santa Clara, California, for $35 million. It is tuned for functions like massive language fashions and has already been in use for tutorial and business work. “Andromeda delivers near-perfect scaling by way of easy knowledge parallelism throughout GPT-class massive language fashions, together with GPT-3, GPT-J and GPT-NeoX,” writes Cerebras in a press launch.
Cerebras
The phrase “Close to-perfect scaling” signifies that as Cerebras provides extra CS-2 laptop models to Andromeda, coaching time on neural networks is lowered in “close to excellent proportion,” in accordance with Cerebras. Usually, to scale up a deep-learning mannequin by including extra compute energy utilizing GPU-based methods, one may see diminishing returns as {hardware} prices rise. Additional, Cerebras claims that its supercomputer can carry out duties that GPU-based methods can not:
GPU unimaginable work was demonstrated by one among Andromeda’s first customers, who achieved close to excellent scaling on GPT-J at 2.5 billion and 25 billion parameters with lengthy sequence lengths—MSL of 10,240. The customers tried to do the identical work on Polaris, a 2,000 Nvidia A100 cluster, and the GPUs have been unable to do the work due to GPU reminiscence and reminiscence bandwidth limitations.”
Whether or not these claims maintain as much as exterior scrutiny is but to be seen, however in an period the place firms typically practice deep-learning fashions on more and more massive clusters of Nvidia GPUs, Cerebras seems to offer another strategy.
How does Andromeda stack up in opposition to different supercomputers? At present, the world’s quickest, Frontier, resides at Oak Ridge Nationwide Labs and may carry out at 1.103 exaflops at 64-bit double precision. That laptop price $600 million to construct.
Entry to Andromeda is obtainable now to be used by a number of customers remotely. It is already being utilized by business writing assistant JasperAI and Argonne Nationwide Laboratory, and the College of Cambridge for analysis.