NVIDIA’s AI computing platform has smashed through performance records once more during the latest round of inference tests hosted via the industry benchmarking consortium called MLPerf, according to a press release shared under embargo with Interesting Engineering.
For the first time, there are more GPUs than CPUs in active inference performance on cloud service platforms.
Taken as a whole, this substantially widens NVIDIA’s lead as the industry’s only independent leader in AI performance, services, and software — as AI applications are moving into more mainstream applications than ever before.
NVIDIA’s A100 and T4 GPUs mopped the floor with all data center inference tests. Source: NVIDIA
NVIDIA smashes AI computing benchmark, GPUs surpass CPUs
NVIDIA GPUs won every test of AI inference in data center and edge computer systems in the most recent round of the only consortium-based peer-reviewed benchmarks in the industry. Edge computing involves AI attached to the point-of-action, where, for example, an AI security drone would need to make real-time decisions without the delay of connection or remote control.
NVIDIA’s A100 Tensor Core GPUs surpassed previous results demonstrated in the first AI inference tests held last year — courtesy of MLPERF — a benchmarking consortium for the industry established in May 2018.
Another GPU called A100 and introduced in May of this year showed up CPU rivals — at a multiple of up to 237 times better performance in data center inference — according to the 0.7 benchmarks of MLPerf Inference.
New levels of performance will help improve cost efficiency for companies to take AI from research to production stages. Source: NVIDIA
The NVIDIA T4 small-form-factor and energy-efficient GPUs eclipsed CPU performance at a multiple of 28 times better. For reference, one NVIDIA DGX A100 system with eight A100 GPUs can offer the same performance as nearly 1,000 dual-socket CPU servers on significant AI applications.
Notably, this latest round of benchmark testing via MLPerf saw more participation — featuring submissions from 23 organizations, which is up from the 12 who submitted in the last round. Crucially, NVIDIA AI partners running the company’s AI platform powered more than 85% of all submissions.
Jetson AGX Xavier joined the A100 and T4 GPUs in leadership performance at the edge. Source: NVIDIA
A100 GPUs, Jetson AGX Xavier, Edge Performance
Additionally, the NVIDIA Jetson AGX Xavier also widened the lead in SoC-based edge devices under constrained power — with support for all practical use cases.
The latest results represent a “tipping point” of distribution in the AI ecosystem — which saw 1,029 results submitted running NVIDIA products. Submissions included partners like Cisco, Fujitsu, Cisco, Altos, Alto, Dividiti, Lenovo, QCT, Gigabyte, Nettrix, and Inspur.
MLPerf enjoys broad support throughout academia and several industries, setting benchmarks for organizations like Facebook, Google, Microsoft, Arm, Baidu, Harvard, Lenovo, Intel, Stanford, and the University of Toronto.
Latest benchmarks include medical imaging, speech recognition
The latest benchmarks offered via MLPerf include four new tests — proof of AI’s expanding landscape. The new suite of tests now scores performance in medical imaging, speech recognition, natural language processing, recommendation systems, and also AI use cases in computer vision.
“The recent AI breakthroughs in natural language understanding are making a growing number of AI services like Bing more natural to interact with, delivering accurate and useful results, answers and recommendations in less than a second,” said Microsoft’s Vice President of search and artificial intelligence Rangan Majumder.
“Industry-standard MLPerf benchmarks provide relevant performance data on widely used AI networks and help make informed AI platform buying decisions,” added Majumder.
AI applications save lives amid the COVID-19 crisis
A substantial impact was seen where AI played a part in medical imaging. For example, a startup called Caption Health uses AI to simplify the task of retrieving echocardiograms — a procedure proven to save lives in U.S. hospitals during the initial days of the COVID-19 crisis.
This is why AI view models like 3D U-Net — used during the recent MLPerf benchmarks — are considered crucial.
“We’ve worked closely with NVIDIA to bring innovations like 3 U-Net to the healthcare market,” said Klaus Maier-Hein, the German Cancer Research Center’s head of medical image computing at DKFZ.
“Computer vision and imaging are at the core of AI research, driving scientific discovery and representing core components of medical care. And industry-standard MLPerf benchmarks provide relevant performance data that helps IT organizations and developers accelerate their specific projects and applications,” added Maier-Hein.
AI use cases in commercial settings have already made big waves. For example, Alibaba used recommendation systems in November 2019 for a $38 billion transaction in online sales during its Singles Day — the company’s biggest shopping day of the year.
GPUs active in major cloud services now make up more inference performance than CPUs, for the first time. Source: NVIDIA
NVIDIA AI Inference adoption has passed ‘tipping point’
Cutting through the technical noise of endless benchmarks, NVIDIA’s AI Inference surpasses a significant milestone this year — with its GPUs delivering a total of more than 100 exaflops of AI inference performance throughout the public cloud in the last year.
In other words, the total cloud AI inference computing capacity running on NVIDIA GPUs overtook cloud CPUs for the first time — following a trend of growth at a factor of roughly 10 every two years.
As AI continues to grow into organizations and industries interlaced with public infrastructure, companies leading the way like NVIDIA are sure to become an all-pervasive facet of daily life. GPU-accelerated software running NVIDIA-certified OEM systems throughout cloud systems are quickly becoming less of a future inevitability than a present-day standard.
RECOMMENDED ARTICLES
0COMMENT
NEWSLETTER
The Blueprint Daily
Stay up-to-date on engineering, tech, space, and science news with The Blueprint.
Brad Bergan Brad Bergan is a senior editor at Interesting Engineering, specializing in space and tech. Previously, he was a contributing editor at Futurism, and his words have appeared in or on VICE, Thinknum, 3:AM Magazine, The World Economic Forum, the National Book Critics Circle, and many other places. He holds a bachelor's in Philosophy and English from the University of Iowa, and studied graduate-level creative writing at The New School. He lives in New York.
0