MLPerf 3.1 includes major language model benchmarks for inference
September 11 2023

MLCommons has expanded the MLPerf AI benchmarks suite with the addition of testing for large language models (LLMs) for inference and a new benchmark for assessing performance of storage systems for machine learning workloads. The organization, which offers a neutral platform for vendors to report on various aspects of AI performance, recently released the second major update of the MLPerf Inference 3.1 benchmarks. Many submitters have reportedly improved their performance by 20% or more over the previous 3.0 benchmark. Participating submitters include major tech companies such as ASUSTeK, Azure, Dell, Google, Intel, Lenovo, Nvidia, and more.
Back to Breaking AI News
What does it mean?
- MLCommons: An open engineering consortium that produces machine learning benchmarks and metrics to improve the technology.
- MLPerf AI benchmarks suite: A collection of tools and standards developed by MLCommons to measure the performance of AI systems.
- Inference: The process of making predictions using a trained machine learning model.
Does reading the news feel like drinking from the firehose? Do you want more curation and in-depth content? Perhaps you'd like to subscribe to the Synthetic Work newsletter.
Synthetic Work an independent, non-technical, free, weekly newsletter written by a former Fortune 1000 tech executive and Gartner analyst to understand how AI is changing our jobs and the way we work.
If you want to understand how artificial intelligence is transforming companies and their workforce, your industry, our economy, and the entire society, this newsletter is for you. Many business leaders read Synthetic Work, including:
CEOs
CIOs
Chief Investment Officers
Chief People Officers
Chief Revenue Officers
CTOs
EVPs of Product
Managing Directors
VPs of Marketing
VPs of R&D
Board Members
and many other smart people.
They are turning the most transformative technology of our times into their biggest business opportunity ever.
What about you?
If you want to understand how artificial intelligence is transforming companies and their workforce, your industry, our economy, and the entire society, this newsletter is for you. Many business leaders read Synthetic Work, including:
CEOs
CIOs
Chief Investment Officers
Chief People Officers
Chief Revenue Officers
CTOs
EVPs of Product
Managing Directors
VPs of Marketing
VPs of R&D
Board Members
and many other smart people.
They are turning the most transformative technology of our times into their biggest business opportunity ever.
What about you?