The best AI conversations don't happen in isolation. They happen when researchers + practitioners + industry leaders are in the same room, working on shared challenges. MLCommons Endpoints Dec 1, San Diego (one day!) Registration ↓ www.eventbrite.com/e/mlcommons-... #Endpoints2025

This is something we’ve put a lot of love into. Good, bad, or indifferent, AI is something we have to deal with. Join us for a bit more of a journey into the fundamentals of AI and some of the research we’ve done with our members and partners. Cc @odihq.bsky.social

Don’t miss #MLCommons Endpoints in San Diego, Dec 1–2! Learn, connect, and shape the future of AI with top experts at Qualcomm Hall. 🗓 Dec 1–2 | 🎟 Free tickets available now! www.eventbrite.com/e/mlcommons-... #AI #MachineLearning #SanDiego

Thanks to Babel Tech Reviews for this deep dive of CORSAIR AI Workstation 300 and its handling of real-world AI workloads. Showing why tools like MLPerf Client matter: they give a clear view of how systems actually perform on common LLM & inference tasks. babeltechreviews.com/the-corsair-...

MLCommons benchmarks are now cited in #ISO /IEC AI testing standards - and we're helping shape what comes next. Contributing to standards on benchmark quality (42119-8) and AI management systems (42003). Standards are made by those in the room. mlcommons.org/2025/11/iso-... #AIStandards #MLCommons

1/ MLPerf Client v1.5 is here! New AI PC benchmarking features: -Windows ML support for improved GPU/NPU performance -Expanded platforms: Windows x64, Windows on Arm, macOS, Linux, plus iPad app -Experimental power/energy measurement GUI improvements out of beta Download: github.com/mlcommons/ml...

MLPerf Training v5.1 results are live! Record participation: 20 organizations submitted 65 unique systems featuring 12 different accelerators. Multi-node submissions increased 86% over last year, showing the industry's focus on scale. Results: mlcommons.org/2025/11/trai... #MLPerf 1/3

Don’t miss #MLCommons Endpoints in San Diego, Dec 1–2! Learn, connect, and shape the future of AI with top experts at Qualcomm Hall. 🗓 Dec 1–2 | 🎟 Free tickets available now! www.eventbrite.com/e/mlcommons-... #AI #MachineLearning #SanDiego

IEEE Spectrum's analysis shows how our benchmarks capture the real industry challenge - LLMs scale exponentially while hardware improves incrementally. This pattern highlights why evolving benchmarks are important. Stay tuned for MLPerf Training v5.1, out on 11/12. spectrum.ieee.org/mlperf-trends

🚨 NEW: We tested 39 AI models for security vulnerabilities. Not a single one was as secure as it was "safe." Today, we're releasing the industry's first standardized jailbreak benchmark. Here's what we found 🧵1/6 mlcommons.org/2025/10/ailu...

We've just released MLPerf Mobile version 5.0.2 on the Google Play Store and GitHub. This version adds support for devices based on the Samsung Exynos 2500 SoC. play.google.com/store/apps/d...

How can LLMs reliably find, understand, & analyze datasets? By combining: 📂 #Croissant – AI-ready #metadata ⚡ #MCP – agentic access to data & tools Our new blog introduces Eclair: tools that let #LLMs discover, download & explore millions of #datasets. mlcommons.org/2025/10/croi...

TinyML benchmarks finally address real-world deployment with MLCommons' new streaming benchmark in MLPerf Tiny v1.3. Tests 20-minute continuous wake word detection while measuring power and duty cycle. Technical deep dive: mlcommons.org/2025/09/mlpe... #MLPerf #TinyML #EdgeAI

New MLPerf Tiny v1.3 results! New streaming wake-word detection test + 70 results measuring sub-100KB neural networks across hardware platforms. Thanks to Qualcomm , ST Microelectronics , Syntiantcorp & Kai Jiang for pushing tiny ML forward. View results: mlcommons.org/2025/09/mlpe...

We have released MLPerf Mobile v5.0.1 with support for MediaTek Dimensity 9400 SoCs and a few other improvements. The app is available for Android phones via the Google Play Store and the MLCommons GitHub repo. Let us know what you think! t.co/715ENpV6W4

@mlcommons.org dropped MLPerf inference 5.1 with three key benchmarks for enterprise AI: a reasoning model (DeepSeek-R1), speech recognition, and a smaller LLM for summarization tasks. Catch up on this essential reading for AI decisions: www.techarena.ai/content/ai-b... #MLCommons #MLPerf

MLPerf Inference v5.1 results are live! Record 27 organizations submitted 1,472 performance results across new and established AI workloads. Three new benchmarks debut: Reasoning with Deepseek R1 Speech to text with Whisper Small LLM with Llama 3.1 8B Read More: mlcommons.org/2025/09/mlpe...

1/6 MLCommons & AVCC announce MLPerf Automotive v0.5 benchmark results—a major step for transparent, reproducible automotive AI performance data. mlcommons.org/2025/08/mlpe...

1/ MLCommons just released results for the MLPerf Storage v2.0 benchmark—an industry-standard suite for measuring storage system performance in #ML workloads. This benchmark remains architecture-neutral, representative, and reproducible. mlcommons.org/2025/08/mlpe...

MLPerf Client v1.0 is out! 🎉 The new benchmark for LLMs on PCs and client systems is now available—featuring expanded model support, new workload scenarios, and broad hardware integration. Thank you to all submitters! #AMD, #Intel, @microsoft.com, #NVIDIA, #Qualcomm mlcommons.org/2025/07/mlpe...

MLCommons just launched MLPerf Mobile on the Google Play Store! 📱 Benchmark your Android device’s AI performance on real-world ML tasks with this free, open-source app. Try it now: play.google.com/store/apps/d...

Today, MLCommons is announcing a new collaboration with contributors from across academia, civil society, and industry to co-develop an open agent reliability evaluation standard to operationalize trust in agentic deployments. 🔗https://mlcommons.org/2025/06/ares-announce/ 1/3

We're all about acceleration! 😉 Watch @priya-kasimbeg.bsky.social & @fsschneider.bsky.social speedrun an explanation of the AlgoPerf benchmark, rules, and results all within a tight 5 minutes for our #ICLR2025 paper video on "Accelerating Neural Network Training". See you in Singapore!

Companies are deploying AI tools that haven't been pressure-tested, and it's already backfiring. In her new op-ed, our President, Rebecca Weiss, breaks down how industry-led AI reliability standards can help executives avoid costly, high-profile failures. 📖 More: bit.ly/3FP0kjg @fastcompany.com

Call for Submissions! #MLCommons & @AVCConsortium are accepting submissions for the #MLPerf Automotive Benchmark Suite! Help drive fair comparisons & optimize AI systems in vehicles. Focus is on camera sensor perception. 📅 Submissions close June 13th, 2025 Join: mlcommons.org/community/su...

1/ The MLPerf Training v5.0 results are here—Let’s have a fresh look at the state of large-scale AI training! This round set a new record: 201 performance results from across the industry. 🔗https://mlcommons.org/2025/06/mlperf-training-v5-0-results/

Call for papers! We are organising the 1st Workshop on Multilingual Data Quality Signals with @mlcommons.org and @eleutherai.bsky.social, held in tandem with @colmweb.org. Submit your research on multilingual data quality! Submission deadline is 23 June, more info: wmdqs.org

MLCommons is partnering with Nasscom to develop globally recognized AI reliability benchmarks, including India-specific, Hindi-language evaluations. Together, we are advancing trustworthy AI. 🔗 mlcommons.org/2025/05/nass... #AIForAll #IndiaAI #ResponsibleAI #Nasscom #MLCommons

MLCommons' MLPerf Training suite has a new #pretraining #benchmark based on #Meta’s Llama 3.1 405B model. We use the same dataset with a bigger model and longer context, offering a more relevant and challenging measure for today’s #AI systems. mlcommons.org/2025/05/trai...

As AI models grow, storage is key to #ML performance. MLCommons' @dkanter.bsky.social joins #Nutanix’s Tech Barometer podcast to explain why and how the #MLPerf #Storage #benchmark guides smarter #data #infrastructure for #AI. Listen: www.nutanix.com/theforecastb... #DataStorage #EnterpriseIT

1/ MLCommons announces the release of MLPerf Client v0.6, the first open benchmark to support NPU and GPU acceleration on consumer AI PCs. Read more: mlcommons.org/2025/04/mlpe...

#MLCommons just released two new French prompt #datasets for #AILuminate: 🔹Demo set: 1,200+ prompts, free for AI safety testing 🔹Practice set: 12,000 prompts for deeper evaluation (on request) Native speakers made both and are ready for #ModelBench. Details: mlcommons.org/2025/04/ailu... #AI #AIRR

"What’s admirable about MLPerf is that everything is shared and benchmarks are open sourced. Results need to be reproducible — no mystery can remain. This openness allows for more dynamic comparisons beyond raw side-by-side speed, like performance..." newsroom.intel.com/artificial-i...

MLCommons introduced a new Automotive PointPainting benchmark in its recent MLPerf Inference v5.0 release. The benchmark is based on established industry methods such as PointPainting, DeepLab3+, and the Waymo Open Dataset. For more details, visit mlcommons.org/2025/03/auto....

@mlcommons.org's MLPerf Inference 5.0 is here — and the message is clear: LLMs now rule the benchmarking world: www.techarena.ai/content/ai-benchmarks-shift-as-mlperf-highlights-llm-dominance?utm_source=threads

1/ MLPerf Inference benchmark v5.0 includes a new GNN benchmark providing a standardized way to assess the performance of GNN implementations and hardware accelerators.

The recent MLPerf Inference v5.0 benchmark release features advancements in benchmarking LLMs. This round introduces two key additions: the Llama 3.1 405B Instruct benchmark and an updated Llama 2 Chat 70B Interactive benchmark.

1/ We are excited to announce the latest MLCommons MLPerf Inference v5.0 benchmark suite results. This round featured robust participation from 23 submitting organizations delivering over 17,000 performance results! mlcommons.org/2025/04/mlperf-inference-v5-0-results/

MedPerf integrates smart contracts & private data objects to improve accountability & integrity. #MedPerf is an open framework for benchmarking medical AI using real-world private datasets to ensure transparency and privacy. mlcommons.org/2025/03/medp... #MedicalAI #smartcontracts

Submissions are open for the MLPerf Training v5.0 Benchmark. We are excited to add a new pretraining benchmark, llama3.1 405B, to showcase the latest innovations in AI. To participate, join the Working Group mlcommons.org/working-grou...

Join us at #HPCA2025! The MLCommons Power Working Group will present "MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from μWatts to MWatts for Sustainable AI." 📅 Tuesday, March 4, 2025 ⏰ 3:20 PM PST 📍 Session 9B

How do we ensure complex AI systems are safe? "In the context of AI applications and agents, work is underway to answer that question. I recently found one answer to that in the MLCommons AI Safety Working Group and its tool, AILuminate." Vinton G. Cerf cacm.acm.org/opinion/buil...

Croissant, a metadata format for ML-ready datasets, has gained traction since its launch in March 2024. Major platforms like Kaggle and HuggingFace now support it. The MLCommons Croissant working group co-chairs shared insights on its rapid adoption and future plans. mlcommons.org/2025/02/croi...

New Video: MLCommons and MLPerf - An Introduction @TechFieldDay.com @MLCommons.org @dkanter.bsky.social #MLPerf #AIFD6

MLCommons, in partnership with the AI Verify Foundation, released the AILuminate v1.1, incorporating new French language capabilities into its first-of-its-kind AI safety benchmark. Learn more: mlcommons.org/2025/02/ailu... #ailuminate #parisaiactionsummit #aiverifyfoundation

@mlcommons.org releases Unsupervised People's Speech dataset: 1M+ hours of multilingual audio 821K+ hours of detected speech 89 languages 48+ TB of data Empowering research in: ✅ Speech recognition ✅ Language ID ✅ Global communication tech Learn more: mlcommons.org/2025/01/new-... #nlp #datasets

MSTS was a huge group effort by 22 co-authors. Special thanks to Bertie Vidgen for co-leading the project. To core contributors @g8a9.bsky.social Felix Friedrich, @jagoldz.bsky.social and Alicia Parrish. And also to @mlcommons.org for making the project possible in the first place!

Check out the AILuminate Benchmark Creative Commons licensed DEMO prompt dataset on GitHub! This 1,200-prompt DEMO dataset is a subset of the AILuminate benchmark Practice Test prompt dataset, which evaluates AI systems across 12 hazard categories. mlcommons.org/2025/01/ailu...

Thrilled to see the Croissant project gaining momentum. This open-source metadata standard is revolutionizing how #ML datasets are shared and understood. Read Thomas Carey Wilson's insights on how #Croissant is shaping the future of open #AI data 🥐 mlcommons.org/2024/12/croi... #OpenSource

🚀 Announcing the public release of MLPerf Client v0.5! This benchmark, developed in collaboration with industry leaders like AMD, intel, Microsoft, nvidia, and Qualcomm, evaluates AI performance on PCs. mlcommons.org/2024/12/mlc-... #AI #MachineLearning