Real Shopify data in MLPerf v6.0. Typos, multilingual text, embedded HTML—all the complexity production AI faces at scale. No synthetic data. Submit Feb 13 https://bit.ly/4k9F5YS

The AI Economy: Flywheels for Agentic Value Creation Luis Oala on the economics that matter: → Data flywheels → Token efficiency → Compounding advantages → Sustainable growth Watch: https://youtu.be/a8psNW72l3Q #Endpoints2025 #AIEconomy

1/4 🧵 First Qwen model in MLPerf. 40M products daily. Real production data from Shopify's e-commerce infrastructure. Submit by Feb 13, 2026 👇 #MLPerf #Shopify #VLM #MLCommons

🎥 The Reliability Basket: What is Risk Coverage? Panel on measuring what can go wrong with AI systems. Beyond accuracy → comprehensive risk assessment. Watch: https://youtu.be/UgETTwnz6kY #Endpoints2025 #AIReliability

Processing 40 million products daily with 78.24% accuracy on noisy, multilingual catalog data. Not a lab benchmark—Shopify's actual production reality. Submit your VLM stack by Feb 13 → https://mlcommons.org/2026/02/vlm-inference-shopify #AIBenchmark

🚀 NEW: MLPerf Inference v6.0 debuts Qwen3-VL + Shopify Product Catalog benchmark 40M products daily. Real production data. First Qwen model in MLPerf. Submit by Feb 13, 2026 → https://bit.ly/4k9F5YS #MLPerf #VLM #Shopify #MLCommons

Bootstrapping Agents, Benchmarking, and Products Sean McGregor on the challenge: - We need agents - We need benchmarks - We need products - We need all three NOW How do we do this responsibly? Watch: https://youtu.be/PriW3_kaxLA #Endpoints2025 #AIAgents

Today! Looking forward to Benchmarking Beyond Borders: Making AI Testing Truly Global—a conversation on how we make AI evaluation more multilingual, representative, and globally relevant. Join us live: https://bit.ly/4rlwncB

Join MLCommons and global researchers Jan 29 for Benchmarking Beyond Borders—exploring community-led, multilingual AI evaluation. Jan 29, 9:00 AM ET 🔗 cdt-org.zoom.us/webinar/regi... 📺 www.youtube.com/live/glO0_Sw... @cdt.org , @mbogen.bsky.social , @alexreevegivens.bsky.social ns.bsky.social

Come kick off 2026 with David Kanter at the AI Benchmark Club. David (Head of MLPerf) will dig into how AI benchmarks measure real progress, joined by Alex Shaw (Terminal‑Bench, Harbor). 📆 Tue Jan 27, 5:30–7:30 PM PT 📍 AWS Builders Loft SF https://luma.com/smrxiq1x

🎥 KEYNOTE: Accelerating AI Trust & Velocity David Kanter on how we move fast AND build trust. Spoiler: It's not either/or. Benchmarking enables both. Watch: https://youtu.be/RkaApbTsNMc?si=FY6VbfUS74LY1N2v #Endpoints2025 #MLCommons

AI systems don’t work for the world if they’re only tested in English. Join MLCommons Jan 29 for Benchmarking Beyond Borders, a global conversation on multilingual, community-led AI evaluation—an official pre-event to the #IndiaAIImpactSummit. Register: https://bit.ly/4sX0c4G

If you've been asking about Endpoints recordings: they're here! This was genuinely one of the best AI events of the year - really practical, substantive discussions about benchmarking, deployment, safety, and economics. https://bit.ly/4jYqlfw #Endpoints2025

Endpoints 2025 sessions are available on YouTube! 14 sessions on: → AI benchmarking → Agentic evaluation → Token economics → Safety & risk → Data quality Valuable content if you're working on AI deployment, benchmarking, or safety. Full playlist: https://bit.ly/4jYqlfw #Endpoints2025 #MLCommons

We're in Hiroshima for the Global Forum for Trustworthy AI! Our co-founder Peter Mattson is presenting on MLCommons' efforts in AI safety and security. It's a privilege to collaborate with the Japan AISI, NRC Canada, and Data61 to build robust, standardized benchmarks. aisi.go.jp/activity/act...

MLCommons' David Kanter and leaders from SNIA, Qualcomm, and Cisco discuss where software-defined infrastructure is headed. The #sdxcentral chat digs into how AI is moving from applications into infrastructure.. www.sdxcentral.com/broadcasts/s... #AI #Infrastructure #SoftwareDefined #TechTrends

MLPerf Mobile app v5.0.4 is here! #MLPerf Mobile release now supports Samsung Exynos 2600 and Qualcomm's newest Snapdragon lineup, from flagship 8 Elite Gen 5 to mid-range 6 Gen 4. Meaning more comprehensive, apples-to-apples #AIperformance data across devices. play.google.com/store/apps/d...

The best AI conversations don't happen in isolation. They happen when researchers + practitioners + industry leaders are in the same room, working on shared challenges. MLCommons Endpoints Dec 1, San Diego (one day!) Registration ↓ www.eventbrite.com/e/mlcommons-... #Endpoints2025

This is something we’ve put a lot of love into. Good, bad, or indifferent, AI is something we have to deal with. Join us for a bit more of a journey into the fundamentals of AI and some of the research we’ve done with our members and partners. Cc @odihq.bsky.social

Don’t miss #MLCommons Endpoints in San Diego, Dec 1–2! Learn, connect, and shape the future of AI with top experts at Qualcomm Hall. 🗓 Dec 1–2 | 🎟 Free tickets available now! www.eventbrite.com/e/mlcommons-... #AI #MachineLearning #SanDiego

Thanks to Babel Tech Reviews for this deep dive of CORSAIR AI Workstation 300 and its handling of real-world AI workloads. Showing why tools like MLPerf Client matter: they give a clear view of how systems actually perform on common LLM & inference tasks. babeltechreviews.com/the-corsair-...

MLCommons benchmarks are now cited in #ISO /IEC AI testing standards - and we're helping shape what comes next. Contributing to standards on benchmark quality (42119-8) and AI management systems (42003). Standards are made by those in the room. mlcommons.org/2025/11/iso-... #AIStandards #MLCommons

1/ MLPerf Client v1.5 is here! New AI PC benchmarking features: -Windows ML support for improved GPU/NPU performance -Expanded platforms: Windows x64, Windows on Arm, macOS, Linux, plus iPad app -Experimental power/energy measurement GUI improvements out of beta Download: github.com/mlcommons/ml...

MLPerf Training v5.1 results are live! Record participation: 20 organizations submitted 65 unique systems featuring 12 different accelerators. Multi-node submissions increased 86% over last year, showing the industry's focus on scale. Results: mlcommons.org/2025/11/trai... #MLPerf 1/3

Don’t miss #MLCommons Endpoints in San Diego, Dec 1–2! Learn, connect, and shape the future of AI with top experts at Qualcomm Hall. 🗓 Dec 1–2 | 🎟 Free tickets available now! www.eventbrite.com/e/mlcommons-... #AI #MachineLearning #SanDiego

IEEE Spectrum's analysis shows how our benchmarks capture the real industry challenge - LLMs scale exponentially while hardware improves incrementally. This pattern highlights why evolving benchmarks are important. Stay tuned for MLPerf Training v5.1, out on 11/12. spectrum.ieee.org/mlperf-trends

🚨 NEW: We tested 39 AI models for security vulnerabilities. Not a single one was as secure as it was "safe." Today, we're releasing the industry's first standardized jailbreak benchmark. Here's what we found 🧵1/6 mlcommons.org/2025/10/ailu...

We've just released MLPerf Mobile version 5.0.2 on the Google Play Store and GitHub. This version adds support for devices based on the Samsung Exynos 2500 SoC. play.google.com/store/apps/d...

How can LLMs reliably find, understand, & analyze datasets? By combining: 📂 #Croissant – AI-ready #metadata ⚡ #MCP – agentic access to data & tools Our new blog introduces Eclair: tools that let #LLMs discover, download & explore millions of #datasets. mlcommons.org/2025/10/croi...

TinyML benchmarks finally address real-world deployment with MLCommons' new streaming benchmark in MLPerf Tiny v1.3. Tests 20-minute continuous wake word detection while measuring power and duty cycle. Technical deep dive: mlcommons.org/2025/09/mlpe... #MLPerf #TinyML #EdgeAI

New MLPerf Tiny v1.3 results! New streaming wake-word detection test + 70 results measuring sub-100KB neural networks across hardware platforms. Thanks to Qualcomm , ST Microelectronics , Syntiantcorp & Kai Jiang for pushing tiny ML forward. View results: mlcommons.org/2025/09/mlpe...

We have released MLPerf Mobile v5.0.1 with support for MediaTek Dimensity 9400 SoCs and a few other improvements. The app is available for Android phones via the Google Play Store and the MLCommons GitHub repo. Let us know what you think! t.co/715ENpV6W4

@mlcommons.org dropped MLPerf inference 5.1 with three key benchmarks for enterprise AI: a reasoning model (DeepSeek-R1), speech recognition, and a smaller LLM for summarization tasks. Catch up on this essential reading for AI decisions: www.techarena.ai/content/ai-b... #MLCommons #MLPerf

MLPerf Inference v5.1 results are live! Record 27 organizations submitted 1,472 performance results across new and established AI workloads. Three new benchmarks debut: Reasoning with Deepseek R1 Speech to text with Whisper Small LLM with Llama 3.1 8B Read More: mlcommons.org/2025/09/mlpe...

1/6 MLCommons & AVCC announce MLPerf Automotive v0.5 benchmark results—a major step for transparent, reproducible automotive AI performance data. mlcommons.org/2025/08/mlpe...

1/ MLCommons just released results for the MLPerf Storage v2.0 benchmark—an industry-standard suite for measuring storage system performance in #ML workloads. This benchmark remains architecture-neutral, representative, and reproducible. mlcommons.org/2025/08/mlpe...

MLPerf Client v1.0 is out! 🎉 The new benchmark for LLMs on PCs and client systems is now available—featuring expanded model support, new workload scenarios, and broad hardware integration. Thank you to all submitters! #AMD, #Intel, @microsoft.com, #NVIDIA, #Qualcomm mlcommons.org/2025/07/mlpe...

MLCommons just launched MLPerf Mobile on the Google Play Store! 📱 Benchmark your Android device’s AI performance on real-world ML tasks with this free, open-source app. Try it now: play.google.com/store/apps/d...

Today, MLCommons is announcing a new collaboration with contributors from across academia, civil society, and industry to co-develop an open agent reliability evaluation standard to operationalize trust in agentic deployments. 🔗https://mlcommons.org/2025/06/ares-announce/ 1/3

We're all about acceleration! 😉 Watch @priya-kasimbeg.bsky.social & @fsschneider.bsky.social speedrun an explanation of the AlgoPerf benchmark, rules, and results all within a tight 5 minutes for our #ICLR2025 paper video on "Accelerating Neural Network Training". See you in Singapore!

Companies are deploying AI tools that haven't been pressure-tested, and it's already backfiring. In her new op-ed, our President, Rebecca Weiss, breaks down how industry-led AI reliability standards can help executives avoid costly, high-profile failures. 📖 More: bit.ly/3FP0kjg @fastcompany.com

Call for Submissions! #MLCommons & @AVCConsortium are accepting submissions for the #MLPerf Automotive Benchmark Suite! Help drive fair comparisons & optimize AI systems in vehicles. Focus is on camera sensor perception. 📅 Submissions close June 13th, 2025 Join: mlcommons.org/community/su...

1/ The MLPerf Training v5.0 results are here—Let’s have a fresh look at the state of large-scale AI training! This round set a new record: 201 performance results from across the industry. 🔗https://mlcommons.org/2025/06/mlperf-training-v5-0-results/

Call for papers! We are organising the 1st Workshop on Multilingual Data Quality Signals with @mlcommons.org and @eleutherai.bsky.social, held in tandem with @colmweb.org. Submit your research on multilingual data quality! Submission deadline is 23 June, more info: wmdqs.org

MLCommons is partnering with Nasscom to develop globally recognized AI reliability benchmarks, including India-specific, Hindi-language evaluations. Together, we are advancing trustworthy AI. 🔗 mlcommons.org/2025/05/nass... #AIForAll #IndiaAI #ResponsibleAI #Nasscom #MLCommons

MLCommons' MLPerf Training suite has a new #pretraining #benchmark based on #Meta’s Llama 3.1 405B model. We use the same dataset with a bigger model and longer context, offering a more relevant and challenging measure for today’s #AI systems. mlcommons.org/2025/05/trai...

As AI models grow, storage is key to #ML performance. MLCommons' @dkanter.bsky.social joins #Nutanix’s Tech Barometer podcast to explain why and how the #MLPerf #Storage #benchmark guides smarter #data #infrastructure for #AI. Listen: www.nutanix.com/theforecastb... #DataStorage #EnterpriseIT

1/ MLCommons announces the release of MLPerf Client v0.6, the first open benchmark to support NPU and GPU acceleration on consumer AI PCs. Read more: mlcommons.org/2025/04/mlpe...

#MLCommons just released two new French prompt #datasets for #AILuminate: 🔹Demo set: 1,200+ prompts, free for AI safety testing 🔹Practice set: 12,000 prompts for deeper evaluation (on request) Native speakers made both and are ready for #ModelBench. Details: mlcommons.org/2025/04/ailu... #AI #AIRR

"What’s admirable about MLPerf is that everything is shared and benchmarks are open sourced. Results need to be reproducible — no mystery can remain. This openness allows for more dynamic comparisons beyond raw side-by-side speed, like performance..." newsroom.intel.com/artificial-i...