Hugging Face's FinePDFs The largest publicly available corpus sourced exclusively from PDFs, containing about 3 trillion tokens across 475 million documents in 1733 languages. - Long context - 3T tokens from high-demand domains like legal and science. - Heavily improves over SoTA

you can run the new apertus LLMs fully locally on your (mac) laptop with just 2 lines of code: pip install mlx-lm mlx_lm.generate --model swiss-ai/Apertus-8B-Instruct-2509 --prompt "wer bisch du?" (make sure you have done huggingface-cli login before)

Am Schluss müssen sich die Medienverlage eine gesonderte Lösung überlegen, da sie kaum für alle Schweizer Blogger, Firmenwebsites, Künstler:innen,Gesundheitsportalen, eCommerce-Plattformen sprechen können. WBK N will weder Opt Out noch Opt In festschreiben.

The next generation of open LLMs should be inclusive, compliant, and multilingual by design. That’s why we @icepfl.bsky.social @ethz.ch @cscsch.bsky.social ) built Apertus.

new extensive evaluation of different optimizers for LLM training arxiv.org/abs/2509.01440

Die Schweiz steigt ins Rennen der grossen Sprachmodelle ein. Unter dem Namen #Apertus veröffentlichen @ethz.ch, @icepfl.bsky.social und das @cscsch.bsky.social das erste vollständig offene, mehrsprachige #LLM des Landes. Fürs MAZ habe ich Apertus kurz analysiert: www.maz.ch/news/apertus...

EPFL, ETH Zurich & CSCS just released Apertus, Switzerland’s first fully open-source large language model. Trained on 15T tokens in 1,000+ languages, it’s built for transparency, responsibility & the public good. Read more: actu.epfl.ch/news/apertus...

EPFL and ETH Zürich are building together a Swiss made LLM from scratch. Fully open and multilingual, the model is trained on CSCS's supercomputer "Alps" and supports sovereign, transparent, and responsible AI in Switzerland and beyond. Read more here: ai.epfl.ch/a-language-m... #ResponsibleAI

Why did Grok suddenly start talking about “white genocide in South Africa” even if asked about baseball or cute dogs? Because someone at Musk’s xAi deliberately did this, and we only found out because they were clumsy. My piece on the real dangers of AI. Gift link: www.nytimes.com/2025/05/17/o...

If you’re at @iclr-conf.bsky.social this week, come check out our spotlight poster INCLUDE during the Thursday 3:00–5:30pm session! I will be there to chat about all things multilingual & multicultural evaluation. Feel free to reach out anytime during the conference. I’d love to connect!

Using the 'right' data can hugely speed up LLM training, but how to find the best training data in the vast sea of a whole web crawl? We propose a simple classifier-based selection, enabling multilingual LLMs 🧵

Dion: A Communication-Efficient Optimizer for Large Models (inspired by PowerSGD) arxiv.org/abs/2504.05295

Prime Intellect's INTELLECT-2 The first decentralized 32B-parameter RL training run open to join for anyone with compute — fully permissionless. www.primeintellect.ai/blog/intelle...

Anastasia @koloskova.bsky.social recently won the European @ellis.eu PhD award, for her amazing work on AI and optimization. She will be joining University of Zurich as a professor this summer, and hiring PhD students and postdocs. You should apply to her group! Her website: koloskova.github.io

The Swiss AI Initiative has launched open calls for disruptive ideas - Democratizing large-scale AI for the benefit of society. Send your idea by end of March 🏃‍♂️‍➡️ , and run on one of the largest public AI clusters globally. Everyone is eligible to apply! swiss-ai.org

excellent starting point for pretraining recipes: arxiv.org/abs/2502.02737

new open weights, 24B model, with comparable performance to Llama 3.3 70B 😮. congrats mistral team! mistral.ai/news/mistral...