Love to see it!

New postdoc position at AarhusNLP, come join us! The research includes efficient post-training, alignment, evaluation, and preference optimization, but we are very flexible for reinterpretation. So, if you think that you might be a partial fit do apply! international.au.dk/about/profil...

Last week at #NoDaLiDa, we presented our work on 🇪🇺EuroEval, a large-scale benchmark for evaluating decoders and encoders.

The framework consists of 9 (🇬🇧🇫🇷🇩🇪🇳🇱🇸🇪🇩🇰🇳🇴🇮🇸🇫🇴) languages, with more to come, where each includes a language understanding and generation benchmark

I am delighted to announce that we have released 🎊 MMTEB 🎊, a large-scale collaboration working on efficient multilingual evaluation of embedding models. This work implements >500 evaluation tasks across >1000 languages and covers a wide range of use cases and domains🩺👩‍💻⚖️

Multilingual MTEB is soon to be released and with it, a shining new benchmark with a zero-shot filter! However, zero-shot is quite hard to define in a time of derivative models and synthetic data. If you have an opinion on how it should define zero-shot, let us know: github.com/embeddings-b...

New favourite razor

📣 Vacancy for Assistant Professor of Cognitive Science at Department of Linguistics, Cognitive Science and Semiotics, Aarhus University, Denmark. (Deadline January 6) international.au.dk/about/profil...

Does anyone know of good methods for tracking dataset contaminations that doesn't rely on generation? Anything would be greatly appreciated Asking as we would like to track and detect dataset contamination in MTEB: github.com/embeddings-b...

Ugens afsnit af Verbos podcast er live med @kennethenevoldsen.bsky.social og Thomas Kobber Panum til en snak om AI's udvikling med udgangspunkt i Ilya Sutskevers talk til NeurIPS 🔥 #dkai #dktech Lyt her 👇 YouTube: youtu.be/IpEla8mZHnU?... Spotify: open.spotify.com/episode/41WT...

Så er vi oppe og på 50%! data-is-better-together-fineweb-c.hf.space/share-your-p...

Any good reason for using (machine) translated datasets/benchmarks for evaluating language models?

On my way to Neurips to present our work on the Scandinavian Embedding Benchmark (SEB) for evaluating embedding for retrieval, classification etc. for mainland Scandinavian languages. Is there anything I should see while I am there?