Check out this amazing work by @karstanczak.bsky.social on rethinking LLM alignment through frameworks from multiple disciplines!

Check out the new MMTEB benchmark🙌 if you are looking for an extensive, reproducible and open-source evaluation of text embedders!

#Repl4NLP will be co-located with NAACL this year in Albuquerque, New Mexico!

Excited to be at #NeurIPS2024 this week. Happy to meet up and chat about retrievers, RAG, embedders etc, or anything LLM-related!

(1/2) jumping back into this! read OpenScholar by @akariasai.bsky.social et al I am quite excited by the abilities of LLMs to assist in scientific discovery and literature review.

Restarting an old routine "Daily Dose of Good Papers" together w @vaibhavadlakha.bsky.social Sharing my notes and thoughts here 🧵

Honoured to be on the list! https://t.co/15CucCbxOu

Join us and be part of an amazing research community! Feel free to reach out of your want to know more about Mila or the application process. https://t.co/Z3QT7hFAS7

Completely agree, super well organised and executed! 👏 https://t.co/wGkts8EGAb

Excited to welcome @COLM_conf to the city of best bagels! 🥯 Looking forward to it! https://t.co/wUxyrDr3x6

A little teaser for LLM2Vec @COLM_conf! Stop by Tuesday morning poster session to know how we officiated the marriage of BERTs and Llamas! 🦙 https://t.co/E3HB1mwVvv

RIP freedom of speech! https://t.co/PXMS9xMnvH

🚀🚀 LLMs are the new text encoders! https://t.co/4FZ2LXCPSd

Amazing talk by @PontiEdoardo. 🙌🚀It is interesting how many different ways exist to make LLMs more efficient! https://t.co/2f2L8zLiH3

First ever arena for embedding models! ⚔️ Excited to see how this will change evaluation in this space! 🚀 https://t.co/H4FoMJrQaA

Looking for an emergency reviewer for EMNLP / ARR familiar with RAG and language models. Please reach out if you can review a paper in the next couple of days.

Great to see LLM2Vec being used for multilingual machine translation! 🚀 I believe LLM2Vec will serve as backbone of many more applications in the future! https://t.co/G18aqJ2xuv

However, this could mean we are past the point where MTEB serves as a useful signal 👀. Improving beyond the numbers we are seeing today (by training on synthetic data) carries the risk of optimizing for the benchmark rather than building general purpose embedding models. 5/N

Interestingly, Meta-Llama-3-8B only slightly outperforms Mistral-7B, the previously best model when combined with LLM2Vec 🤔. We might have reached a point where better base models are not sufficient to make substantial improvements on MTEB. 3/N

In the supervised setting, applying LLM2Vec to Meta-Llama-3-8B leads to a new state-of-the-art performance (65.01) on MTEB among models trained on publicly available data only. 2/N https://t.co/UJoOTJ4L5r

Exciting discovery! Triggers DON’T transfer universally 😮. Check out the paper for detailed experiments and analysis. https://t.co/Op7gGWBEdb

Applying LLM2Vec costs same as ~2 cappuccinos! https://t.co/O6iFXAJgoB

Very nice and intuitive explanation of our work lLM2Vec by @IntuitMachine! Using causal LLMs for representation tasks without any architecture modifications is like driving a sports car in reverse 🏎️🤯 All resources available at our project page - https://t.co/hwAiv2yrPT https://t.co/RfBNydFW9y

Great summary of our recent LLM2Vec paper! Thanks @ADarmouni! All resources available at our project page - https://t.co/hwAiv2yrPT https://t.co/bY4DoP5ms1

This is going to be my new way of bookmarking papers now! https://t.co/Dbn5juFBD9

Huggingface paper page by @_akhaliq - https://t.co/MRuPwtYCsZ

This work was done with wonderful collaborators - @ParishadBehnam @mariusmosbach @DBahdanau @NicolasChapados and @sivareddyg 10/N

We also analyze how enabling bidirectional attention without training affects the representations of decoder-only LLMs 🔍. We find that Mistral-7B is surprisingly good at using bidirectional attention out-of-the-box 🤯 and speculate it was likely trained as a prefix-LM 🤔. 7/N https://t.co/rlaub1ZQDC

LLM2Vec-transformed models are not only capable of achieving state-of-the-art results, they are also much more sample-efficient compared to baselines 🏃. This makes us particularly excited about future work that applies LLM2Vec in low-resource scenarios. 6/N https://t.co/PheoTy8fOa

We apply LLM2Vec to three different decoder-only LLMs (Sheared-LLaMA-1.3B, LLaMA-2-7B, and Mistral-7B) and evaluate in both unsupervised and supervised settings 🥁. We achieve new state-of-the-art performance in the unsupervised setting on the challenging MTEB benchmark🔥. 4/N https://t.co/iLk0d1Anlu

Encoder-only models have been the dominant models for text embedding tasks. Only recently, LLMs gained popularity for these tasks. One factor limiting their adoption is the use of causal attention which prevents tokens from capturing information across the entire sequence 👎2/N

We introduce LLM2Vec, a simple approach to transform any decoder-only LLM into a text encoder. We achieve SOTA performance on MTEB in the unsupervised and supervised category (among the models trained only on publicly available data). 🧵1/N Paper: https://t.co/1ARXK1SWwR https://t.co/L4jotnufn2

5/n Sometimes it's best not to say anything 🤫 We test the model's ability to refrain from answering when provided with an incorrect passage. Models are sensitive to input prompts. When given explicit instruction, they refrain from answering even with the gold passage https://t.co/EeNMRbf4R3

3/n For evaluating correctness ✅, we propose a simple fix. We calculate Recall - the proportion of reference answer tokens present in the model response. Recall correlates better 📈 than many established metrics and is competitive with LLM-based evaluation that uses GPT4 🤖 https://t.co/o57Z7fbZOA

2/n We evaluate these models on three QA tasks. We pair them with a retriever 🔍 and inspect 1. Did the model correctly answer the user's query? ✅ 2. Was it faithful to provided knowledge? 📖 F1, typically used to evaluate correctness in QA, fails due to model verbosity 🗣️ https://t.co/w3JbtNyLQp

Having worked directly with @harmdevries77, I can speak to his superb mentorship and management skills. Do consider this great opportunity at @ServiceNowRSRCH https://t.co/aka4n6me0m

Huge congratulations to @viks_rum, @_apoorvnandan and the entire @DaoLens team! 🎉🥳 https://t.co/CLnHabULn9

Happy to be part if this work led by @andreas_madsen. Got to learn a lot! Do read to know how faithful are current explanation methods in #NLProc https://t.co/3j1HiuJyVJ

Very excited to share our work in conversational question-answering. By introducing a dataset that is open-domain and includes topic switching, we make the task extremely challenging and closer to real-world scenarios! More details coming soon! https://t.co/BDZBtNySVf

Thanks, @sivareddyg for the incredible support and guidance over the past year! https://t.co/hD3skkncbp

Check out our work at @akbc_conf https://t.co/ZGkGq2ehhD We propose a novel task of answering regex queries over incomplete knowledge bases and establish a strong baseline with our proposed model - RotatE-Box. #AKBC2021 Joint work with Parth, @mishumausam, and @srikanta

Any leads will be appreciated. Please contact me or @pcPriyanshu https://t.co/zM5BFJKFkT

🚨 🚨 https://t.co/7ISc2bOLcf

Excited to share first conference acceptance! Out paper titled “Constrained Iterative Labeling for Open Information Extraction” has been accepted as a long paper at #emnlp2020. Joint work with @keshav_kolluru @samarth_agg1 @mishumausam and Soumen.

What If I give @neuralink a try and don’t like it, am I stuck with a hole in my skull for my entire life?#askneuralink

“I am much more frightened about robots always obeying orders than about robots rebelling against the humans.” Came across this today, a great discussion about possible future of AI and other tech from social, economic and geopolitical viewpoints https://t.co/aenR4GKrZV

Masterpiece. https://t.co/SYkOn7EQhX

#gpt3 trying to be an annoying know-it-all!😆 “Q: How do you sporgle a morgle? A: You sporgle a morgle by using a sporgle.” Giving GPT-3 a Turing Test https://t.co/HqQc9mJDwv

Algorithms shouldn’t be used for problems such as assigning grades, bail decisions, identifying criminal suspects etc. We aren’t there yet. the Secret Algorithm That's Keeping Students Out of College | WIRED https://t.co/73sSxEPrxA

With language models outpacing the growth of GPU memory, model parallelization will be indispensable in the near future https://t.co/ANdELUDTSM