colbert-muvera-micro a 4M(!!) late interaction model late interaction models do embedding vector index queries and reranking at the same time leading to far higher accuracy huggingface.co/NeuML/colber...

Let the Model Write the Prompt | Drew Breunig #dspy #promptengineering #llms #generativeai

Here's the write up of my Data+AI Summit talk on the perils of prompts in code and how to mitigate them with DSPy. www.dbreunig.com/2025/06/10/l...

Have you heard the news? #MLflow now supports tracking for DSPy optimization workflows—just like it does for #PyTorch training! Keep reading to see what this means for your #LLM projects… 👇 #opensource #dspy #oss

📣 TODAY at 4PM PT - MLflow Community Meetup! 🔗 Register today 👉 lu.ma/mlflow423 Join the global MLflow community for two exciting tech deep dives: 🔹 MLflow + #DSPy Integration 🔹 Cleanlab + #MLflow 🎥 Streaming live on YouTube, LinkedIn, and X 💬 Live Q&A with the presenters #opensource #oss

MLflow now supports tracking for #DSPy (Community) optimization — just like it does for @pytorch.org training! 🙌 #MLflow is the first to bring full visibility into DSPy’s prompt optimization process. More observability, less guesswork. Get started today! ➡️ medium.com/@AI-on-Datab... #opensource

Join us for the next MLflow Community Meetup — Wednesday, April 23 at 4PM PT! 🗓️ 🔹 Explore the new MLflow + #DSPy integration 🔹 Learn how Cleanlab adds trust to AI workflows with MLflow 💬 Live Q&A + demos 📺 Streamed on YouTube, LinkedIn, and X 👉 RSVP: lu.ma/mlflow423 #opensource #mlflow #oss

Yes there's an evals crisis, but evaluating *models* is not even the right question most of the time LangProBe from Shangyin Tan, @lakshyaaagrawal.bsky.social, Arnav Singhvi, Liheng Lai, @michaelryan207.bsky.social et al begins to ask what complete *AI systems* we should build & under what settings

🧵Introducing LangProBe: the first benchmark testing where and how composing LLMs into language programs affects cost-quality tradeoffs! We find that, on avg across diverse tasks, smaller models within optimized programs beat calls to larger models at a fraction of the cost.

Composition & abstraction are the foundations of CS, but are clearly absent in modern ML. It's not that they're not crucial for intelligent software. But it takes building many half-working systems to abstract successfully, and it takes good abstractions to have primitives worth composing. 🧵1/2

Some quick thoughts: On why we gave the ColBERT paradigm the name "late interaction" instead of "multi-vector", a term that emerged later and that has proven to be more intuitive. **The mechanism is actually not about having multiple vectors at all.** You can see this in four different ways. 🧵1/7

Someone needs to write the book “Modern Machine Learning — the math, the myth, the legend”

a statistician walks into an error bar surprisingly, not everyone was mean to him

What do you call LLMs that exhibit reward hacking for translation? Agents gone ROUGE.

Reminder to self: Much like a good paper always starts with (i) the status of the field at the time of the proposal, (ii) what’s an annoying gap, and (iii) the novel intuition that motivates a new insightful proposal, a good course lecture must also situate every major concept this way. [1/3]

When building ColBERT, I sort of assumed it will pave the way for hypernetwork-based, pruning-capable retrieval indexes. Let me explain. The big insight in ColBERT is that we can encode each document upfront *not* into a vector, but into a rich scoring function, f: query -> float, which ... [1/3]

Anyone here have experience with working with someone in @aclmeeting.bsky.social to get a confirmation of reviewing/awards/etc for the purpose of US work visa / green card? If so, please can you share their contact with me 🙏

Strongly agree. Especially when you need to be nimble enough to adopt new or different models.

"prompt engineering" is much more about effective processes for creating good prompts for your data, than it is about specific techniques. which is to a large extent why platforms like dspy work: they kinda force you into a processes, while attempting to automate the techniques.

Using an LLM and DSPy to generate a glossary from my Jekyll site: www.dbreunig.com/2024/12/27/g...

City 311 employees spend a lot of time parsing service requests. To make this easier, I built a proof-of-concept that uses DSPy and vision models (like LLaVA) to augment 311 workflows: 📝: hershgupta.com/posts/smarte... 💻: github.com/hersh-gupta/...

Really enjoy DSPy’s workflow for LLM work. Handing off the specifics of prompt generation and engineering back to the LLM makes a lot of sense: www.dbreunig.com/2024/12/12/p...

Using DSPy to categorize historic events with Llamas www.dbreunig.com/2024/12/12/p...

Natural Language Processing—artificial intelligence that uses human language—has been on a roll lately. You’ve probably noticed! So the Stanford NLP Group has been growing, and diversifying into lots of new topics, including agents, language model programs, and socially aware #NLP. nlp.stanford.edu

Listening to this awesome talk from @cgpotts.bsky.social .. so in love with the message here .. As I'm building systems the most common questions (and review comments) I get asked is about the LL(M)M I'm using and not the systems and the problems they're solving .. youtu.be/vRTcE19M-KE?...

Dspy + Gradio + Huggingface = Magic !!

Compound AI Systems, Inference-time Compute Meetup @ NeurIPS 2024, with many AI luminaries as panelists. Poster submissions are open: lu.ma/q5r8b67t

The authors of ColPali trained a retrieval model based on SmolVLM 🤠 TLDR; - ColSmolVLM performs better than ColPali and DSE-Qwen2 on all English tasks - ColSmolVLM is more memory efficient than ColQwen2 💗 Find the model here huggingface.co/vidore/colsm...

machine learning has taught me that there are truly many ways to ablate a cat

With only 35 followers at this point, there may never be a better time for posting puns than right now.

Mat is not on 🦋—posting on his behalf! It's time to revisit common assumptions in IR! Embeddings have improved drastically, but mainstream IR evals have stagnated since MSMARCO + BEIR. We ask: on private or tricky IR tasks, are rerankers better? Surely, reranking many docs is best?

Hello world!