Batch processing using transformers and litelines libraries. In this video, I process 900 prompts in 30 seconds with an RTX A4000 with 16GB of VRAM. www.youtube.com/watch?v=7hVU...

The new litelines release should work much better in marimo notebooks. You can try it in a marimo molab: molab.marimo.io/notebooks/nb... Here is litelines documentation: alonsosilvaallende.github.io/litelines/ Here is the release changelog: github.com/alonsosilvaa...

New blog post: Force a Qwen model not to use Chinese alonsosilvaallende.github.io/blog/posts/2...

How is it possible that a 1.7 billion parameter model succeeds where a model with hundreds or thousands of billions of parameters fails? www.youtube.com/watch?v=wmgw...

The latest release of Litelines supports batch processing for Transformers library. `pip install --upgrade litelines` Here is a colab to get started: huggingface.co/datasets/alo... And here is the library documentation: alonsosilvaallende.github.io/litelines/

My talk, "Processors for Language Models," at PyData Paris 2025 is now available. I discuss my personal project, Litelines, as well as common libraries used to transform unstructured data into structured data. Link to the video: www.youtube.com/watch?v=VP4I...

I gave a talk about structured code generation for domain-specific languages: Abstract Syntax Trees (ASTs), Concrete Syntax Trees (CSTs), Deterministic Finite Automata (DFA), Regular Expressions (Regex), Pushdown Automata (PA), Context-Free Grammars (CFGs), outlines, guidance, Georges Perec, etc.

Here are the starting notebooks I presented at @pydataparis.bsky.social tinyurl.com/litelines-hf And here is the documentation of litelines: tinyurl.com/litelines #PyDataParis

Featured in marimo newsletter 🤩 marimo.io/blog/newslet...

Doing a slightly better documentation than just the README.md alonsosilvaallende.github.io/litelines/ge... Feeback?

Want to visualize the response format constraints on the LLM when working in a Jupyter notebook? Then you might be interested in my new project `litelines`. Litelines lets you visualize the selected path by the LLM. It supports a Pydantic schema as a response format, as well as regular expressions.

The recording of my presentation "Certainty-Guided Reasoning: A Dynamic Thinking Budget Approach" at the Laboratory for Information, Networking and Communication Sciences (LINCS) is now available: www.youtube.com/watch?v=8a44...

New blog post: Constrain a language model not to use the letter 'e' alonsosilvaallende.github.io/blog/posts/2... In this post, I constrain a small language model (0.6B parameters) with a logits processor to accomplish something GPT-4o fails to do (see chatgpt.com/share/687908...).

TIL how to add notebook cells running on @pyodide.org to Quarto projects, such as my blog: alonsosilvaallende.github.io/til/posts/20... It's very easy to change the Pyodide version if needed. Thanks @coatless.bsky.social for this great Quarto extension

tfw u need neo4j to put together the polycule chart

New blog post: Understanding Logits Processors alonsosilvaallende.github.io/blog/posts/2... I start with a basic min length example, then force the model to check its answer, followed by making reasoning models stop thinking once they reach a thinking budget & finally force the model to think longer

New blog post: Understanding Structured Outputs alonsosilvaallende.github.io/blog/posts/2... This is the power behind structured ouputs libraries such as Instructor and Marvin. I provide a basic example of extraction, a slightly more complex one, then classification, and finally SO in WebAssembly.

New blog post: Understanding Function Calling I provide a basic example of FC, then a slightly more complex example by allowing an LM to use Python. I explain the conversational response as a tool trick. Finally, FC in the browser by using WebAssembly alonsosilvaallende.github.io/blog/posts/2...

So I appear in the Celebration of 100 years of Bell Labs video with our LLM robots (a.k.a. IndustrialGPT) for less than a second :-D www.youtube.com/watch?v=Fu_I...

New blog post: Understanding LLM Memory alonsosilvaallende.github.io/blog/posts/2... Using the Marimo extension for Quarto.

New post: Understanding Chat Templates alonsosilvaallende.github.io/blog/posts/2...

New post: Understanding Tokenizers alonsosilvaallende.github.io/blog/posts/2...

Celebrating Bell Labs’ 100 years anniversary #NokiaBellLabs100

TIL to do quizzes with JupyterQuiz: alonsosilvaallende.github.io/til/posts/20... I like this simple way to provide feedback to students. I’m using these quizzes to motivate my kid to learn new things.

TIL you can embed a live REPL on a website: alonsosilvaallende.github.io/til/posts/20... I think this is a great tool for writing tutorials and blog posts.

Finally meeting @remilouf.bsky.social !

this was good

The new release of logits-processor-zoo supports batch inference of the logits processors with vLLM `pip install logits-processor-zoo` Here is the repo: github.com/NVIDIA/logit... Here is the commit 😊: github.com/NVIDIA/logit...

My talk "Building Knowledge Graph-Based Agents with Structured Text Generation" at PyData Global 2024 is now available on YouTube: www.youtube.com/watch?v=94yu... #PyData #PyDataGlobal @dottxtai.bsky.social @pydata.bsky.social

More coverage of our work at @ieeespectrum.bsky.social 🤩

Chat Templates Chat is an abstraction. LLMs know nothing about system messages, user messages, etc. The translation between messages and the received string representation is handled by the chat template. Code: github.com/alonsosilvaa... To try it: uv run raw.githubusercontent.com/alonsosilvaa...

Next token The code can be found here: github.com/alonsosilvaa... If you have uv installed, you can try it with this command: uv run raw.githubusercontent.com/alonsosilvaa...

I figured out a minimal pattern for building a completely custom website using GitHub Actions and deploying the result to GitHub Pages, so I wrote it up as a TIL til.simonwillison.net/github-actio...

The answer is wrong but DeepSeek's reasoning traces are always fun to watch :-)

Revive la charla "Una introducción amigable a modelos de difusión" de Cristian Garcia disponible en YouTube: www.youtube.com/live/uHvP5uR... cc: @aastroza.bsky.social @sebastiandres.bsky.social @alonsosilva.bsky.social

I use Kùzu but I must admit these are cute 😜

Este jueves 23 de enero, @capetorch.bsky.social Senior MLE en @weightsbiases.bsky.social compartirá su experiencia en implementación de LLMs en producción junto a la comunidad IADevs en las oficinas de @fintual.bsky.social. Regístrate aquí: lu.ma/bcny2f4l Cupos limitados. ¡No te quedes fuera!

AI Eats the World www.youtube.com/watch?v=LGDa...

My 2025 resolution is to take more risks

Me entrevistaron para el podcast IA para los negocios de EvoAcademy ¡Espero que les guste! youtu.be/-56EfNIGy0k

This is super cool, run an AI chatbot, pure Python fully in the browser using Solara. Great work @alonsosilva.bsky.social ❤️

The pictures from the @iadevs.bsky.social conference are now available: photos.google.com/share/AF1Qip... It was a super cool conference!

Mañana presento “Creación de agentes basados en gráficos de conocimiento con generación de texto estructurado y modelos open-weights” en el Centro de Modelamiento Matemático de la Universidad de Chile www.cmm.uchile.cl?p=59116

Hoy estaré presentando Mejorando la generación aumentada con recuperación (RAG) con la búsqueda híbrida en la conferencia EVIC 2024: www.evic.cl ¡Nos vemos!

Este lunes estaré presentando Potenciando aplicaciones RAG con grafos de conocimiento. La inscripción es gratuita: iadevs.cl ¡Los espero!

🗣️ "Instead of relying on the Pandas API, we are using Narwhals [...]. This means faster, more efficient handling of tabular data and serious performance gains for data apps at scale." 😍 Love this from @plotly.com Plotly v6 isn't out yet but you can try it with `pip install -U --pre plotly`

Are you frustrated by how GitHub renders Jupyter notebooks? I have public service that renders GitHub notebooks with Quarto nbsanity.com It now works with gists!

Stratified cross-validation considered harmful? Since the early days of @scikit-learn.org, we have nudged users into using stratified cross-validation in the presence of class imbalance. I think this was a design mistake. Here is a notebook to back my claim: gist.github.com/ogrisel/af21... 👇

🎉 NEW Cookbook: Mistral AI and Solara - Check out this cookbook on how to create a chatbot with Solara to talk to your PDFs with Mistral AI models: github.com/mistralai/co...