A lot of extraction systems have the same problem: EITHER cram thousands of possible values into the schema OR leave the field open & fix mistakes later The better option: keep the structure fixed and narrow the valid values at runtime See how it works: blog.dottxt.ai/external-constraints

The .txt team is growing. We're looking for a: - Founding DevRel Engineer, - Member of Technical Staff - LLM Inference, and - a Rust Developer Apply here: jobs.ashbyhq.com/dottxt

Be our developer zero. Join .txt as our Founding DevRel Engineer. Apply here: jobs.ashbyhq.com/dottxt

We changed the way we handle multimodal inputs in Outlines. Most libraries will give the prompt a special role, separate it from the other inputs. We don’t anymore.

Check out this cool project by @aastroza.bsky.social to automatically convert any TV stream into a real-time movie script! Amazingly cool project using Outlines to enforce movie script formatting, and generally an impressive project. Here's the entire JSON schema he uses. github.com/aastroza/tvtxt

Another example from our internal context-free grammar hackathon. You can force a model to generate an acrostic, i.e. make each line start with a letter in a word -- HELLO in this case. Grammars are cool.

During our internal hackathon, one of our teams (named "Too Many Cooks") wrote a grammar-powered recipe generator. The language model can ONLY generate text consistent with this recipe format. Here's a recipe for the universe.

🎵 Cameron with your eyes so bright 🎵 won't you join our Discord tonight

We've got a Discord channel if you want to talk about AI engineer, compound AI systems, low overhead agents (though who knows what that means), constrained decoding, or whatever you're building. Come on by! discord.gg/ErZ8XnCmkQ

Loved this paper. Smart idea. Basically they "guess and check" whether a token would appease the compiler when an LLM is doing code generation.

"Type-Constrained Code Generation with Language Models" is a relatively new paper that addresses a common challenge with LLM-generated code. The researchers are from ETH Zurich and UC Berkeley.

My talk "Building Knowledge Graph-Based Agents with Structured Text Generation" at PyData Global 2024 is now available on YouTube: www.youtube.com/watch?v=94yu... #PyData #PyDataGlobal @dottxtai.bsky.social @pydata.bsky.social

Do people like this type of thread? They're fun to write.

We recently came across an interesting paper that helps LLMs be better at handling domain-specific languages like database queries or probabilistic programming languages, using an approach called "grammar prompting". Link + brief thread below.

Check out the sieves library from Mantis NLP! sieves is a library for structured document processing. It works with Outlines! Go check out the library, it's pretty cool. Repo: github.com/mantisai/sie...

We're considering adding an llms.txt file to the upcoming Outlines v1.0 documentation to make it more machine readable! Opinions welcome. github.com/dottxt-ai/ou...

We're delighted to sponsor the AI User Conference! It's next week, April 15th-17th. Find us at online or at San Francisco's delightful Fort Mason. Tickets: www.aiuserconference.com @cameron_pfiffer will be presenting a design for Minerva, a frontier space colony resource management system.

Check out our advocacy team @willkurt.bsky.social and @cameron.pfiffer.org on the Weaviate Podcast!

YOOOOOOOOOOOOOOOOO I got to teach a @deeplearningai.bsky.social course! I'm so happy. It was a really fun experience -- go take a look at the course and let us know how it went. bit.ly/4cdQlzG

We are excited to announce our partnership with Andrew Ng and @deeplearningai.bsky.social! Our advocacy team @willkurt.bsky.social and @cameron.pfiffer.org created a course on building AI systems using structured output from LLMs. Oh, and it’s free. bit.ly/4cdQlzG

People love our Outlines library. We figured we'd release our next big product -- Inlines! Inlines are high-performance inline roller blades for the discerning AI engineer in your life. Reply in the comments and we'll add you to the pre-order list!

Wednesday, April 2nd, at 8am PST. Stay tuned.

Outlines supports "recursive schemas", which are JSON schemas that refer to themselves. For example, here is a simple schema to generate a recursive concept map. This defines a concept, which has a title, a description, and a list of related concepts with the same schema.

Outlines v1.0 has a few interface changes coming up. One is the addition of the Application class, which combines a prompt templating function and an output type. Then, you call that application with the language model you want to use, as well as the arguments to the prompt function.

Here's what happens if you disable the use of "R" and "r" in the thinking block of Qwen 7B R1. We asked if roses are red, and it invented the phrase "conclusion point" to use instead of "answer". It eventually got the right answe.

What features do you want to see in Outlines?

Here's the code for this one: github.com/theophilec/f...

Is that our advocacy team @willkurt.bsky.social and @cameron.pfiffer.org hanging out with Andrew Ng? Wonder why.

Let us know how you're using this!

Highly recommend this post by @simonwillison.net about extracting structured data from text with LLMs. Simon puts it well: "the single most commercially valuable application of LLMs is turning unstructured content into structured data."

Théophile Cantelobre built a map of France's culinary universe by extracting structured data from restaurant reviews using Outlines. They created a knowledge graph connecting chefs and restaurants that traditional prompting couldn't achieve.

We have an open discussion on GitHub about providing token-triggered logits for Outlines, which would allow separate logit processors for different parts of the model's response, e.g. - Easy reasoning blocks for R1 and others - Disallow/allow words in think blocks 👇

Your quarterly reminder to just use structured generation. Credit: @voooooogel on twitter

Come join our Discord office hours! We want to hear what you're tinkering with.

Outlines 0.2.1 is out! - Renaming `prompt` to `Template`. `prompt` is now deprecated. - First commit by .txt rockstar Dan Palmer-Bancel - Documentation fixes by EdAbati - Adding our bluesky account to the docs (come find us there) Full changelog: https://buff.ly/4bbwWyN

Oh interesting, what could this PR be for? 👀

Check out our stellar engineer Yvan at the Rust In Paris conference! Discount link: https://buff.ly/3CMTE3E Speakers: https://buff.ly/4b7fQlF Schedule: https://buff.ly/4gKwc4P

Come visit our weekly office hours on Discord! Tell us what you're working on and what you think is cool. https://buff.ly/417gDhM

We're looking for a Staff Rust Developer! We're looking for people with - Great communication skills - Experience withlarge Rust projects - Skills in performant parsing - Understanding of automata/state machines, text indexing + search, serialization Application here if you fit:

Looking for great learning resources about structured generation? Check out this GitHub repo from Sacha Ichbiah, one of our Discord users. Repo: https://buff.ly/4jOI3Bz Our Discord: https://buff.ly/4fULKn9

Big thanks to Outlines contributor multi-stager, who reduced the size of the Outlines docker container to 7.5GB to 957MB (!) We love our contributors 🥰

We're always looking for blog posts and writing demonstrating how to use Outlines and structured generation. If you've written anything you'd like us to read and share, please reach out!

DeepSeek's R1 model is a reasoning model, which means that it generates "thinking text" before responding. A challenge is allowing the model to "think" while structuring its final output. Here's some experimental code to get JSON from R1, using Outlines! Let the tokens flow.

The .txt Discord office hours are coming up. Tuesday January 4th at 8am PST! It's a good place to - Get technical support - Share what you're working on - Chat about AI systems https://buff.ly/3Ck3zgV

Here's ANOTHER excellent blog post by Romain Florentz (https://buff.ly/4hayY4hon using Outlines to understand the relationships between entities in Dune. The Mentats are quietly updating their résumés.

We'll be hosting weekly office hours on our Discord server! Our developer relations engineer Cameron will be there to answer questions, talk about AI engineering, and generally chat about what you're building. Come see us on Tuesday mornings at 8am PST!

Want to talk about building AI systems? People have been sharing all the cool things they're working on, and they'd like to see whatever you're working on too. Come join our Discord!

New blog post! Using structured generation with Outlines + OpenAI to perform NER across Dune 🌶️🪐. Check it out: v4nn4.github.io/posts/playin... cc @dottxtai.bsky.social

Check out this demo from a Discord user (@romflorentz.bsky.social) They use structured generation to constrain output to valid addition, and then used NanoGPT to learn how to add. Code: https://buff.ly/3CoCclO Blog post: https://buff.ly/4ggHEou

HAPPENING TOMORROW GO SEE @cameron.pfiffer.org