Papers #2-3: arxiv.org/abs/2402.10210 and arxiv.org/abs/2405.00675 from the incredible @quanquangu.bsky.social. I really like how they explore new techniques for RLHF

Pretraining will only end once we find the optimal scaling law.

With the delivery of MARS complete, the focus now shifts to delivering new architectures.

This Thanksgiving, I want to express my heartfelt gratitude to all the students, colleagues, and collaborators who have contributed to the success of SPIN, SPPO, DPLM, GPM, MARS, and many other projects. Your hard work and dedication continue to be truly inspiring.

Please reply to this message or DM me if you’d like to be added!

Tulu 3 SFT mix trending on HuggingFace :D , next step make preferences and RL datasets more accessible.

OLMo 2 is out 🥳 7B and 13B trained on 5T tokens, and meticulousy instruction tuned using Tulu 3 recipe. Simply the best fully open models yet. Really proud of the work & the amazing team at @ai2.bsky.social

Scene thru my window, today, Pittsburgh.

Replacing the mean by an arbitrary constant, part 2. This very cool fact is from Tom Cover's 1990 Shannon Lecture: #MathSky

PSGD ❤️ MARS MARS is a new exciting variance reduction technique from @quanquangu.bsky.social 's group which can help stabilize and accelerate your deep learning pipeline. All that is needed is a gradient buffer. Here MARS speeds up the convergence of PSGD ultimately leading to a better solution.

Deep learning theory has arrived (on blue sky)!

Tomorrow at 6 PM UTC, Asaf Cassel will talk about "Warm-Up Free Policy Optimization: Improved Regret in Linear Markov Decision Processes".

We are on Bluesky as well! We will keep posting on both X and here.

Just created the Starter Pack for Optimization Researchers to help you on your journey into optimization! 🚀 Did I miss anyone? Tag them or let me know what to add! go.bsky.app/VjpyyRw

The Oregon coast is incredibly beautiful

Join us tomorrow for the: AI for Science summit. In person places are full, but you can still join us online (click on the below to sign up). science.ai.cam.ac.uk/events/ai-fo...

In addition to the Deep Learning Theory starter pack, I've also put together a starter pack for Reinforcement Learning Theory. Let me know if you'd like to be included or suggest someone to add to the list! go.bsky.app/LWyGAAu

Graduate school application season is here again, and I just spent the whole weekend writing a few letters! 🎓 When submitting, many schools still require overly detailed forms filled with endless questions and checkboxes. Isn’t it time to simplify this process and focus on what truly matters?

A gentle reminder to @iclr-conf.bsky.social authors: your revision must not exceed 10 pages, adhering to the same requirements as the original submission.

This is accurate af

Just put together a starter pack for Deep Learning Theory. Let me know if you'd like to be included or suggest someone to add to the list! go.bsky.app/2qnppia

RLC will be held at the Univ. of Alberta, Edmonton, in 2025. I'm happy to say that we now have the conference's website out: rl-conference.cc/index.html Looking forward to seeing you all there! @rl-conference.bsky.social #reinforcementlearning

It's Sunday morning so taking a minute for a nerdy thread (on math, tokenizers and LLMs) of the work of our intern Garreth By adding a few lines of code to the base Llama 3 tokenizer, he got a free boost in arithmetic performance 😮 [thread]

Write math on 🦋 with UnicodeIt! For example: θ ∈ ℝⁿ or pp̅ → μ⁺μ⁻ Use website or install system-wide in Linux, macOS, or windows www.unicodeit.net (Created several years ago with @svenkreiss.bsky.social)

I’m increasingly unsure there are specific rules/laws for pretraining schedule except that peak LR instability means that you need a warmup and a cooldown. And in the post-Chinchilla era you probably want to slow cook the model (so go easy on lr, smooth the schedule).

Everything you want to know about FSDP and more... FSDP works on layers at a time and enables perf wins by overlapping the data-fetching part of the next layer with the computation of the current layer. This overlap means that the data-fetching can be “hidden”.

LLMs generate novel word sequences not contained in their pretraining data. However, compared to humans, models generate significantly fewer novel n-grams. RLHF = 30% *more* copying than base! Awesome work from the awesome Ximing Lu (gloriaximinglu.github.io) et al. 🤩 arxiv.org/pdf/2410.04265

🚀 Excited to announce KVPress — our open-source library for efficient LLM KV cache compression! 👉 Check it out (and drop a ⭐): github.com/NVIDIA/kvpress 🔗 Full details in the thread 🧵 (1/4)

if you have a @github.com profile, can i ask you to update it with your @bsky.app handle? 🙏 👉 it enables some very cool integrations, like auto curated feeds and starter packs for contributors and tech

What are some "nice-in-theory but unhelpful-in-practice" approaches in LLMs?

Just created the Starter Pack for Optimization Researchers to help you on your journey into optimization! 🚀 Did I miss anyone? Tag them or let me know what to add! go.bsky.app/VjpyyRw

The real competition isn’t between platforms, it’s between how we choose to use them. Agree or disagree?

Blast from the past, wrote these equations in this form 6 years ago, I remember it felt quite clarifying at the time ... A bygone era unfortunately

For those who don’t feel like they fit into my Grumpy Machine Learners list (which I still need to update based on 100+ requests) I’ve created another starter pack: go.bsky.app/Js7ka12 (Self) nominations welcome.

Always so gratifying to see people citing (and, more importantly, using) one of my expository notes!

There’s a single formula that makes all of your diffusion models possible: Tweedie's Say 𝐱 is a noisy version of 𝐮 with 𝐞 ∼ 𝒩(𝟎, σ² 𝐈) 𝐱 = 𝐮 + 𝐞 MMSE estimate of 𝐮 is 𝔼[𝐮|𝐱] & would seem to require P(𝐮|𝐱). Yet Tweedie says P(𝐱) is all you need 1/3

(1/n) 💡How can we speed up the serial runtime of long pre-training runs? Enter Critical Batch Size (CBS): the tipping point where the gains of data parallelism balance with diminishing efficiency. Doubling batch size halves the optimization steps—until we hit CBS, beyond which returns diminish.

In addition to the Deep Learning Theory starter pack, I've also put together a starter pack for Reinforcement Learning Theory. Let me know if you'd like to be included or suggest someone to add to the list! go.bsky.app/LWyGAAu

Just put together a starter pack for Deep Learning Theory. Let me know if you'd like to be included or suggest someone to add to the list! go.bsky.app/2qnppia

The first list filled up, so here's a second list of AI for Science researchers on bluesky. Let me know if I missed you / if you'd like to join! bsky.app/starter-pack...

I'm making a list of AI for Science researchers on bluesky — let me know if I missed you / if you'd like to join! go.bsky.app/AcP9Lix

🗣️ ICLR response period ends soon -- remember to check author responses, ask follow-up questions, and revise scores. Reviewers have until Nov 26 AoE to ask questions. Authors have a one day grace period until Nov 27 to respond to last minute questions.

MARS optimizer has been out for four days now, and we’re eager to hear your feedback! How does it perform on your projects compared to AdamW and other optimizers? Github:

RNN--> Transformer-->❓ 🤔

Just crash-landed on BlueSky. Who’s here to show me around?