Despite the huge amount of training data, ESM-C is still unable to distinguish mean-pooled CDRH3 representations of real antibodies from those of antibodies w/ scrambled CDRs. Only VHHBERT seems to do this, and only for natural sequences

This should replace CAPTCHAs on biorxiv

But they were all of them deceived, for another leche was made...

LinkedIn as an employer looking for candidates is like ipTM: great for filtering candidates discovered elsewhere, but absolute dogshit for discovery. Quite a few people find ways to fluff up mostly-empty CVs and post a firehose of AI-generated posts. Anyway, role still open

Indeed.. here is "generate the first page of a great Nature paper that will get lots of citations. It must be very novel."

Takeda Pharma is hiring scientists at both the straight-out-of-PhD and senior levels in the AIML group, particularly those with experience training, modifying, and applying foundation models. Please reach out if youre interested and want to make the jump to industry, link below👇

Added a JAX translation of the excellent Proteina-Complexa (from nvidia, @kdidi.bsky.social , @karstenkreis.bsky.social ) to mosaic. You can do beam search with any mosaic loss (e.g. protenix + mpnn) and JAX with generate efficient GPU/TPU code.

Does anyone know what paper this figure is from? The citation I have for it turned out to be wrong

PDB 7E5Y is a funny case of two copies of the same protein in the same asymmetric unit, one of which has a register shift error (chain C, green) while the other does not (chain H, orange). Here's conserved J-gene Trp 118 (IMGT; 108 in PDB). Trastuzumab shown in teal for reference

The proteina-complexa paper is the 2nd this month suggesting that high K+E content, a staple of ProteinMPNN designs, is predictive of poor expression (the bits in bio benchmark paper showed a few weeks ago). In their case, they measure poor sequence recovery in phage display

This is new? Instead of relying on intermediate estimates of the denoised state, Proteina-Complexa just does the full denoising, calculates whatever it needs to, and sends the info back to continue guiding the unfinished diffusion roll-out

Sometimes you hit a home run, and sometimes you get to first base

So I can't say I've ever seen residual cross-attention before (where the final representations attend to earlier representations of the input data); is there any literature on when and where to use this?

Diffusion models actually learn a series of time-indexed energy landscapes, which are corrupted with different amounts of noise. The ranking ability of ProteinEBM peaked slightly above t=0. Inspired by this finding we trained an "expert" model only on low time levels, which we call ProteinEBM-x.

You can triple your hit rate without changing your NN by simply using a different search algorithm during sampling

> paper claims astonishing progress on protein folding problem > ask if it’s interesting proteins or villin headpiece denatured with urea > they say results are robust to protein sequence properties > open the pdf > villin headpiece in urea

I wrote a blog post about the future of structural bioinformatics. Where to go after AlphaFold? How do we avoid the field becoming a load of half-baked LLMs? Let me know what you think. jgreener64.github.io/posts/struct...

One of the most interesting parts of this workflow is the "sunk cost fallacy" estimator that predicts how promising a particular mutational line of inquiry is, and whether it is worth abandoning in favor of others

Most benchmarks for drug discovery AI don't effectively evaluate generative models; instead, due to the data's incompleteness, they rely on surrogate functions, like fwd folding, property prediction, or ranking previously characterized designs. No idea what the solution is here

New paper from former PhD student @tkschulze.bsky.social on supervised learning of protein variant effects across large-scale mutagenesis datasets MAVE/DMS experiments provide large amounts of data for benchmarking variant effect predictors, but may be difficult to use in supervised learning. 1/5

you and me both claude

I regret to inform you that if you are job hunting on LI and this is your profile pic then you’re ngmi

RIP my LinkedIn after mentioning a job opening in my group ☠️

The two co-first authors of this research paper, Yang & Yang, has decided to sort their names alphabetically

Protenix trained an identical model with way more training data (2025 cutoff instead of 2021), demonstrating that antibody-antigen modeling, but not protein-ligand modeling, is currently data-limited (DQ SR % means % DockQ≥0.23) with this architecture

Scribbled on at least one whiteboard in every office with "do not erase" written below

I have a concern with this paper and I want someone who knows more than me to confirm if it is founded or not. The title makes a pretty specific claim about epistasis predictions, but the method does not seem sound for masked LMs (1/3)

Hot take: this "chain X[auth Y]" notation on FASTA files pulled from the PDB is needlessly confusing, adds nothing, and needed to be changed yesterday

This plot is quite the indictment of fine-tuned PLMs, showing how performance is entirely data-dependent and, at the upper end of performance, equally achievable with randomized model weights

We're hiring - looking for folks with experience designing and training foundation models from scratch, particularly biomolecular language models, GNNs, or vision models. Lots of room for creativity in this role. Contact me if you have any interest

sir, those are my emotional support unclosed tabs

Does anyone know if the Boltz team has made any effort to get their papers peer-reviewed and published “officially”? Because if not, I have tons of respect for them for choosing not to play that game

Another LinkedIn influence confidently declaring that cryo-EM and X-ray crystallography are no longer necessary because we have AI now

After comparing these results to those of an equivalent method, I'm struck by how they don't stack up quite as well against existing baselines. Wondering if its for reasons outlined in the EmbedOpt paper, that "strong likelihood guidance often produces unphysical conformations"

A detailed comparison of de novo designed proteins by NMR finally puts to rest the idea that structure prediction confidence metrics have any correlation with local flexibility

Finding one mutation that improves a protein is hard. Finding five that work together is exponentially harder. Today in @science.org, Hsu and Konermann labs present MULTI-evolve, a lab-in-the-loop framework that does it in just one machine learning-guided round.

My reply to @anshulkundaje.bsky.social on the other site has been quoted in this @nature.com report on Isomorphic's new DDE model, which relies on private data to a degree that is probably unprecedented in the history of this field 🧪🧬

🧪🧬

Does anyone have an example of a Boltz-2 prediction of a small molecule that is reliably predicted with the wrong chirality?

Finally out! Miriam's 1st, 1st author article! 1st last author paper as a group leader! A work that started 3 years ago when I was a postdoc in Alfonso Valencia's group 🥹 doi.org/10.64898/202... Thread🧵

The fitness landscapes of binders targeting protein surfaces that are evolutionarily predisposed to engage in PPIs is much narrower and deeper than those that target surfaces with no known interacting proteins. This might be b/c random binding events is bad, and selected against

Metadynamics meets diffusion-based structure prediction: the same trick that drives enhance sampling of MD trajectories can be adapted and applied as steering terms during generation to sample diverse conformations. Seems obvious in hindsight!

Another interesting finding was how little the loss of token-selection methods tells about downstream factual accuracy on its own. Across multiple methods and model scales, validation loss (both on tokens where the model called, didn’t call, etc) is not correlated with factuality. 🧵 7/7

Why hasn't this paper gotten more attention?

@karpathy.bsky.social 's microgpt.py Train and inference GPT in 243 lines of pure, dependency-free Python. gist.github.com/karpathy/862...

Isomorphic Labs Drug Design Engine (IsoDDE), a unified computational drug-design system Announcement: www.isomorphiclabs.com/articles/the... Report: storage.googleapis.com/isomorphicla...

IMO academia creates high-quality data and novel bespoke ML architectures, but often doesn't train for enough time and doesn't scale them enough. So big tech can "innovate" by scaling alone, and the field ends up using ESM-IF instead of GVP-GNN and ProGen2 instead of ProtGPT2

New research published! I wanted to write about this, because I think the DE-Score *might* be the thing I'm proudest of so far! All my career I have dreamt of finding something so clean. The DE-Score is a way to tell when amino acid data stops being informative. 1/7 academic.oup.com/sysbio/advan...

Does anyone with familiarity with Claude Skills have a sense of how to reliably test these things? I've seen it succeed by peeking at the code instead of just running the skills as-is, which defeats the purpose of managing the context window

Training antibody language models on evolutionary trajectories inferred from ancestral sequence reconstruction and NGS data, rather than a standard masked objective, leads to improved prediction of fitness and expression