Some random thoughts after chatting with multiple friends: I do feel that one reason the general ML research community is getting worse (imo, maybe not for others) is that we don't share the bad things we found with others more often. 1/n

I will be at #icml2025 next week to present our work on LLM unlearning evaluation [https://arxiv.org/abs/2412.08559]. We also have a work on AI copyright to be presented at the MemFM and R2FM workshop. Please let me know if you're also around! I will be around 7/15-7/17.

This is a great paper! It resonate with one of our recent work (a short version to appear at ICML MemFM workshop!). We really need to be careful on "defining meaningful" copyright measure.

What needs to be take care of when applying privacy amplification by iteration to zeroth-order optimization? Can it even be done? What's the "good design" for DP zeroth-order method? Check out our latest work! It's so nice to collaborate with Wei-Ning (as usual) and Pan!

Open Problem: Selection via Low-Sensitivity Queries

Our paper about LLM unlearning evaluation is accepted by #icml2025 ! Thanks to the leading author Rongzhe, and my collaborators @mufei-li.bsky.social @xiangyue96.bsky.social (and others may not be on Bluesky). It's my first "last" author paper. Feels quite special :p 1/n

I wonder how well this result can be applied to convert the KL-based result in the sampling literature (i.e., LMC convergence) to Renyi divergence, compared to those results that directly bound the Renyi divergence (i.e., the results in Sinho Chewi's book or the paper by Vempala and Wibisono 😂)

I wrote a post explaining why, in practice, privacy amplification by subsampling doesn't quite work as well as promised. This is a significant problem for differentially private machine learning applications, but I don't know if this is as widely known as it should be.

PSA — if you’re interested in learning about statistical aspects of optimal transport, check out this new monograph by Sinho Chewi, Jonathan Niles-Weed, and Philippe Rigollet: link.springer.com/book/10.1007...

Privacy Amplification by Subsampling

Life Update: I am happy to share the news that I will be an Assistant Professor at the National Taiwan University EE department! I am very grateful for this opportunity to be back in my home country, especially at the university where I was an undergrad! 1/3

I am so shocked to learn that Poisson (in French) means fish...... As a person who constantly deals with Poisson distribution, Poissonization, etc I now have a completely different feeling about Poisson 🤣. I guess we always learn something unexpected on the internet 🤣

I will give a talk at GaTech CSE seminar this Friday on the topic: "Machine Unlearning: The General Theory and LLM Practice of Privacy". Please join if you are around :) cse.gatech.edu/events/2025/...

I am glad to share that our paper on hidden-state Noisy SGD DP analysis for non-convex non-smooth problems has been accepted at #ICLR2025! I really appreciate the effort from reviewers, AC, and all my friends who provided valuable comments and feedback!

With @adamsmith.xyz and @thejonullman.bsky.social, we have compiled a set of profiles of 29 people in the "foundations of responsible computing" community ("mathematical research in computation and society writ large") who are on the faculty job market. Link: drive.google.com/file/d/1Hyvg... 1/3

Why do we need "theoretical guarantees" for trustworthy AI? We need to prevent the worst-case scenario, where theory in AI truly shines and is necessary, in my opinion. That's also why my work with theoretical guarantees for machine unlearning and DP matters! 😉

It's my great pleasure to contribute to the great A3D3 community. Congrats to all #A3D3 members!

The last time when I attended NeurIPS in 2019 Vancouver, I missed my flight back to Urbana due to a border check. Today after NeurIPS 2024 I got stuck in Dallas due to a flight cancellation...🥲🥲🥲

I will present 1 spotlight in the noon and 1 poster in the evening tomorrow, both in West building about unlearning theory! Feel free to come and chat if you’re around 😀 #NeurIPS2024

Privacy-Preserving Retrieval Augmented Generation with Differential Privacy Tatsuki Koga, Ruihan Wu, Kamalika Chaudhuri http://arxiv.org/abs/2412.04697

[NeurIPS 2024] I will be at NeurIPS this year to present our works on privacy! Two papers on the theory of machine unlearning (including a spotlight) and one on DP-PageRank with hidden-state DP analysis! Let me know if you are also around and would like to chat about privacy/trustworthy AI :)

On the TCS job market: Eli Chien! Eli works on TCS∩ML: e.g., Machine Unlearning, hidden-state DP analysis and their application to graph ML. He recently generalized the celebrated shifted-divergence analysis to non-convex non-smooth setting for hidden-state DP-SGD. 1/2 #TCSSky #AcademicJobMarket

🤔: How's your lemma different from the lemma of Chien et al.? Me thinking: You are asking the right person 😆 (after some explanation) Me: We agree that there are some similarities and we will give Chien et al. more credit in our revision. 🤣