1/ Most membership inference attacks (MIAs) have seemingly converged to black-box settings, driven by empirical evidence and theoretical folklore suggesting black-box access was optimal. But what if this assumption missed something critical? 😨 tl;dr? It did 🧵

Temporally shifted data splits in membership inference can be misleading ⚠️ Be cautious when interpreting these benchmarks!