Kyle OโBrien
@kyletokens.bsky.social
๐ค 53
๐ฅ 105
๐ 12
AGI Alignment & Control Pretraining @ Geodesic Research |
https://kyobrien.io
reposted by
Kyle OโBrien
Geodesic Research
about 1 month ago
Thanks to a generous philanthropic grant (pending final logistics) from Coefficient Giving, ๐๐ฆ๐ฐ๐ฅ๐ฆ๐ด๐ช๐ค ๐ช๐ด ๐ฉ๐ช๐ณ๐ช๐ฏ๐จ ๐๐ฆ๐ฎ๐ฃ๐ฆ๐ณ๐ด ๐ฐ๐ง ๐๐ฆ๐ค๐ฉ๐ฏ๐ช๐ค๐ข๐ญ ๐๐ต๐ข๐ง๐ง. Come build the base of alignment with us ๐ค Applications now open:
airtable.com/appuugUGFPJE...
1
3
2
Geodesic mission is to develop the science of providing robustly aligned initializations for RL, where alignment priors persist through the remainder of training. Do considering applying if you want to help make sharing the world with superintelligence go well for humanity.
add a skeleton here at some point
about 2 months ago
0
0
0
Very excited to see pretraining safety efforts! Weโre only now beginning to understand how promising pretraining safety and alignment interventions are. Much in the way that curating the base model is important for capabilities like reasoning, so too might it be important for safety.
add a skeleton here at some point
5 months ago
0
3
0
I've joined Geodesic Research to build the open-science field of AI safety pretraining research. Our first paper is wild. TL;DR โ LLMs pretrained on data about misaligned AIs themselves become less aligned. Luckily, pretraining LLMs with data about good AIs helps them become more aligned.
loading . . .
Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment
LLMs trained on data about misaligned AIs themselves become less aligned. Luckily, pretraining LLMs with synthetic data about good AIs helps them become more aligned. These alignment priors persist th...
https://alignmentpretraining.ai/
6 months ago
0
3
0
reposted by
Kyle OโBrien
Alex Turner
7 months ago
The first pretraining results are in, and it looks like models indeed have self-fulfilling misalignment properties. Great work by Tice et al!
alignmentpretraining.ai
loading . . .
Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment
LLMs trained on data about misaligned AIs themselves become less aligned. Luckily, pretraining LLMs with synthetic data about good AIs helps them become more aligned. These alignment priors persistโฆ
https://alignmentpretraining.ai/
0
6
1
You know you're AGI-pilled when your Spotify Wrapped looks like this.
7 months ago
0
1
0
Applications to apply for the ERA:AI Fellowship close November 3rd! Participating in this Summer's fellowship was my gateway into pursuing AGI safety research full-time. I will be a research manager for the upcoming Winter fellowships. Feel free to DM me with questions. :)
erafellowship.org
loading . . .
ERA Fellowship
ERA is a talent programme supporting early-career researchers and entrepreneurs to understand and mitigate risks from frontier AI, based at Cambridge, UK.
https://erafellowship.org/
8 months ago
0
0
0
reposted by
Kyle OโBrien
Cas (Stephen Casper)
10 months ago
๐๐๐ I'm excited to be on the faculty job market this fall. I just updated my website with my CV.
stephencasper.com
loading . . .
Stephen Casper
Visit the post for more.
https://stephencasper.com/
0
18
5
reposted by
Kyle OโBrien
Sharon Goldman
11 months ago
Thanks to
@stellaathena.bsky.social
for chatting with me about Deep Ignorance: the new paper/project from Eleuther AI and the UK AISI. Bottom line: Worried AI could teach people to build bioweapons? Donโt teach it how
fortune.com/2025/08/14/w...
loading . . .
AI safety tip: if you donโt want it giving bioweapon instructions, maybe donโt put them in the training data, say researchers
New research shows that scrubbing risky material from AI training data can build safeguards that are harder to bypass โ and one author calls out tech giants for keeping such work under wraps.
https://fortune.com/2025/08/14/worried-ai-could-teach-people-to-build-bioweapons-dont-teach-it-how-say-researchers/
0
11
3
This articles covers our work for a general audience. :)
add a skeleton here at some point
11 months ago
0
4
0
Big and True :)
add a skeleton here at some point
11 months ago
0
3
0
I like that OpenAI published this. They were able to fine-tune away GPT-oss's refusal, decreasing refusal rates to ~0%. These results aren't surprising. Acknowledging that existing safeguards don't generalize to open models is the first step in developing solutions.
arxiv.org/abs/2508.031...
loading . . .
Estimating Worst-Case Frontier Risks of Open-Weight LLMs
In this paper, we study the worst-case frontier risks of releasing gpt-oss. We introduce malicious fine-tuning (MFT), where we attempt to elicit maximum capabilities by fine-tuning gpt-oss to be as ca...
https://arxiv.org/abs/2508.03153v1
11 months ago
0
1
0
I've learned a lot over the past two years of getting into research, mostly from mistakes. Iโve made many mistakes. Such is science. Good research is often at the adjacent possible. I've written up much of what I've learned now that I'm beginning to mentor others.
open.substack.com/pub/kyletoke...
loading . . .
Donโt "Think", Just Think
Lessons From Breaking Into AI Research
https://open.substack.com/pub/kyletokens/p/dont-think-just-think?r=3gtmk8&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false
11 months ago
0
0
0
I led an effort at Microsoft last Fall that studied whether SAE steering was an effective way to improve jailbreak robustness. Our paper on SAE steering has been accepted to the ICML Actionable Interpretability Workshop! Venue:
actionable-interpretability.github.io
Paper:
arxiv.org/abs/2411.11296
loading . . .
Steering Language Model Refusal with Sparse Autoencoders
Responsible deployment of language models requires mechanisms for refusing unsafe prompts while preserving model performance. While most approaches modify model weights through additional training, we...
https://arxiv.org/abs/2411.11296
about 1 year ago
0
2
0
I'll be in England this summer as an AI Safety Research Fellow with ERA!
erafellowship.org/fellowship
I will be studying data filtering and tamper-resistant unlearning for open-weight AI safety so that the community can continue to benefit from open models as capabilities improve.
loading . . .
Fellowship โ ERA Fellowship
https://erafellowship.org/fellowship
about 1 year ago
1
5
0
you reached the end!!
feeds!
log in