Alignment is an incomplete contracting problem.
You can't fully specify a contract (i.e. a payoff space *cough cough RL*) for every contingency, because of the limits of observability, what courts uphold, etc.
So there are hold-up risks.
If you can read an AI's mind, it underinvests in learning.
9 days ago