Alignment
Alignment is the problem of making an AI system reliably pursue the goals its designers and users intend, rather than some proxy that merely scored well in training. A system rewarded for a measurable target can find shortcuts that hit the target while missing the point, a failure called reward hacking. Alignment research tries to specify, train and check intended behavior, especially for systems more capable than the people overseeing them.
Why it comes up
Amodei's 2016 "Concrete Problems in AI Safety" framed alignment as ordinary engineering, Hinton and Bengio warn it may not be solved before systems exceed human ability, and LeCun argues that safe objectives can be designed in.
People and milestones
Deep reinforcement learning from human preferences, 2017
Dario Amodei
Geoffrey Hinton
Yoshua Bengio
Yann LeCun
Ilya Sutskever
Dario Amodei
Geoffrey Hinton
Yoshua Bengio
Yann LeCun
Ilya Sutskever