What is AI alignment?

How it works

The International AI Safety Report 2026, a research synthesis chaired by Yoshua Bengio, was published February 3, 2026. It defines alignment in its glossary as the tendency of an AI model or system to use its capabilities in keeping with human intentions, values or norms. Whose intentions count depends on context: developers, users, specific communities or society as a whole.

The report describes two ways misalignment can arise. In goal misspecification, the objective a developer or user sets is an imperfect stand-in for what they want. In goal misgeneralization, a model learns a goal that fits its training data but applies it wrongly in new situations. In one cited experiment, an agent trained to collect a coin that always sat in one place kept going there after the coin was moved.

The 2016 paper Concrete Problems in AI Safety, by Dario Amodei, Chris Olah and four co-authors, described unintended, harmful AI behavior as accidents. It listed five research problems, including avoiding side effects and avoiding reward hacking. The 2026 report describes reward hacking as scoring well through an unintended shortcut without doing the intended task.

Why it matters

The report ties misalignment to loss of control scenarios. In these, AI systems operate outside anyone's control and regaining it is extremely costly or impossible. A misaligned system, it says, might give false information, hide its actions or resist shutdown. Experts disagree on the likelihood. Some consider such scenarios implausible. Others see them as likely enough to merit attention because the harm could be severe. The report finds today's systems show early forms of the capabilities involved, short of what loss of control would require.

Where things stand in 2026

The report calls alignment an open scientific problem. It notes that since January 2025 models have become better at recognizing tests and exploiting loopholes in evaluations, making their behavior harder to assess. Research directions include interpretability (examining a model's internal workings) and scalable oversight (using AI systems to oversee other AI systems).

The UK government said in February 2026 that about £27 million would be available through the AI Security Institute's Alignment Project, including £5.6 million from OpenAI. It reported first grants to 60 projects in 8 countries and a second round due that summer. The project's website, checked in October 2026, says applications are closed and a similar program is unlikely to run in 2026.