The meaning of alignment is widening
In technical circles, AI alignment has often meant ensuring that a model does what its designers or users intend, even in unfamiliar settings. That remains a serious problem. Large models can still hallucinate, pursue proxy objectives, reward superficial optimisation and behave unpredictably under changing prompts or incentives. Yet this classical definition is too narrow for the systems now entering ordinary life.
Today’s AI is not only a laboratory artefact. It is embedded in hiring tools, recommendation engines, decision-support systems, customer service, educational software and public-sector workflows. In these contexts, the key issue is not simply whether a model follows instructions faithfully. It is whether the instructions themselves reflect legitimate goals, whether trade-offs are visible, and whether affected people have recourse when systems cause harm.
Alignment is increasingly less about getting a model to obey, and more about deciding who gets to specify the goals worth obeying.
This broader view has been building for some time. A major review in Nature argued that the challenge is not merely technical but sociotechnical, requiring institutions, norms and public accountability alongside better methods. Research from Stanford’s Institute for Human-Centred Artificial Intelligence and policy work from the OECD make a similar point: AI systems inherit values from the data, objectives, metrics and organisations surrounding them. In that sense, alignment is not a property of the model alone. It is a property of the entire deployment regime.
Why technical success does not settle ethical legitimacy
A system can be technically well-aligned to a narrow objective and still be ethically misaligned with social expectations. A content-ranking model may optimise engagement exactly as designed, while amplifying sensationalism. A hiring filter may minimise recruitment costs while disadvantaging certain applicants. A school proctoring tool may deter cheating while normalising intrusive surveillance. In each case, the question is not whether the system failed to execute instructions. It may have succeeded. The deeper concern is that the objective function was too thin to capture what the institution owed to people affected by the outcome.
This is one reason why fairness, transparency and accountability have become inseparable from alignment. The US National Institute of Standards and Technology’s AI Risk Management Framework treats trustworthy AI as a combination of validity, safety, security, accountability, explainability, privacy and fairness. The European Union’s AI Act, meanwhile, does not assume that technical performance alone is enough. It sorts applications by risk and imposes obligations around testing, documentation, human oversight and post-market monitoring.
Neither approach solves the problem outright. But both recognise a crucial fact: an AI system can satisfy internal performance metrics while violating external norms. Alignment, then, cannot be reduced to optimisation quality. It must also address the legitimacy of the target being optimised.
Value pluralism is not a bug in the debate
One reason alignment is difficult is that societies do not agree on a single set of values. Privacy, efficiency, safety, liberty, equality and dignity can all matter at once, and often pull in different directions. There is no universal scalar objective into which these commitments can be neatly compressed. Political theorists have long understood this. AI governance is now rediscovering it.
The implication is uncomfortable for engineers seeking clean formalisation. Some conflicts cannot be resolved by more data or better tuning because they concern legitimate disagreement. Should a medical triage system prioritise speed or explainability? Should a classroom assistant maximise personalisation if doing so requires detailed behavioural monitoring? Should a moderation tool err towards free expression or protection from harm? These are not only modelling choices. They are contestable social judgments.
Alignment is increasingly less about getting a model to obey, and more about deciding who gets to specify the goals worth obeying.
The OECD’s AI Principles and UNESCO’s Recommendation on the Ethics of Artificial Intelligence both acknowledge this pluralism by grounding AI governance in human rights, democratic values and cultural diversity rather than a single technocratic standard. Such frameworks can appear abstract, but their abstraction is deliberate. They accept that alignment must accommodate disagreement, not pretend to eliminate it.
The hidden politics of datasets and benchmarks
Much discussion of alignment still gravitates towards model behaviour at inference time: refusals, honesty, harmlessness, robustness. These are important. But upstream choices often matter more. Datasets determine what a model sees as normal; labels encode human judgment; benchmarks privilege measurable traits over harder-to-measure social effects. The politics of AI often enters before a model generates its first answer.
The now-classic literature on dataset bias demonstrated that training corpora can reproduce historical discrimination and cultural asymmetries. More recent work has shown that benchmark-driven development can encourage systems that perform well on narrow tests while masking brittleness in real-world settings. This matters for alignment because a system cannot be aligned to human values if its underlying representations systematically distort whose humanity counts.
The problem is not only bias in the pejorative sense. It is also omission. Many communities most affected by automated systems are under-represented in data collection, in annotation processes and in evaluation design. As a result, harms emerge not because designers explicitly sought them, but because the social world was simplified to fit what could easily be measured.
What looks like model misbehaviour is often a mirror of upstream choices about data, labels and benchmarks that were treated as neutral when they never were.
For that reason, serious alignment work has to move beyond model tuning towards documentation, provenance, impact assessment and participatory evaluation. The model is only the visible tip of a much larger epistemic structure.
Human oversight is necessary, but not automatically meaningful
Calls for human-in-the-loop systems have become a standard response to alignment concerns. In principle, this is sensible. Human review can catch errors, contextualise outputs and provide a moral backstop where automation is brittle. In practice, however, oversight can become ritual rather than reality.
Research in human factors has long shown that operators are prone to automation bias: they defer to machine recommendations, especially under time pressure or when systems appear authoritative. In high-volume environments, nominal reviewers may function as rubber stamps. If they lack the information, training or institutional authority to challenge a system, then “human oversight” becomes a compliance phrase rather than a real safeguard.
This is especially acute in public administration and frontline services, where staff may inherit AI systems procured elsewhere and be judged on throughput rather than deliberation. A human may remain formally responsible for the final decision, yet have little practical scope to inspect the model’s assumptions or contest its outputs. Accountability then fragments: the operator blames the system, the supplier blames the user, and the institution blames neither.
Meaningful oversight therefore requires more than retaining a person in the loop. It requires redesigning workflows, clarifying liability, improving contestability and ensuring that human reviewers can genuinely intervene without prohibitive costs. Otherwise, alignment is delegated to organisational theatre.
Interpretability helps, but explanation is not exoneration
What looks like model misbehaviour is often a mirror of upstream choices about data, labels and benchmarks that were treated as neutral when they never were.
Another common response to alignment concerns is interpretability. If systems can explain their reasoning, perhaps users can verify whether they are acting in line with human values. There is truth in this. Better interpretability can support auditing, debugging and trust calibration. It can reveal spurious correlations, unsafe strategies or hidden failure modes that remain invisible in black-box deployments.
Yet explanation has limits. Some methods illuminate local patterns without establishing causal understanding. Some generated explanations are plausible narratives rather than faithful accounts of internal computation. And even a clear explanation does not settle whether the underlying decision rule is justifiable. A perfectly interpretable model can still be used for objectionable ends.
This matters because the politics of AI often turns on the relationship between intelligibility and legitimacy. A public body may be able to explain why a system flagged a claimant or ranked an application, yet still fail to justify the values built into the ranking process. Explanation can support scrutiny, but it does not substitute for normative debate. To know how a system works is not necessarily to know whether it ought to work that way.
The most useful role for interpretability may therefore be modest: not as a silver bullet, but as one instrument among many for making systems governable. It should enable challenge, not foreclose it.
The frontier risk debate should not eclipse present harms
Debates over highly capable future systems have sharpened attention on catastrophic risk, loss of control and the possibility that models may pursue goals misaligned with human survival. These concerns have attracted substantial academic and policy interest, and not without reason. As model capabilities advance, uncertainty about general-purpose systems deserves careful study.
Still, there is a risk that long-term alignment discourse crowds out immediate governance failures already visible in deployed systems. Labour displacement without adjustment mechanisms, discriminatory decision support, manipulative recommender systems, privacy erosion and concentration of informational power are not speculative. They are present-tense alignment failures because they show institutions using AI in ways that diverge from publicly defensible social aims.
This is not an argument against frontier safety research. It is an argument for a wider aperture. The same habits that make institutions neglect current harms may also make them unfit to manage future ones. A polity that cannot govern procurement, auditing, appeals and redress for today’s systems is unlikely to handle more capable systems well tomorrow.
A society that cannot align the institutions deploying today’s AI will struggle to align the more powerful systems of tomorrow.
The practical conclusion is that near-term governance and long-term safety are complements. Both require humility about uncertainty, stronger evaluative institutions and a willingness to slow deployment where safeguards are weak.
Democratic legitimacy is becoming the core test
If alignment now reaches beyond technical robustness, what should replace the older, narrower frame? One answer is democratic legitimacy. Not in the thin sense of periodic political approval, but in the richer sense that systems affecting people’s life chances should be shaped by processes that are contestable, transparent and responsive to those affected.
This shifts attention from whether a model mirrors abstract human preferences to whether the institutions around it can justify and revise the values they encode. Public consultation, independent auditing, rights of explanation, sector-specific regulation and avenues for appeal all matter here. So does the distribution of expertise. If only a small technical elite can interpret or challenge a system, formal accountability will remain shallow.
A society that cannot align the institutions deploying today’s AI will struggle to align the more powerful systems of tomorrow.
The most promising policy frameworks now recognise this. UNESCO’s ethics recommendation emphasises impact assessment and public participation. The Council of Europe’s Framework Convention on Artificial Intelligence, Human Rights, Democracy and the Rule of Law situates AI squarely within constitutional principles. The emerging consensus is that alignment cannot be left to private optimisation criteria or voluntary codes alone.
Democratic legitimacy is messy. It introduces friction, delay and disagreement. But those are features, not defects, when societies are deciding how much discretion to hand to machines.
What serious alignment practice would look like
If alignment is partly a governance problem, then serious practice must begin earlier and extend further than model training. First, institutions need clear use-case justification. Not every process should be automated merely because automation is possible. The threshold question is whether an AI system is appropriate at all, given the stakes, reversibility of errors and availability of less intrusive alternatives.
Secondly, impact assessment should be continuous rather than ceremonial. Pre-deployment testing matters, but post-deployment monitoring matters more because social effects emerge in context. Systems should be evaluated not only for accuracy, but for disparate impact, rates of appeal, user comprehension and downstream behavioural change.
Thirdly, documentation needs to be operational. Model cards, data sheets and audit logs are useful only if they inform procurement choices, regulatory review and internal accountability. Paper trails without enforcement create the appearance of diligence without its substance.
Fourthly, affected groups should be involved in evaluation, especially where systems shape access to essential services. Participation will not resolve every conflict, but it can expose harms invisible to designers and help institutions understand which trade-offs are politically and ethically acceptable.
Finally, redress must be real. People need accessible ways to challenge, correct and appeal AI-assisted decisions. Without recourse, alignment remains a claim made by system builders rather than a condition experienced by those subject to the system.
The strategic question beneath the technical one
At bottom, the alignment debate is about collective agency. Advanced AI systems compress judgment into scalable infrastructures. They can encode priorities, standardise discretion and redistribute power across institutions at remarkable speed. That creates efficiencies. It also raises a strategic question that no amount of parameter tuning can answer on its own: who should wield that power, according to which norms, and under what checks?
The answer will not come from a single technical breakthrough or one universal charter. It will emerge from a mixture of engineering discipline, legal constraint, public scrutiny and institutional experimentation. Some sectors will demand tighter control than others. Some uses will prove too harmful or too opaque to justify. Others may become acceptable only once mechanisms for audit and appeal are robust.
That is why the next phase of alignment work should resist two temptations. The first is solutionism: the belief that a sufficiently clever technical method can dissolve political disagreement. The second is fatalism: the belief that because values conflict, governance is futile. Between those poles lies the more difficult but more realistic task of building institutions capable of steering AI under conditions of uncertainty and pluralism.
In that sense, alignment is maturing. It is no longer merely a question of making systems follow instructions. It is becoming the harder, older and more important question of whether the people and organisations giving those instructions deserve to be trusted.
