From model behaviour to institutional behaviour
For years, discussion of AI alignment centred on an engineering ambition: ensuring that advanced systems do what their designers and users intend. That framing remains important. Large language models can hallucinate, reinforce social bias, reveal unsafe instructions and behave unpredictably when prompted in unfamiliar ways. But as these systems are incorporated into education, healthcare, employment, security and public administration, a narrower focus on model-level fixes is proving insufficient.
The emerging evidence points to a broader reality. Alignment is not simply a matter of training objectives or safety layers. It is also about governance: who sets the objectives; what risks are considered tolerable; how harms are measured; and what recourse exists when an automated output causes damage. Technical alignment and ethical alignment are therefore intertwined, but not identical. A system can be statistically improved and still remain politically or morally misaligned with the community it affects.
Alignment is no longer just a question of whether a model follows instructions; it is a question of whose instructions count, and under what oversight.
This shift matters because institutions tend to adopt tools faster than they redesign accountability around them. The result is a widening gap between capability and governance. In that gap sit many of the most pressing ethical risks.
What the current evidence actually shows
The policy conversation has matured because the empirical base has broadened. The National Institute of Standards and Technology's AI Risk Management Framework describes AI risk as partly stemming from system performance and partly from social context, deployment conditions and downstream use. In other words, the same model may be acceptable in one setting and harmful in another, depending on the stakes, affected populations and available safeguards.
Research from the Stanford Institute for Human-Centered Artificial Intelligence has similarly shown that benchmarks for model capability are improving faster than reliable methods for evaluating societal impact. The annual AI Index reports rising deployment, growing investment in responsible AI practices, and persistent concern about fairness, transparency and accountability. Yet standardised reporting on incidents, post-deployment monitoring and external auditing remains uneven.
The OECD's work on trustworthy AI points in the same direction. Robustness, safety and fairness are presented not as stand-alone technical properties but as governance goals requiring risk management, documentation and human accountability. This is important because it reframes ethics from a voluntary aspiration into an operational discipline.
Even the most sophisticated evaluation regimes remain incomplete. Benchmarking can reveal known failure modes, but alignment failures often emerge in live settings, where users improvise, incentives shift and models are integrated with databases, workflows and authority structures. Ethical risk is therefore cumulative: it resides in the full sociotechnical system, not only in the model.
Why bias remains the clearest signal of misalignment
Among the many ethical concerns surrounding AI, bias remains the most concrete and measurable indicator that alignment is incomplete. The reason is straightforward. If an AI system systematically produces worse outcomes for certain groups, it is revealing a mismatch between optimisation and social expectation. That mismatch may originate in skewed data, flawed proxies, inadequate testing or institutional indifference. But the result is the same: the system is not aligned with widely accepted norms of fairness.
The landmark study by Joy Buolamwini and Timnit Gebru on gender shades demonstrated substantial performance disparities in commercial gender classification systems across skin tone and gender groups. Although that work examined a specific class of computer vision models, its broader lesson endures. Aggregate performance can conceal unequal error distribution, and unequal error distribution becomes ethically significant when systems shape opportunities or burdens.
Alignment is no longer just a question of whether a model follows instructions; it is a question of whose instructions count, and under what oversight.
Subsequent scholarship from the AI Now Institute and academic researchers has shown how automated systems can reproduce structural inequalities in hiring, welfare administration, policing and credit. What makes these cases particularly relevant to alignment is that the systems were often functioning as designed. The failure was not random malfunction but the faithful execution of objectives and datasets that embedded contested assumptions.
Many harmful AI outcomes are not accidents in the ordinary sense; they are the predictable result of systems optimised for the wrong proxy.
This is why fairness cannot be treated as a final-stage compliance check. It must inform problem formulation itself. Before asking whether a model is accurate, institutions must ask whether the target variable is legitimate, whether the training data reflect historical discrimination, and whether affected people can challenge an outcome. These are governance questions before they are modelling questions.
The limits of transparency on its own
Transparency has become a near-universal prescription in AI ethics. Documentation, explainability and disclosure are all useful. Model cards, system cards and incident reporting can improve scrutiny. The European Union's AI Act and guidance from international bodies place considerable weight on information duties and traceability. But transparency on its own does not resolve alignment problems.
There are at least three reasons. First, disclosure can be too technical to enable meaningful accountability. Publishing system documentation is not the same as making a decision understandable to the person affected by it. Secondly, transparency can reveal that a system is risky without providing the authority or incentives to change it. Thirdly, some of the most consequential aspects of alignment concern values and trade-offs, which cannot be settled simply by exposing internal mechanisms.
The Royal Society has argued that explainability should be matched to context, purpose and audience. That is a sensible corrective. In a low-stakes consumer setting, basic disclosure may suffice. In a high-stakes public-sector setting, however, stronger standards are needed: contestability, independent review and evidence that the system performs adequately across relevant populations.
Put differently, transparency is a governance input, not a governance substitute. Without enforcement, institutional responsibility and pathways for redress, it risks becoming a procedural comfort blanket.
Human oversight is necessary but often overstated
Many policy frameworks rely on the idea of human-in-the-loop oversight. The intuition is appealing: if machines are fallible, a person should remain responsible. Yet research in human factors and automation has long shown that oversight can be nominal rather than effective. When humans monitor complex automated systems, they may become complacent, deferential or unable to intervene meaningfully in real time.
This is especially relevant for generative AI. Outputs can appear fluent and plausible even when they are false or inconsistent. In fast-paced organisational settings, the human reviewer may lack the time, expertise or authority to detect subtle errors. Oversight then becomes ceremonial. Responsibility is retained on paper while discretion is hollowed out in practice.
Guidance from NIST and the OECD increasingly reflects this concern by stressing that human oversight must be appropriately designed, not merely asserted. Effective oversight requires training, clear escalation procedures, usable interfaces and organisational cultures that reward challenge rather than passive acceptance. It may also require limits on deployment in settings where meaningful review is unrealistic.
Ethically, this matters because human oversight is often presented as the bridge between technical systems and democratic values. If that bridge is weak, institutions may overestimate how aligned a system is simply because a person can, in theory, intervene.
Safety evaluation is improving, but assurance remains patchy
Many harmful AI outcomes are not accidents in the ordinary sense; they are the predictable result of systems optimised for the wrong proxy.
One reason the alignment debate has intensified is that evaluation methods are becoming more rigorous while still revealing major gaps. Organisations now conduct red-teaming, adversarial testing and capability evaluations for dangerous behaviour. Researchers study whether models can be induced to provide harmful instructions, whether they resist jailbreaks, and whether they behave consistently across scenarios. These are valuable advances.
Yet assurance remains patchy for several reasons. Evaluations are often non-standardised, difficult to compare and not always independently replicable. The pace of model iteration can outstrip the pace of auditing. Some tests focus on model behaviour in isolation rather than behaviour once embedded in a product or workflow. And many evaluations remain confidential, limiting public scrutiny.
The UK's frontier AI safety reports and work by leading public research institutions have underscored deep uncertainty around model capabilities and emergent risks. Importantly, uncertainty itself is an ethical issue. When institutions deploy systems whose failure modes are not well characterised, they are making a judgement about acceptable public risk. That judgement should not be hidden behind technical opacity.
Alignment, in this sense, depends not just on reducing risk but on being candid about uncertainty. Where evidence is incomplete, governance should become more cautious, not more permissive.
The regulatory mood is shifting from principles to duties
For much of the past decade, AI ethics was dominated by high-level principles: fairness, accountability, transparency, safety, privacy. These principles remain influential, but regulators are moving towards more concrete duties. The European Union's AI Act adopts a risk-based approach with obligations linked to use case, including requirements around data governance, documentation, human oversight and post-market monitoring for higher-risk systems.
Elsewhere, the White House Blueprint for an AI Bill of Rights, while not binding law, articulates expectations around safe and effective systems, algorithmic discrimination protections, data privacy, notice and explanation, and human alternatives. UNESCO's Recommendation on the Ethics of Artificial Intelligence similarly anchors ethical AI in human rights, environmental stewardship and governance mechanisms.
The ethical centre of gravity is shifting from voluntary principles to enforceable duties, because principles without institutions rarely constrain power.
This regulatory evolution reflects a practical lesson: values become meaningful only when translated into procedures, accountability and sanctions. It also signals that alignment cannot be left solely to developers or procuring organisations. Public authority is asserting a role in defining minimum standards for systems that shape social outcomes.
That does not guarantee effectiveness. Rules can be under-enforced, gamed or outpaced by technical change. But the movement from abstract ethics to operational obligations marks a significant turning point.
Why public-sector deployment deserves special scrutiny
AI used by public authorities raises sharper alignment concerns than many commercial applications because the state exercises coercive or quasi-coercive power. Decisions about benefits, migration, policing, education and health access carry asymmetrical consequences. Individuals may have little choice but to submit to automated or semi-automated processes, and errors can be difficult to contest.
Reports from civil-society groups, public auditors and academic researchers have repeatedly shown that public-sector algorithmic systems can suffer from weak procurement standards, inadequate transparency and limited avenues for appeal. The issue is not that public bodies are uniquely careless. Rather, they operate under political, budgetary and administrative pressures that can make automation attractive before governance is mature.
The ethical threshold should therefore be higher. Public-sector AI should be subject to stronger evidentiary requirements, meaningful impact assessments and independent oversight. If a system materially affects rights or entitlements, the presumption should be that contestability and auditability are indispensable.
The ethical centre of gravity is shifting from voluntary principles to enforceable duties, because principles without institutions rarely constrain power.
Seen through the lens of alignment, the question is not merely whether public systems work. It is whether they embody administrative justice. Accuracy matters, but legitimacy matters more.
Alignment also concerns labour and knowledge
Another underappreciated ethical dimension of alignment lies in labour and epistemology: who produces the data, who evaluates the outputs, and whose knowledge is treated as authoritative. Training and moderation pipelines often rely on dispersed forms of human labour that are poorly visible in public debate. At the same time, deployment can alter professional judgement in fields such as law, journalism, medicine and education by shifting what counts as a plausible answer or acceptable evidence.
This matters because alignment is often discussed as if it were a direct relationship between a model and a user. In reality, it is mediated by workers, institutions and professional norms. If annotators are under-supported, if domain experts are excluded from design, or if organisational incentives prioritise speed over deliberation, the resulting system may be formally compliant yet substantively misaligned.
Scholars at institutes focused on AI and society have warned that generative systems can reshape information environments by flooding them with plausible synthetic content. In such environments, truthfulness is not only a model property; it is a public-good problem. Alignment therefore includes preserving the conditions under which trustworthy knowledge can be produced and verified.
What better alignment would look like in practice
If alignment is increasingly a governance problem, then improvement requires more than better fine-tuning. First, institutions need robust pre-deployment impact assessments that test not only accuracy and safety but distributional effects, contestability and appropriateness for the use case. Secondly, high-stakes systems should face independent auditing and ongoing monitoring rather than one-off approval. Thirdly, documentation should be intelligible to affected communities and regulators, not only to technical teams.
Fourthly, procurement and deployment should be tied to clear accountability. A named authority should be responsible for outcomes, incident response and withdrawal if harms emerge. Fifthly, users and affected individuals need credible channels for complaint, appeal and human review. Sixthly, governments and standards bodies should invest in shared evaluation infrastructure, so that safety and fairness claims become more comparable across systems and sectors.
None of this eliminates the need for technical advances in robustness, interpretability or controllability. On the contrary, governance depends on them. But technical advances become ethically meaningful only when embedded in institutions capable of setting limits and bearing responsibility.
The deeper implication is that alignment should be treated less as a final engineering milestone and more as a continuing public obligation. Systems evolve, contexts change and social expectations shift. Governance must therefore be iterative, evidence-based and open to revision.
A narrower machine problem, a wider political one
It is tempting to imagine that alignment will be solved by sufficiently capable tools for supervision, red-teaming and optimisation. Those tools will certainly help. But the evidence now suggests that the enduring difficulty is not simply making AI systems better behaved. It is deciding how much authority to grant them, in which domains, under what conditions, and with whose consent.
That is why the alignment debate is widening. It touches fairness because systems distribute opportunity and error. It touches legitimacy because institutions delegate judgement through technical means. It touches democracy because public values are being translated into operational rules, often without clear public deliberation.
The most important ethical question, then, may not be whether AI can be aligned in the abstract. It is whether the institutions deploying AI are themselves equipped to act in aligned ways: transparent where needed, cautious under uncertainty, accountable for harm and willing to limit automation when the case for it is weak. The future of alignment may depend less on perfect models than on imperfect but serious governance.
