# The Illusion of Neutrality: Why No Algorithm Is Objective
There is a particular kind of innocence that modernity reserves for its machines. We assume that what a human being decides may be prejudiced, but what an algorithm computes must be fair, because mathematics has no memory of injustice. This assumption is the quiet theology of the digital age, and it is false. In his book ALGORITHMUS. Wer die KI kontrolliert kontrolliert die Zukunft, Dr. Raphael Nagel (LL.M.) describes the illusion of algorithmic neutrality as one of the most consequential misunderstandings of our time. It is not a technical error that can be patched with a software update. It is a cultural habit of delegating responsibility to systems that inherit our history, translate it into statistical form, and return it to us as prophecy. The essay that follows reconstructs this argument in the spirit of his book: through the documented cases, the regulatory architecture now emerging in Europe, and the consequences for any enterprise that believes it can outsource judgement to code.
## The Amazon Case, or the Industrialisation of Prejudice
In 2018, engineers at Amazon discovered that an internal recruiting system, designed to automate the screening of applications, was systematically disadvantaging women. The system had been trained on the company's historical hiring data and had learned, with mathematical precision, what the human decision makers of previous years had preferred. Because the technology sector and Amazon's own hiring history had been overwhelmingly male, the model treated signals such as attendance at a women's college or the captaincy of a women's sports team as negative indicators. It favoured stylistic patterns statistically more common in applications written by men. The system was discontinued before it reached operational deployment.
What is remarkable about this episode is not the bias itself, which was predictable, but the scale at which it would have operated. A human recruiter prejudices one decision at a time, and can be questioned, corrected, overruled. An algorithm prejudices every decision, simultaneously, silently, and with the appearance of rigour. Dr. Raphael Nagel (LL.M.) argues in his book that this is the deeper danger of algorithmic systems: they do not invent new forms of discrimination. They industrialise the old ones, and they dress the product in the uniform of objectivity.
The lesson of Amazon is therefore not that a particular model failed. The lesson is that any model trained on historical data will, in the absence of deliberate correction, reproduce the power structures embedded in that history. The data are not neutral. They are a sedimentation of past decisions, and past decisions were made by human beings with their own horizons, interests and blind spots.
## NIST and the Arithmetic of Unequal Error
The second piece of evidence belongs to a different register. In 2019, the National Institute of Standards and Technology in the United States published a study of 189 commercially deployed facial recognition systems. The finding was unambiguous. For women with darker skin tones, the error rate was up to one hundred times higher than for men with lighter skin tones. Not twice as high. Not five times. One hundred times. This is not statistical noise. It is the arithmetic signature of training sets overrepresenting certain faces and underrepresenting others, built by teams whose demographic homogeneity mirrored the markets they first imagined serving.
Despite the publication of this evidence, several of these systems continued to be used by law enforcement agencies in more than thirty American states. In 2020, Robert Williams was arrested on the basis of a false facial recognition match and held for thirty hours before the investigators themselves acknowledged the error. The Federal Bureau of Investigation later documented that erroneous facial recognition had contributed to a series of wrongful arrests. The technology was known to fail disproportionately. It was deployed anyway.
Here the question is no longer whether algorithms can be biased. The question is why institutions continue to entrust them with consequential decisions after the bias has been measured. The answer, Dr. Nagel suggests in ALGORITHMUS, lies less in technical ignorance than in a political convenience. An algorithm that errs unequally, but confidently, relieves the institution of the burden of justifying its errors. The machine has spoken, and the machine is presumed impartial.
## Proxy Discrimination, or the Return of the Forbidden Variable
The most difficult form of algorithmic bias is neither the crudely biased training set nor the mislabelled image. It is proxy discrimination, the quiet mechanism by which variables that are legally permitted come to stand in for variables that are legally forbidden. In the United States, it is illegal to base credit decisions explicitly on race, gender or national origin. It is not illegal to base them on postal code, purchasing patterns, educational institutions attended, or the structure of one's social network. Yet each of these variables correlates, in societies shaped by decades of redlining, segregated schooling and stratified employment, with precisely the categories that the law has excluded.
The model is not lying when it reports that a particular postal code has historically exhibited higher default rates. It is telling a statistical truth about the past. The moral and legal question is whether that truth ought to govern the future, and whether its use as a predictor perpetuates the very structural disadvantage that produced the correlation in the first place. This is the juridical hinge on which the European AI Act now turns. The regulation treats credit scoring, employment decisions, access to essential services, law enforcement applications and critical infrastructure as high risk domains, subjecting them to documentation duties, transparency obligations and audit requirements, with penalties of up to three percent of global annual turnover.
A proxy is not an accident. It is the shape that prohibited variables take when they are forced underground. Any enterprise deploying predictive systems in regulated domains must therefore assume that the absence of a protected category from the feature set does not, by itself, produce a lawful outcome. The law, in its European form, has begun to read through the proxy and to hold the operator responsible for what the model reconstructs.
## The Rhetoric of Objectivity as an Immunising Device
The most strategically dangerous element of the entire complex is not any specific bias. It is the rhetorical function that the word algorithmic performs in public discourse. When a decision is described as algorithmic, it acquires an aura of the technical, the impartial, the beyond-politics. It appears to stand outside the arena in which legitimate criticism is possible. A 2022 experimental study at Cornell University found that identical decisions, presented once as human and once as algorithmic, were accepted more readily in their algorithmic guise. The result was the same. The authority attributed to it was not.
This is what Dr. Raphael Nagel (LL.M.) calls the immunising function of objectivity rhetoric. To name a process algorithmic is to remove it, in the public imagination, from the sphere of contestable judgement and to relocate it in the sphere of mathematical necessity. The effect is political even when the intention is not. Institutions that once had to justify their decisions in natural language can now present them as outputs, and outputs do not require argument. They require only interfaces.
To dismantle this illusion is not to reject algorithmic systems. It is to insist that every such system carries within it a set of choices that were made by human beings: which data to collect, which features to encode, which objective function to optimise, which errors to tolerate, which populations to test on. Each of these choices is a normative act. The sum of them is a worldview. The worldview is not revealed by the interface. It is obscured by it.
## Regulatory, Reputational and Economic Consequences
For European enterprises, the consequences of taking algorithmic bias seriously unfold along three axes. The regulatory axis is the most explicit. The AI Act imposes, for high risk systems, obligations of bias testing, documentation of mitigation measures, and periodic auditing. The scale of the penalties, calibrated against global turnover, signals that the European legislator intends to treat algorithmic discrimination as a category of corporate liability rather than as a matter of voluntary ethics. In parallel, the American Federal Trade Commission has already pursued several companies for algorithmic discrimination, with settlements in the millions.
The reputational axis is less codified but not less material. Every publicly documented case of algorithmic discrimination, from recruiting tools to facial recognition deployments to insurance pricing models, has produced reputational damage that persisted long after the technical problem was addressed. The memory of the market, unlike the memory of the model, is not retrained on request. The economic axis is the one most often overlooked. A system that systematically misclassifies candidates, customers or risks is not merely unfair. It is operationally suboptimal. It mistakes the distribution it was meant to describe, and its errors cost money before they cost lawsuits.
The practical implication is straightforward. Preventive bias auditing, conducted before deployment, is a fraction of the cost of reactive crisis management after a public incident. Yet most enterprises continue to act only once the problem has become visible. This is not an economic calculation. It is an organisational reflex, and it is the reflex that the next decade of European regulation is designed to correct.
The question that remains, after the cases have been counted and the regulations cited, is philosophical rather than technical. Objectivity has never been a property of instruments. It has been, at best, a human aspiration, pursued through procedure, argument, review and the patient discipline of self-correction. To attribute objectivity to a machine is to forget this intellectual history and to mistake the absence of visible deliberation for the presence of genuine impartiality. Dr. Raphael Nagel (LL.M.) writes in ALGORITHMUS that whoever confuses the two delegates discrimination to the machine and calls it progress. The sentence is severe because the situation is severe. An algorithm is a compressed record of the choices made by those who built it, trained it and deployed it. Its outputs are the continuation of those choices by other means. The European response, embodied in the AI Act, begins from this recognition. It does not demand that machines be neutral, which they cannot be. It demands that the human beings responsible for them remain accountable, which they must be. For enterprises in Germany and across Europe, this is the horizon of the coming years. The competitive advantage will not belong to those who deploy the most systems, but to those who deploy them with the clearest understanding of what such systems are: instruments of judgement, built by people, operating within a legal and moral order that has decided, finally, to look through the interface and to ask who stands behind it.
For weekly analysis on capital, leadership and geopolitics: follow Dr. Raphael Nagel (LL.M.) on LinkedIn →