Ethics × Second-Order Cybernetics × Artificial Intelligence

How do we remain non-trivial in the age of AI?

A field guide to the ethical relationship between humans and algorithms—based on Dietmar Koering’s 2023 paper in Enacting Cybernetics.

Work 02 · Research Paper · 2023Enacting Cybernetics 1(1), Article 5DOI 10.58695/ec.4 ↗

Explore the questions Read the article ↗

Research mapping sketch for Human–AI Nexus: ethics, agency, second-order cybernetics and artificial intelligence.

High-res ↗

00 / Position

The ethical question is not only what AI does to humans, but what humans become when algorithms predict, frame and increasingly structure their choices.

Die ethische Frage lautet nicht nur, was KI mit Menschen macht, sondern auch, was aus Menschen wird, wenn Algorithmen ihre Entscheidungen vorhersagen, rahmen und zunehmend strukturieren.

01—09 / Ethical Questions

Nine entry points into the human–AI nexus

The paper stages a “friendly dispute” between second-order cybernetical ethical thinking and AI ethics, drawing especially on Bernard Stiegler, Heinz von Foerster and Yuval Harari.

01HUMAN–AI NEXUS

What relationship should we examine: human–machine or human–algorithm?

The paper shifts attention from machines themselves to the algorithms people live by and the values these relationships reshape.

02SECOND-ORDER CYBERNETICS

Why does AI ethics begin with self-reflection?

The observer is inside the system; ethical responsibility therefore cannot be outsourced to technology.

03FREE WILL + AGENCY

What happens when algorithms know us better than we know ourselves?

Prediction and profiling can narrow autonomy when decisions become opaque, pre-emptive and difficult to contest.

04TRIVIAL MACHINES

How can AI lead humans to become trivial machines?

When reflection and choice disappear, behaviour risks becoming predictable input–output rather than open-ended action.

05ETHICS BY DESIGN

Can ethical values be embedded in algorithms?

Ethical regulators and ethical black boxes promise safeguards—but immediately raise the question of whose values are encoded.

06TRUST + TRANSPARENCY

What does trust in AI require?

People need accountable systems whose decisions can be inspected, challenged and corrected.

07VALUES + CULTURE

Can there be a universal ethical code for AI?

Common human values are necessary, yet cultures differ on happiness, equality, justice and the good life.

08WORK + PARTICIPATION

What is the human role in an increasingly automated society?

Automation raises questions of unemployment, education and how to retain meaningful human participation.

09HUMANISTIC FUTURE

What kind of AI future should we actively shape?

The paper argues for a humanistic path in which AI expands responsible action instead of replacing it.

What is the Human–AI Nexus?

The central issue is not simply the relation between a human and a machine, but the algorithms humans live by—and how those algorithms alter humanistic values.

From objects to relations

The paper treats technology as part of human development rather than an external force. Drawing on Stiegler, technological environments preserve knowledge, structure memory and participate in the ongoing adaptation between people and their surroundings. AI therefore becomes an epistemological and social condition.

Ethics across the whole lifecycle

Ethical reflection must address more than the final output of an AI system. It includes how systems are developed, deployed and used, which data they depend on, who defines their goals and how their effects return to society.

Kurzfassung: Entscheidend ist nicht nur die Mensch-Maschine-Beziehung, sondern die Beziehung zwischen Menschen und den Algorithmen, nach denen sie leben.

Source context: Abstract; Introduction.

Why second-order cybernetics?

Second-order cybernetics turns ethics back toward the observer: the person judging an AI system is already part of the social and technological system being judged.

Self-reflection before prescription

Foerster’s approach emphasises responsibility, communication and context rather than a fixed external checklist. The ethical act begins with awareness of one’s own position, assumptions and consequences.

The ethical imperative

“Act always so as to increase the number of choices” becomes a way to test whether technological systems preserve viable alternatives and meaningful action. The paper also discusses the criticism that more options alone are not necessarily better; what matters are thoughtful alternatives that enable effective action.

Kurzfassung: Ethik beginnt bei der Selbstreflexion des Beobachters. Technologie soll sinnvolle Handlungsmöglichkeiten erweitern, nicht bloß Optionen vermehren.

Source context: Section 2.2; Foerster, Ashby and Pangaro discussion.

What happens to free will and agency?

Algorithmic prediction becomes ethically critical when people no longer know how they are classified, cannot contest the result or gradually surrender authority over their own choices.

Prediction is not neutrality

The paper uses examples such as COMPAS and Cambridge Analytica to show how data-driven systems can reproduce bias, manipulate behaviour and create opaque categories with real consequences. Mathematical form does not make an algorithm inherently fair.

Agency as a civic capacity

The risk is not only external control. Humans also supply the data, accept conveniences and participate in the systems that profile them. This makes self-responsibility and critical participation central ethical questions.

Kurzfassung: Agency geht verloren, wenn algorithmische Entscheidungen und Profile undurchsichtig, nicht anfechtbar und handlungsbestimmend werden.

Source context: Section 2.1 — The Understanding of AI and Free Will.

How do humans become “trivial”?

A trivial machine produces a predictable output from a given input. Applied to humans, the metaphor warns against reducing judgement, emotion and reflection to deterministic patterns.

Predictability as an ethical danger

If algorithms increasingly preselect what people see, choose and do, human action can become more predictable and less deliberative. The paper warns that this can erode empathy, responsibility and the ability to create genuinely new knowledge.

Non-triviality as human potential

Foerster’s non-trivial machine stands for systems whose internal state matters and whose behaviour cannot be reduced to a fixed input–output rule. The paper associates human non-triviality with learning, unpredictability, wonder and the continuing possibility of surprise.

Kurzfassung: Menschen werden dann „trivial“, wenn Reflexion, Unvorhersehbarkeit und eigenständige Wahl zugunsten vorhersehbarer Reaktionen verschwinden.

Source context: Section 2.2 — How AI Can Lead People to Become Trivial Machines.

Can ethics be embedded in AI?

The paper supports ethical foundations for AI while exposing a paradox: translating ethics into computation inevitably requires decisions about rules, methods, priorities and values.

Ethical regulator vs. ethical method

Ashby’s ethical regulator proposes explicit ethical constraints; Misselhorn’s formulation opens another possibility—embedding a method that allows a system to test the admissibility of rules. The paper asks whether these two approaches can ever be cleanly separated.

The problem of value selection

Health may be comparatively easy to state as a goal, while happiness, equality and justice depend on culture, context and competing interpretations. Encoding ethics therefore cannot eliminate human responsibility for defining and revisiting values.

Kurzfassung: Ethik kann nicht einfach als neutrale Regelmenge „eingebaut“ werden. Schon die Auswahl und Gewichtung von Werten ist eine verantwortliche menschliche Entscheidung.

Source context: Sections 2.2 and 3; Ashby, Misselhorn and Hui discussion.

What builds trust in AI?

Trust requires more than performance. It depends on transparency, inspectability, accountability and the possibility of correcting harmful decisions.

The black-box problem

When people cannot understand why a system reached a conclusion, confidence erodes. The paper discusses Winfield and Jirotka’s “ethical black box” as a way to reconstruct decisions and assign responsibility after failures.

Standards before strong AI

The paper argues for beginning AI-safety and ethical standard-setting while humans still control the development process. Legal and institutional standards can help make responsibility explicit rather than leaving it implicit inside technical systems.

Kurzfassung: Vertrauen entsteht durch nachvollziehbare Entscheidungen, Verantwortlichkeit und die Möglichkeit, fehlerhafte Systeme zu prüfen und zu korrigieren.

Source context: Section 3.1 — Lack of Trust in Human’s Coexistence with AI.

Can AI have a universal ethical code?

The paper argues that shared ethical foundations are necessary, while also acknowledging that values are culturally situated and difficult to universalise.

Common ground, plural contexts

Truth, honesty, loyalty, love and peace are introduced as basic human values, but the paper repeatedly asks who defines concepts such as happiness, equality and justice. Ethical systems therefore need both a common humanistic baseline and sensitivity to different cultural norms.

Ethics cannot be outsourced

A universal code is presented as a goal for a positive AI future, aligned with broadly applicable human-rights principles. Yet implementation remains a recursive social process: guidelines must be debated, tested and revised rather than treated as a finished technical object.

Kurzfassung: Gemeinsame humanistische Grundwerte sind notwendig, müssen aber kulturelle Unterschiede und die eigene Perspektive derjenigen berücksichtigen, die sie formulieren.

Source context: Introduction; Sections 3.1–3.2; Conclusion.

What is the human role in automated society?

The question is not only which jobs AI removes, but how society can preserve or increase meaningful human participation as capabilities once considered uniquely human become automated.

Beyond replacement

The paper discusses unemployment, universal basic income and education as responses to automation. It notes that social and creative intelligence, once used to distinguish humans from machines, is already being challenged by generative systems for text and images.

Participation as an ethical objective

Education and anticipation may help people choose future roles, but the deeper issue remains societal: what work and participation should be protected or created so that AI benefits human beings rather than merely maximising efficiency?

Kurzfassung: Die zentrale Frage ist, wie menschliche Teilhabe erhalten und erweitert werden kann, wenn KI zunehmend kreative und wissensbasierte Tätigkeiten übernimmt.

Source context: Section 3.2; discussion of employment and participation.

What future should we shape?

The paper ultimately favours Foerster’s humanistic attitude: preserve responsibility, self-reflection and meaningful choice while developing AI on explicit ethical foundations.

AI as a shaping force

The conclusion reformulates the image of technology as a mirror: AI is described instead as a tool that actively shapes society. That makes passivity impossible; the values embedded in development today become part of tomorrow’s environment.

A positive future is an active project

Researchers, scientists, governments and citizens are called upon to act as responsible participants rather than spectators. The desired future is one in which AI assists humans in creating something new without trivialising the human capacity to deliberate, learn and choose.

Kurzfassung: Eine positive KI-Zukunft entsteht nicht automatisch. Sie muss durch Verantwortung, Selbstreflexion, ethische Standards und reale Wahlmöglichkeiten aktiv gestaltet werden.

Source context: Conclusion.

Ethical loop / Recursive by design

Reflect. Expose values. Preserve agency. Revisit.

Reflect from within

Recognise that developers, institutions, users and critics are participants in the systems they evaluate.

Expose the values

Make goals, assumptions, data choices and ethical priorities visible rather than hiding them behind technical neutrality.

Preserve agency

Design systems that maintain meaningful alternatives, contestability and human responsibility.

Return responsibility

Inspect consequences, learn from failures and let feedback revise both algorithms and the values guiding them.