What is AI Ethics Dilemma? — one analogy

1The analogy

Snow fell all night, and there's exactly one plow. Someone has to decide where it starts, and three people each argue for a different answer.

One says clear the main road first, since that gets the most people moving the fastest. Another says go to the neighborhood that got plowed last, last time, since someone shouldn't always end up at the back of the line. A third says take the steep hill first, since that's where a fall does the most damage.

All three have a reason, and none of them is acting in bad faith. But there's only one plow, so an order has to be picked, and arguing about it on a snowy morning is already too late. An AI ethics dilemma looks like this scene. Good reasons collide with each other, and the answer has to be written down ahead of time.

2In detail

Why people split

Each of the three arguments comes from a different yardstick. The first is the size of the outcome: pick whichever choice produces more good, added up across everyone. It's easy to compute and persuasive, but a weakness hides inside the sum, since harm concentrated on a few people can get buried in the total.

The second is a line that shouldn't be crossed regardless of outcome: not treating people as means to an end, not using someone's information without consent. Principles like these don't bend, but when two lines collide with each other, the argument stalls.

The third is a fair share: who benefits, who bears the cost, and whether the same group keeps ending up at the back. It rarely shows up in a single decision and only becomes visible after many decisions pile up. Most disagreements come down to which yardstick gets put first, and rarely to one side meaning harm.

Deciding in the moment versus writing it down ahead of time

A dilemma a person runs into is usually settled on the spot: they read the situation, hesitate, choose, sometimes regret it later. Judgment shifting a little each time isn't seen as a flaw.

AI doesn't get that option. What to prioritize has to be set in advance, and if it isn't, the system drifts along whatever pattern sat in the training data. Not deciding is itself a decision.

A standard written down ahead of time has a real upside: what got chosen is visible from the outside, can be questioned, and can be revised. A judgment made in the moment is hard to trace back later, but a written policy stays put. Putting a dilemma into writing feels awkward, but it's what makes the argument possible in the first place.

One choice, repeated millions of times

Line up a hundred people to judge the same situation and the answers scatter. Some lean one way, some the other, and that scatter keeps any single direction from taking over.

A machine works differently. Once a standard is set, it applies identically to every case. If that standard leans slightly in one direction, the lean repeats identically, millions of times over. One person's bias is one person's problem; a system's bias becomes a problem at scale.

So the ethical question for AI isn't only "is this choice right." It's also "is it right to repeat this choice at this scale." A call that would be fine once can turn into something else entirely when it runs a million times.

What can be decided instead of the answer

Getting everyone to agree on what's right is hard. What can still be decided: who sets the standard, whether the people affected by it get a voice, and whether the choice gets disclosed to the outside.

Two more things belong here: what process changes the standard once it's shown to be wrong, and where exceptions the standard doesn't handle well get routed. A dilemma can't be erased, but a way back from a bad call can still be built in.

3More precisely

The three yardsticks map onto long-standing views in ethics: weighing outcomes, weighing duties and rights, and weighing fair process and distribution. None has been settled as beating the other two. Real-world judgment usually blends all three, and disagreement tends to be about which gets more weight.

The analogy breaks down in one place: a plow route has a countable number of options, and there's time to sit down and decide before the snow falls. What AI runs into branches out endlessly, so every case can't be written down in advance. A written standard always eventually meets an exception, and how that exception gets handled becomes another standard in its own right.

One more gap: a standard shaped by training isn't written in a document, it's woven into the data and the training process itself. Often even the people who built it can't fully say what standard went in, since it emerged from millions of examples rather than a rule anyone sat down and wrote. Because of that, handling a dilemma starts less with picking an answer and more with surfacing what went in to begin with.

4Try it yourself

5Common misconceptions

It's easy to think that with no right answer, any answer will do, but actually a clearly bad answer does exist, and the disagreement sits among the good answers.
It's easy to think whatever most people choose is the right answer, but actually a majority vote is only a reference point, and it shifts a great deal across regions and cultures.
It's easy to think better technology makes the dilemma go away, but actually it only shrinks the ambiguous cases; the job of choosing what to prioritize stays exactly where it was.

7One-line summary

In shortAn AI ethics dilemma is a place where good reasons collide and no single answer emerges, so disclosing what got chosen and building in a way to reverse it matter just as much as the choice itself.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02