Fairness

Bias is a fact, fairness is a judgment call

Bias is the observation that "this model's results lean one way." Measure it and a number comes out. Fairness is the question that follows: "is this much of a lean acceptable, and if not, which direction should it be corrected toward?" Measuring and deciding are connected, but they aren't the same job.

Blur that line and strange requests start showing up, like asking the tool that measures the lean to also "make it fair." A tool can tell you how far apart two groups are on some axis. Whether closing that gap is the right call is still something a person has to decide.

Put a thermometer in front of the fan and you'll learn exactly how many degrees each seat is. The thermometer won't tell you which seat deserves the breeze.

Fairness isn't just one definition

The first answer is matching outcome rates: fair means every group passes at roughly the same rate. The second looks at people who got the same verdict: fair means the actual outcome for people the model rated similarly should look similar across groups. Meaning: among everyone marked as a pass, the share who genuinely did well should be the same no matter the group.

The third answer looks at the direction of the mistakes: fair means the rate of wrongly rejecting someone who deserved to pass is even across groups. The fourth looks at individuals directly: similar people should get similar treatment.

All four sound reasonable, and each has people who back it. The trouble is that on the same data, these answers can point to different decisions.

Some of them can't all be satisfied at once

If the underlying rates already differ across groups, matching outcome rates guarantees the "same verdict, same result" measure will come apart. Fix that one instead, and the direction of the mistakes drifts apart next. Nailing one exactly forces another out of line, and it's a known result that satisfying all three at once is only possible in special cases, not as a rule.

So the real work isn't "make it fair." It's "choose what gets protected first." Where the cost of a wrongful rejection is high, minimizing missed cases usually comes first; where resources are being divided up, matching the rates usually comes first.

It's like the fan: rotating and staying fixed can't both happen at once. Not because there's no way to do it, but because a next step only becomes possible once something has been decided as more important.

Covering your eyes doesn't make it fair

The first idea that comes to mind is dropping the sensitive category entirely. But the remaining categories very often carry that same information anyway. Neighborhood, school, way of speaking, time of day someone logs in, all of these leak a little bit of the same signal. Drop the category and the model tends to land on a similar conclusion regardless.

Worse, dropping the category also removes any way to measure the lean. Checking whether one group is doing worse requires being able to tell that group apart in the first place. That's also the argument for keeping the information around under tight access, rather than deleting it, so it stays available for auditing.

Writing down what got chosen is part of the job

Fairness work is three steps: decide which axis to look across, measure how far apart things are on that axis, and choose which standard to prioritize, then write down that choice and why. Skip the last step and the first two are just numbers sitting in a drawer.

Writing it down makes two things possible: someone reaching a different conclusion can point to exactly where the disagreement is, and the choice can be reopened later once conditions change. Fairness isn't a setting you configure once and leave; it's a decision that keeps getting reopened.