Policy
A standing rule for what to do in each situation
- A policy is a standing rule for what to do in a given situation. It's also the actual thing reinforcement learning ends up producing once it's done.
- Sometimes it pins down one action per situation, and sometimes it splits the odds across several actions instead.
- There are two ways to fix a policy: score every action and pick whichever scores best, or nudge the policy itself, a little at a time.
- How good a policy is gets measured by running it through many rounds and adding up the total score it earns.
- A policy is a method, not the goal. The goal is the score — the policy is just the standing rule chosen to earn more of it.
Contents
1The analogy
Walk into an amusement park and the choices don't stop coming all day. A sign reading a sixty-minute wait in front of a ride means deciding whether to skip it or not, and a twenty-minute sign means deciding all over again whether to get in line. Some people just go with their mood in the moment, but anyone who's been enough times sets standing rules and moves by them instead. Skip anything over sixty minutes. Always get in line under twenty. Step out of any line at noon and eat first. This short list of what to do in each situation is what changes how the day turns out. Same park, same hours, yet how many rides someone ends up on by the end varies completely. In reinforcement learning, a standing rule like this is called a policy.
2In detail
Feed in a situation, get back an action
A policy is a standing rule that takes a situation and returns an action. At the amusement park, the situation is where you're standing, the posted wait, and the time of day, and the actions are things like getting in line, walking past, or stopping to eat. Every time the reinforcement learning agent sees the current state, it consults this rule once and pulls out its next action.
The shape a policy takes depends on the size of the problem. When there are only a few dozen possible situations, it can be written out as a single list. When situations are effectively endless, like raw screen pixels, no list could ever hold it — a neural network that takes in a situation and returns an action does the job of a policy instead. Learning, here, means gradually correcting this rule.
Pinning one action down versus splitting the odds
A policy can take two shapes. One pins a single action to every situation. A sixty-minute wait always means walking past. It's simple and steady, but it always takes the same path, so it never finds out what's waiting down a road it never takes.
The other splits the odds across several actions. A sixty-minute wait might mean walking past nine times out of ten, and getting in line once. A policy built this way keeps testing something a little new all the time, which also comes with the advantage of being harder to read in problems where there's an opponent involved. A common setup mixes odds during training and then, once learning is done, locks onto whichever single action scores highest.
Two ways to correct a policy
The first way scores things first. It writes down, for every situation, roughly how much score each action is expected to earn going forward, and the policy just picks whichever scores highest. Here the policy isn't learned separately — it falls straight out of the scoreboard. This works well when the number of possible actions is small and countable.
The second way pushes the policy directly. Run several rounds under the current policy, then nudge up the odds of whatever action was picked in the rounds that scored well, and nudge down the odds of whatever was picked in the rounds that scored badly. This path gets used when an action is a continuous value, like the angle of a turn, and when a policy that splits its odds is what's actually needed. Combining both paths is common too — one side scores things, and the other side pushes the policy based on that score.
How to measure whether a policy is good
Whether one policy is good or bad can't be judged from a single round. A day with a lucky short line and a rainy day end up with different results. So the same policy gets run through many rounds, and the average score it earns gets checked instead. That average is called the value of the policy.
When correcting a policy, the push always has to go in the direction that raises this value. Swing the policy hard based on one lucky result and it tends to get worse, not better, so a device that only shifts things a little at a time rides along with the correction. A well-trained policy shows it in a total score that's noticeably higher — like ending the day having ridden a lot more rides than usual.
A policy is a method, not the goal
However good a policy looks, it isn't itself the goal. The goal is always the score. Change where the score gets placed, and even the best policy flips entirely. Put the score on number of rides taken, and a policy that only chases short lines becomes the best one. Put the score on riding one popular attraction no matter what, and a policy willing to wait an hour becomes the best one instead.
For the same reason, it's hard to look at a policy and call it good or bad just by eye. A policy that looks strange to a person often earns a higher score than expected. In the end, the total score it earns is what makes the call.
3More precisely
A policy is a function that maps a state to an action. Pin down a single action and it's called a deterministic policy; assign probabilities across several actions and it's called a stochastic policy. Learning a value function first and picking the best action from it is called value-based; pushing the policy's own values directly is called policy gradient; and using both together is called actor-critic. What reinforcement learning is really searching for is the optimal policy — the one that maximizes expected cumulative reward.
The analogy breaks down in a few places. An amusement park's standing rules are sat down and worked out by a person, but a reinforcement-learning policy gets nudged into shape gradually, out of the results of tens of thousands of rounds. So open up a finished policy and you won't find a list of clauses — often it's a solid block that only produces an action once you feed in a situation. One more thing: a person tries hard to stick to their amusement-park rules, but a policy still being trained will deliberately break its own rule sometimes. Trying an untraveled road is the only way to find a better one.
4Try it yourself
5Common misconceptions
It's easy to think a policy is a rule handed down by a person, but actually it's a result that learning gradually refines on its own, guided by the score it earns.
It's easy to think a good policy always does the same thing, but actually splitting the odds across several actions turns out better more often than not.
It's easy to think a policy and a goal are the same thing, but actually the goal is pinned to the score, and the policy is just the method chosen to earn more of it.
7One-line summary
In shortA policy is a standing rule for what to do in each situation, and it's the very thing reinforcement learning refines and finally hands over once it's done chasing the score.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02