Act 01
The game
Before any of the machine learning, the thing it has to learn: a two-player Wild West card duel with hidden hands, dice-free luck, and 24 characters that all break the rules differently.
The board
Law versus Outlaw, four characters each.
BANG! The Duel is the two-player version of BANG!. One player takes the Law, the other the Outlaws. Each side brings four characters, but only two stand on the table: an active character in front and a rearguard behind it. The other two wait in reserve.
Every character has its own printed life total. Lose them all and that character is eliminated, a reserve steps into the same slot, and the side that eliminated it draws two cards as a reward.
There is exactly one way to win: eliminate all four characters on the other side of the table.
The draft
24 characters. You get four of them.
Twelve Law characters, twelve Outlaws, and every one of them has an always-on or once-per-turn ability that bends a rule: an extra BANG! each turn, a free counter-shot, damage that splits, a card that can be played as something it isn't.
Setup is a simultaneous, blind choice — both players pick four, both commit, then two of each four are revealed and enter play. Law draws four cards, Outlaw draws five, and Law takes the first turn.
Which four you hold is the first real decision of the game, and it is made before you know anything at all.
The turn
Draw two, play, discard down.
A turn starts with whatever is already stuck to you — Dynamite, Rattlesnake, Stampede all resolve before anything else — then you draw two cards from your own 40-card deck.
Then you play as much as you like, with one hard brake: one BANG! card per turn unless a Gun Belt or the right character says otherwise. You may swap your active and rear characters once. At the end, you discard down to a hand limit equal to your active character's remaining life.
Hurt characters hold fewer cards. Falling behind on life means falling behind on options.
The fight
Every shot is a negotiation.
A BANG! card does not simply deal damage. The target can answer with Missed!, with a Hat, or by revealing the top card of the deck and hoping for a Barrel symbol. Return Fire sends the hit straight back. Some characters get an extra answer for free.
Reach matters too. By default you can only shoot the character standing in front of you; hitting the rearguard needs a Winchester, a Scope, or the one character who ignores the rule.
Which is why the interesting question is rarely "can I shoot" — it is "what can they still do about it".
The hard part
Half the board is a secret.
The agent sees its own hand, both boards, the shared discard pile, and everything play has already revealed — including cards it watched change hands. It never sees the opponent's hand, their reserve identities, or the order of any deck.
The move list is never the same twice either. The engine defines 997 stable action IDs covering every card instance, target, ability and response; on a typical decision only a handful are legal.
Two different secrets can produce the exact same view. Against a hidden hand, the best you can do is play the average.
The only signal
One bit of feedback, once.
No points for damage. No bonus for a clever save. The duel pays +1 for a win, −1 for a loss, and stays silent for everything in between — deliberately, because hand-written rewards teach the agent to chase the reward instead of the win.
A finished game runs about — turns and roughly 270 engine steps.
Hundreds of decisions, one bit of truth at the end. Working out which move deserves the credit is the entire problem.
The wrong tool
You cannot search your way out of this.
The classic way to beat a board game is to search: expand the tree of futures, score the leaves, back the values up. That works when you can see the position and the rules are deterministic.
Here neither holds. You do not know their hand, so you cannot expand their real options. Every draw and every Barrel reveal branches on chance. Games run 35 turns deep, and a "position" includes two hidden hands and two shuffled decks — the tree does not just get big, it gets unknowable.
So don't evaluate the tree. Learn a reflex: a function that looks at what you can see and says what to do.
The object being trained
The player is a probability distribution.
The thing being optimised is a policy: a function from the visible state to a probability for every currently legal move. To play, it samples from that distribution rather than always taking the top move — partly to keep exploring, partly because in a hidden-information game being predictable is itself a losing strategy.
Every decision it makes is recorded with the probability it assigned at the time. That number is what makes the later maths work.
Training does not edit rules or heuristics. It edits the odds.
The learning rule
Do more of what beat expectations.
The naive version — "we won, so every move we made was good" — is far too noisy: a won game is full of mediocre moves. So a second network, the critic, learns to predict the eventual result from any position. The learning signal is the difference between what actually happened and what the critic expected.
That difference is the advantage, smoothed backwards along the game so credit reaches the moves that set a win up, not just the one that finished it. Positive advantage pushes a move's probability up; negative pulls it down.
Not "did we win" but "did that go better than predicted". That is what makes one bit of feedback usable.
The brake
Improve, but never leap.
The batch is — games — a small sample of a game with this much variance. Trusting it fully would lurch the policy somewhere stupid, and because the next batch is generated by that same policy, a bad lurch poisons everything after it.
This is the "proximal" in PPO. Each update measures how much a move's probability has moved since the batch was collected, and once that ratio leaves a ±20% band the objective goes flat: pushing harder earns nothing. A separate check stops the update early if the policy has drifted more than — overall.
Small, provable steps beat big, hopeful ones — every time.
The loop
Play — games. Learn. Forget. Repeat.
One iteration plays a fresh batch of games across many boards at once, keeps only the decisions the learner actually made — forced responses are environment steps, not lessons — and grades every one of them.
It then makes — passes over that batch in — minibatches, with a small entropy bonus that penalises collapsing to a single move too early, and throws the data away. Data from an older policy would be answering a question nobody is asking any more.
Then it does that — times.
The trap
Beating yesterday's you proves nothing.
Self-play has one famous failure. Train only against your newest copy and the target moves with you: you learn the counter to whatever it does now, it learns the counter to that, and the pair can circle forever.
Card games are especially prone to this because strength is not a ladder. Aggression beats greed, greed beats control, control beats aggression. There is no single "best" to converge on.
You can win every single game you play and go absolutely nowhere.
The fix
So it fights a whole population.
Every game picks an opponent from a league. Half face a frozen snapshot of the current policy — frozen so the opponent cannot shift inside an update. Forty percent face older checkpoints pulled from an archive of past selves.
The last tenth is the safety rail: a scripted tactical bot and a random legal player. They never improve, which is the point — they are the fixed measuring stick that catches basic competence quietly rotting away.
The taste
The best sparring partner is the one you barely beat.
Opponents you crush teach nothing: the advantage is positive everywhere and no move stands out. Opponents that crush you teach noise. So the archive is sampled by how close each matchup is to even — prioritized fictitious self-play.
Every archived policy keeps a floor probability, so an old trick that nothing currently punishes stays in rotation instead of being forgotten and re-learned.
The state
The table is — numbers.
Phase, both characters in play and their life, equipment, ability usage, the discard pile and its top card, the current pending hit and any queued response — all flattened into one fixed-length vector.
The card-identity parts are strictly observer-scoped: its own hand, its known reserve group, the public discard, and cards it legitimately watched move. Opponent hands and deck order are simply absent, not zeroed-out placeholders it could learn to peek at.
And it is always written from the acting seat: me versus them, never Law versus Outlaw. One network plays both sides.
The moves
Describe the move, don't number it.
The lazy design gives the network one output slot per action — all 997 — and learns each from scratch. Most would be illegal at any given moment, and every one of the four Missed! copies in your hand would be a separate lesson.
Instead each legal move is written as — semantic features: what kind of move it is, which card name and type, whose card, which relative target, which character, which option in a pending choice.
Two copies of the same card produce identical numbers — learn one, get the rest for free.
The choice
Score them all, then pick.
The network reads the situation once and each candidate move, then returns a single number per move: how much it likes that move, here, now.
A softmax over only the legal moves turns those scores into probabilities. Nothing has to be masked out afterwards and no decision is padded to some worst-case width, so an illegal play is not unlikely — it is unrepresentable.
The shape
— weights. That is the whole player.
Two encoders read the observation: one for the public board state, one for card identities, which are far sparser and deserve their own compression. They fuse into a 128-wide trunk with two residual blocks — the part that actually does the thinking.
From there a situation vector meets each move's features in a small scoring head. It is a plain feed-forward network: no recurrence, no memory of the game so far beyond what the observation encodes.
No search, no opening book, no rollouts. One forward pass per move, which is why it can live in a browser tab.
The trick
Where the move meets the moment.
Concatenating "the board" with "the card" and hoping a couple of layers work it out is weak: the network has to discover multiplication from scratch, and a card is only ever good in a situation.
So the scoring head is handed the elementwise product and difference of the two vectors as well. The product is a match detector — high only when the situation asks for what the card offers. The difference measures how far off the fit is.
Not "good card" plus "good board". Good card, this board.
The coach
One critic is allowed to cheat.
The deployable critic sees exactly what the player sees, so it can be honest about a fogged position. But learning to predict a winner from a fogged position is hard, and a bad critic means a noisy advantage means slow learning.
So training adds a second, auxiliary critic that sees — extra numbers: both hands, both decks, the full card-zone truth. It shapes the shared trunk into better features, while its gradients never hand the actor a single privileged input.
Then both critics are cut off before export. The coach knew everything; the gunslinger walks out blind.
The receipts
Forty hours of shooting itself.
— complete duels. — game steps. — decisions it had to answer for, across — update cycles.
All of it on one desktop CPU, no GPU. The rules engine is C++ and fast enough that generating the games took — hours; the gradient steps took —.
Thinking about the games cost nearly three times more than playing them.
The curve
The win rate flatlines — and that's fine.
The easy wins come first, against a random player and a scripted bot that never change. Then the line settles near 55% and stops moving for the rest of the run.
That is not a plateau in skill. It is the scoreboard of a fair fight: the opponents are copies of itself, so improving twice as fast changes the graph not at all.
In self-play, a flat win rate is the expected shape of progress. You have to measure against something that stands still.
The fixed yardstick
Which is why the anchors matter.
Pooled over the last — iterations, the learner beats the scripted tactical bot — of the time and a random legal player — of the time. Those two numbers are the ones that moved.
Against a frozen copy of itself it wins —, and against the archive of its own past selves — — both hovering at even, which is exactly what a converged league looks like.
Near-perfect against anything that stands still, a coin flip against itself.
The tell
It stopped hedging.
Early on it spreads its bets: at the first measured checkpoint its favourite move takes — of the probability, with entropy over the legal moves at — of maximum. By the final checkpoint the favourite takes — and entropy has fallen to —.
It never goes fully deterministic — the entropy bonus keeps a little doubt in the system, and in a hidden-information game some randomness is correct play, not indecision.
Confidence is not proof it is right. It is proof it has stopped guessing.
The shipping
And it fits in a browser tab.
At the end of the campaign the best snapshots play a paired round robin — every seed from both seats — and the highest mean field win rate is promoted to champion. The critics are stripped, and the player's weights become a small JSON file.
The rules engine that trained it compiles to WebAssembly, so the browser runs exactly the same rules, not a JavaScript reimplementation that might disagree.
No server, no telemetry. When it shoots you, that decision happened on your own machine.
Act 05
The meta
Put the final weights on both sides of the table, deal the same games repeatedly, and the policy's opinions about BANG! The Duel fall out.
Finding one
Outlaw is simply the better seat.
Identical weights, both chairs, — games on fixed deals. Outlaw takes — of them and Law —.
The extra starting card and nothing else changes hands, and it never flips: across — separate checkpoint evaluations spanning the whole run, Law is under 50% in every single one.
Same player, same skill, different chair. That gap belongs to the game, not the gunslinger.
Finding two
A tier list nobody wrote down.
Measured against its own side's baseline, — tower over their fields, while — drag theirs down. The spread inside one side is wider than the gap between the sides.
And it tracks one thing almost exactly: elimination rate. The characters at the top are the ones that are still standing at the end.
In this meta, power means not dying.
Finding three
It is a power meta, not a combo meta.
Because characters are drafted in fours, the obvious question is which of them combine. So compare every same-side pair's uplift with the sum of its two members measured separately.
The answer is boring in an interesting way: pairs land within a couple of points of additive. Good duos beat their parts slightly, bad duos undershoot slightly, and nothing behaves like a real combo.
Draft the strongest characters. There is no clever engine waiting to be built.
Finding four
The matchups are brutal.
Across the full Law-versus-Outlaw grid, Law's win rate ranges over — percentage points depending on nothing but who was dealt against whom.
The pattern is not subtle. The best cells for Law are the ones facing the weakest Outlaws; the worst are almost all the same handful of Outlaw carries. The single question that decides a Law roster's life is whether it dodged them.
Character selection here is not flavour. It is most of the game.
Finding five
It plays scared. It wins anyway.
Counting choices against opportunities rather than raw totals: it defends — of the times it can, but attacks only —. It fires character abilities — of the time and plays — cards a turn, a fifth of them equipment.
And it almost never reaches for the back row — — of its targeting. Long-range shots need a specific weapon and give up tempo; it would rather kill what is in front of it.
Stay alive, keep the board, win later. Nobody taught it that.
Finding six
Steal, block, breathe.
Top of its shopping list: —. Bottom: —.
The shape is a control deck. Strip the opponent's resources first, draw ahead second, refuse to die third — and systematically avoid anything high-variance or self-damaging. Dynamite is offered tens of thousands of times and taken least of all.
It taught itself that resources beat fireworks, and that a card which can hurt you is barely a card.
The honest part
One equilibrium, not the truth.
These evaluations are greedy: the policy takes its top move every time. Deterministic play exaggerates gaps, so treat the magnitudes as an upper bound. They also share one seed schedule, so the stability above is stability across checkpoints, not across fresh randomness.
And it is one policy's opinion. A converged self-play agent can be blind in ways nothing in its own league punishes, and a character it rates as unplayable may only be unplayable against this strategy.
Strong opinions, honestly measured. Not proofs about the game.