Skip to content

docs(experiments): add PyTorch video poker example with a full W&B run lifecycle - #657

Merged
jstack-wandb merged 6 commits into
mainfrom
pytorch-video-poker-example
Sep 23, 2026
Merged

jstack-wandb merged 6 commits into
mainfrom
pytorch-video-poker-example

Conversation

@jstack-wandb

@jstack-wandb jstack-wandb commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

What

Adds examples/pytorch/pytorch-video-poker-bot/, a PyTorch example that trains a small network to play 9/6 Jacks or Better video poker and tracks training and evaluation in W&B. No existing example changes.

Why

Most examples in this repo are several years old, and a newer example was requested that shows current W&B usage end to end. The poker task is a stand-in: the README is written for readers learning W&B, not poker.

How

  • Three scripts run in order: generate_dataset.py builds the training data, train.py trains and logs, and evaluate.py plays hands with the trained model and logs the score.
  • wandb.init() records the config, puts each run in a group (train or eval), and takes an optional --entity for logging to a team.
  • run.log() logs metrics every epoch in training and running totals every 100 hands in evaluation.
  • run.log_artifact() uploads the best checkpoint as a versioned model artifact. evaluate.py downloads it with run.use_artifact(), which links the evaluation run to the training run in the artifact's lineage.
  • Runs finish when the with wandb.init(...) block exits; the README explains that run.finish() is needed without it.
  • File paths and W&B names are not hardcoded. The dataset path, checkpoint path, artifact name, project, and run name are required flags.
  • Seeds are random unless --seed is passed. The seed used is saved in the run config, so any run can be replayed.
  • The best checkpoint is the epoch with the lowest val_mean_regret: the average expected profit per credit bet that the network's holds give up compared with the optimal hold.
  • The training data is not committed. generate_dataset.py writes 1M hands (about 44 MB) in about two minutes. Each hand is labeled with the exact expected profit of all 32 hold choices, which ev.py computes by counting every possible draw from precomputed tables instead of simulating. .gitignore covers generated data, checkpoints, downloaded artifacts, wandb/, and .venv/.

Testing

  • Ran the README steps against a local W&B server and against QA (lucas-defino-weights-biases/pytorch-video-poker): 5 training runs with random seeds, then 5 evaluations of the lowest-regret model at 250,000 hands each.
  • QA training runs reached a best val_mean_regret of 0.0005 to 0.0014 and 98.8% to 99.2% validation return.
  • The five QA evaluations returned 99.14% combined over 1.25M hands, against about 99.5% for optimal play. Individual runs ranged from 98.34% to 99.71%. Most of that spread comes from royal flushes, which pay 4,000 credits and occur about once per 40,000 hands.
  • ev.py matched brute-force enumeration of every possible draw for 3 random hands and all 32 holds, with zero difference.
  • Passing a run's logged seed back with --seed reproduced its training metrics exactly.
  • Not tested: Python versions other than 3.9, and GPUs. The code runs on CPU only.

Anything Else

  • New dependencies: requirements.txt lists numpy, torch, and wandb with no version pins, like 15 of the other 21 requirements*.txt files under examples/. Tested with numpy 2.0.2, torch 2.8.0, and wandb 0.26.1.
  • evaluate.py downloads the model with run.use_artifact(), so it does not run with WANDB_MODE=offline.
  • Should requirements.txt pin the tested versions instead of matching the unpinned majority?

Train a small PyTorch network to imitate an exact hold-EV calculator for
9/6 Jacks or Better, logging training and evaluation to W&B.

The training dataset is not committed. generate_dataset.py builds it
(about two minutes for 1M hands), and the scripts take its path
explicitly:

    python generate_dataset.py data/hands.npz
    python train.py --dataset data/hands.npz

data/dataset.py only loads datasets; generation defaults live in
data/generate.py.
…r example

Every file path, the W&B project, and the run name are now required
flags, with no hardcoded defaults:

    generate_dataset.py --output
    train.py --dataset --checkpoint --project --run-name
    evaluate.py --checkpoint --project --run-name

The README now leads with the W&B run lifecycle (init, log, artifact,
finish), explains video poker in plain terms with the 9/6 Jacks or
Better paytable in credits, and describes each file in one line.
… W&B

- Fix a train/play mismatch: training encoded real suits while play
  relabeled them first. There is now one encoder, used by both.
- evaluate.py fetches the model with run.use_artifact() (linking it to
  the training run) and logs running totals every 100 hands.
- train.py takes --artifact-name, logs val_mean_regret (used to pick the
  best checkpoint), and records only real hyperparameters in its config.
- Fold data/generate.py into generate_dataset.py and move the loader to
  dataset.py. Datasets now hold just cards and targets, split 90/10.
- Remove unused code across game.py, jacks_or_better.py, ev.py and
  model.py; share hand-size constants from game.py; rename the model
  class to Network; checkpoints store only hidden size and weights.
- train.py and evaluate.py pick a random seed unless --seed is passed.
  The seed used is recorded in the run config, so any run can be
  replayed exactly.
- Training runs go in the "train" group and evaluation runs in the
  "eval" group.
- README notes both.
- ev.py: describe what it computes (expected profit, not "expected
  value") and how, including the inclusion-exclusion step that removes
  draws reusing discarded cards.
- game.py: no deal/hold/draw loop and no paytable values live here.
- jacks_or_better.py: EvaluatedHand no longer carries anything richer
  than the rank.
- generate_dataset.py, train.py: the targets are expected profit per
  credit bet, not expected reward.
Lets runs log to a W&B team instead of relying on the machine's default
entity. Leaving it off keeps the old behavior.
@jstack-wandb
jstack-wandb force-pushed the pytorch-video-poker-example branch from e268336 to 4af823f Compare September 23, 2026 18:03
@jstack-wandb jstack-wandb changed the title Add PyTorch video poker example showing a full W&B run lifecycle docs(experiments): add PyTorch video poker example with a full W&B run lifecycle Sep 23, 2026
@jstack-wandb
jstack-wandb marked this pull request as ready for review September 23, 2026 18:08
@jstack-wandb
jstack-wandb merged commit b58c231 into main Sep 23, 2026
7 of 8 checks passed
@jstack-wandb
jstack-wandb deleted the pytorch-video-poker-example branch September 23, 2026 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants