Train a dice reader and score Yahtzee

In this workshop you teach a webcam to read dice, then use what it sees to score Yahtzee. You capture and label your own images, train two detection models two different ways, measure which one reads dice better, and build a Yahtzee scorer on the winner that runs on your machine.

A webcam on a purple post at the back of a white desk, facing eight translucent colored dice rolled out in front of it.

Dice are a good first vision project. They’re cheap and the same everywhere, a pretrained model can’t read them, and there’s never any doubt about what a die shows.

You’ll learn how to:

  • Add images from a camera to a dataset
  • Label images, and use a starter model to label more of them for you
  • Train an object detection model with built-in training
  • Train a YOLO model with a training script from the registry
  • Evaluate models on held-out test datasets and compare the results
  • Choose a confidence threshold from evaluation results
  • Deploy a model with the ML model and vision services
  • Build an inline module that scores rolls from your model’s detections

Requirements

Hardware

  • A USB webcam connected to your computer, such as a Logitech C920. Any webcam that viam-server can stream from works.
  • Five or more dice whose pips stand out clearly from the die, such as white pips on colored dice. Any color works; the model learns the colors you train it on.
  • A way to hold the webcam about 30 cm above your workspace, and later about 45 cm. A stand, a clamp, a 3D-printed post or tape all work.
  • A plain background to roll on, such as a dice tray or a sheet of plain paper.
  • A Mac or Linux computer that runs viam-server.

Software

What you should know first

Check your setup

Everything in this workshop happens in the Viam app, including the code in Phase 10, so you don’t need to install anything on your computer besides viam-server.

Open your machine’s page in the Viam app and check that it shows Live. If it shows Offline, check that viam-server is running on your computer.

Phases

The phase times are hands-on time. Each training job also runs for some time in the cloud, and the phases tell you what to do while you wait.

PhaseWhat you end withTime
1. Capture a starter set from your webcamThe webcam streams in the camera card’s Test section and 30 images of dice are in a dataset.About 20 minutes
2. Label the starter setEvery die in the starter set has a box and a value.About 20 minutes
3. Train a starter and capture moreA starter model trains while you capture about 70 more images.About 20 minutes
4. Auto-label the training setThe starter model labels your training set, you correct it, and you add your starter images. Milestone: a model you trained labels your data.About 20 minutes
5. Train a built-in modelA TFLite detection model trains on the full training set.About 10 minutes
6. Train a YOLO model from the registryA YOLO model trains on the same dataset.About 15 minutes
7. Build the test datasetsA clean and a hard test dataset, labeled and checked.About 25 minutes
8. Evaluate both models and pick a winnerFour evaluations and a chosen model and threshold. Milestone: a measured answer to which model reads dice better.About 20 minutes
9. Deploy the winning modelThe winning model draws boxes on the live camera feed.About 15 minutes
10. Score a Yahtzee rollAn inline module reads a roll and returns its score. Milestone: an application built on your own model.About 20 minutes

When you finish, the wrap-up suggests where to go next.