Svigl

AI Lab · Experiment 10

Comparative sketch recognition on Quick, Draw!

DJS23IPC302L · Artificial Intelligence Laboratory

Aim. Compare open-vocabulary vision (Gemini 3.5 Flash-Lite) with closed-set stroke/raster models and a typed decision baseline on the same doodle prompts. Sketches are loaded from Google’s Quick, Draw! dataset (simplified ndjson, CC BY 4.0), rasterized to 256px PNG for Gemini 3.5 Flash-Lite.

Six hold-out doodles (harder, often unrecognized in-game). Exact-match scoring; ~4s between live API calls. Use sample results if the network or API key is unavailable.

  1. Step 1

    Dataset & protocol

    Scroll to sections 01–04: 24 noun classes from Quick, Draw!, exact-match scoring, paired IDs, published baselines vs measured Gemini.

  2. Step 2

    Hold-out evaluation

    Run the six-sketch batch (highlighted corpus tiles). Each card is real ndjson ink, rasterized to 256px PNG—the same vision path as the write-up.

  3. Step 3

    Read the metrics

    Top-1 / top-3 and latency update in the stats row and comparison table. Published CNN/RNN/Jev rows stay fixed at 345-way literature values.

  4. Step 4

    Sketch-RNN caveat

    Explain why Magenta Sketch-RNN is shown but not ranked: it generates strokes; it does not output a class label.

Gemini top-1 (measured)

—

0 / 6 on this sheet

Gemini top-3 (measured)

—

Any of three ranked guesses

Mean latency

—

Raster + Gemini round trip

01

How we score

01

Same prompts

Sketches come from Google’s Quick, Draw! simplified ndjson. We skip the first ~90 drawings in each class and prefer unrecognized doodles — the ones the in-game classifier already missed — so the live sheet is not a pile of easy hits.

02

Same pixels for vision

Gemini sees a 256×256 white PNG of the ink. Stroke models would see ordered offsets; that difference is the point of the comparison.

03

Exact-match scoring

A guess hits if it matches the prompt after lowercase, punctuation collapse, and a stripped leading article — the production Svigl matcher.

04

Separate evidence

Gemini is measured on this page when you run the batch. CNN, RNN, and Jev rows are published figures from papers and tutorials—not re-trained during the demo.

02

Hold-out evaluation batch

Each card is a real Quick, Draw! stroke record (later in the class file, preferring recognized=false). Ink is rasterized and sent to the vision guesser; guesses are compared to the prompt with exact-match scoring.

  • ice cream

    Ready

  • chair

    Ready

  • shoe

    Ready

  • moon

    Ready

  • bird

    Ready

  • key

    Ready

03

Five methods, two kinds of evidence

Measured

Open-vocabulary vision

Gemini 3.5 Flash-Lite

256px PNG

—

top-3 —

Open-vocabulary vision: 256px PNG in, three ranked text guesses out. Exact-match scoring against the prompt word.

Measured on this page · Gemini API

Published

Closed-set sequence

Stroke BiLSTM

pen offsets

70.0%

top-3 86.0%

Google’s Quick, Draw! tutorial classifier. Softmax over 345 prompt classes. Not re-trained here.

TensorFlow recurrent_quickdraw · ~70% top-1, 345-way

Published

Closed-set raster

28×28 CNN

bitmap

65.6%

top-3 82.7%

Tutorial CNN on official 28×28 Quick, Draw! bitmaps. Cannot name a class outside its softmax.

SketchRecognition 345-way · 65.6% / 82.7%

Published

Typed decision model

Jev System One

SVG as text

35.0%

top-3 52.0%

TypeSafe Jev is paid and not keyed in this app. Figure is the public SVG-as-text doodle study, not a sketch specialist.

Mikulski 2026 · 10 classes, n=400

Out of task

Generative VAE

Sketch-RNN

stroke sequence

No top-1 — generation only.

Ha & Eck predict the next pen move. It draws. It does not emit a class label, so it is not ranked.

arXiv:1704.03477

04

Paired comparison

Gemini rows update after you run the batch (or load sample results). Other methods stay at published 345-way reference values so families are comparable without mixed evidence.

MethodEvidenceTop-1Top-3Latency
Stroke BiLSTMpen offsetsPublished70.0%86.0%18 ms
28×28 CNNbitmapPublished65.6%82.7%4 ms
Jev System OneSVG as textPublished35.0%52.0%140 ms
Gemini 3.5 Flash-Lite256px PNGMeasured———
Sketch-RNNOut of taskNext-stroke generator. No class label, so no rank.

05

Prompt corpus

Twenty-four classes from the official 345-way dump. Highlighted tiles are in the evaluation batch. Ink is a later drawing, preferring recognized=false.

  • cat
  • dog
  • fish
  • bird
  • house
  • car
  • tree
  • apple
  • cup
  • hat
  • star
  • moon
  • chair
  • shoe
  • flower
  • pizza
  • clock
  • key
  • boat
  • sun
  • guitar
  • book
  • ice cream
  • bicycle

Sketch-RNN reminder

It draws. It does not guess.

Magenta Sketch-RNN is a VAE over pen strokes. Ranking it as a classifier would invent a peer that the paper never claimed.

Quick, Draw! dataset © Google, Inc. github.com/googlecreativelab/quickdraw-dataset · CC BY 4.0