Ugg point at beast: what it, AND where it

Eye-rock grunt "CAT" when see cat. Good! But useless when picture have cat AND dog AND duck. Ugg must circle AND name EACH beast. That is Object Detection — for every beast say WHERE (box) and WHAT (name). Two question, every beast. Whole hunt-story: move slow work onto fast fire-brain (GPU), one shared eye-rock.

Four steps up the hunting rock

Classification — one name for whole picture. Classification + Localisation — picture have ONE beast, so name AND one box $(x,y,w,h)$. Detection — MANY beast, count unknown; give (name, box) for each. Instance Segmentation — colour every pixel of each beast.

Why detection hard? Three reason. Variable count — one picture 2 beast, next 47; eye-rock like fixed-size grunt, not floppy. Mixed output — say WHAT (pick from list) AND WHERE (smooth number) same time. Big picture — naming work at $224\times 224$, but tiny beast vanish small, so detection need ~$800\times 600$.

The box that hug the beast

Ugg warn about two kind box! Modal box hug only part Ugg can SEE (rock hide dog leg, box stop at rock). Amodal box hug WHOLE beast, hidden leg too. Big tribes (PASCAL VOC, COCO) all use MODAL — visible only. Exam trap.

Shiny stone that measure box-match: IoU

Ugg box, true box — how good match? Use IoU (Intersection over Union), also named Jaccard index (same thing).

$\text{IoU}(A,B)=\frac{|A\cap B|}{|A\cup B|}$

Overlap-part over all-part-together. 0 = no touch, 1 = perfect sit. Rule of thumb: $>0.5$ okay, $>0.7$ good, $>0.9$ near perfect. Ugg warn: IoU 0.5 is NOT "half overlap"! Two same-size square sharing only a third each still give 0.5. No treat like percent. IoU same both way.

Too many box! Squash them: NMS

Point eye-rock at one dog, get FIVE box all yelling "dog" (0.9, 0.8, 0.75...). Fix with Non-Maximum Suppression (NMS): keep loudest box, throw away every box overlapping it too much (IoU over ~0.5), repeat. Ugg warn: do NMS PER CLASS! Split by name first, so dog no squash a near person. Bad case: two people shoulder-to-shoulder overlap lots, hard NMS kill one real person. Soft-NMS fix — no zero the score, shrink it: $s_i\leftarrow s_i\cdot (1-\text{IoU})$, or gauss $s_i\leftarrow s_i\cdot \exp (-{\text{IoU}}^2/\sigma )$.

How good is whole eye-rock? Precision, Recall, mAP

Pick one name, one IoU line (0.5). Each box is TP (hit fresh true beast), FP (hit nothing, or hit already-taken beast); a missed true beast is FN.

$\text{Precision}=\frac{TP}{TP+FP},\text{Recall}=\frac{TP}{TP+FN}$

Precision = how many Ugg-box are right. Recall = how many real beast found. Average Precision (AP) = area under precision-recall curve as you slide the loud-threshold (sort by score FIRST!). mAP = mean AP over all name. VOC use mAP@0.5. COCO meaner: average mAP over ten line $0.5,0.55,\dots ,0.95$ — same eye-rock, smaller number, stricter tape.

Hunting grounds: VOC 20 name / 2.4 beast-per-picture. ImageNet-Det 200 name but only 1.1 (one middle beast — GOOD for naming, BAD for detection). COCO 80 name / 7.2 — crowded modern ground.

Journey of the hunt: R-CNN family

First need cheap "these spots worth look": Selective Search — old, not-learned. Over-cut picture, greedy glue similar piece (colour, texture, size). Spit ~2000 maybe-box on CPU.

R-CNN — five step: pretrain eye-rock on ImageNet; fine-tune head to $20+1=21$ name (beasts + background); push all ~2000 warped proposal through CNN, save features to rock-pile (~200 GB on disk!); train one yes/no SVM per name; train box-fixer for $(\Delta x,\Delta y,\Delta w,\Delta h)$. Three flaw Ugg chant: SLOW (~47 s/picture — 2000 CNN push!); POST-HOC (CNN froze before SVM, features can't adapt); MULTI-STAGE (five fire, none end-to-end).

Fast R-CNN — 2000 push waste, all same picture! Push whole picture through backbone ONCE, then cut each proposal from feature-map with RoI Pooling: project box onto feature map, chop into fixed $7\times 7$ grid, max-grab per cell. Every proposal give same-shape tensor, and RoI Pool differentiable so all train end-to-end, one loss. $47$ s → $0.32$ s ($146\times$), mAP $66.0\to 66.9$. But Selective Search still ~2 s on the side!

Faster R-CNN — kill Selective Search, make CNN propose. New piece RPN (Region Proposal Network): slide $3\times 3$ conv on feature map, two $1\times 1$ head branch — "is beast?" and "fix box." At each spot predict anchors: $k=9$ = 3 size $\times$ 3 shape. Predict offset from anchor, not raw:

$t_x=\frac{x-x_a}{w_a},t_y=\frac{y-y_a}{h_a},t_w=\log \frac{w}{w_a},t_h=\log \frac{h}{h_a}$

Anchor give rough size-guess (easier learn); the $\log$ make it scale-blind. Same anchor every spot = translation-invariant. Rest is Fast R-CNN. ~0.2 s, **$250\times$ faster than R-CNN**, same mAP.

One look enough: YOLO

Redmon grunt: "Why two stage? Human glance once!" YOLO (You Only Look Once) — one CNN push spit all box+name in one tensor. Resize to $448\times 448$, cut into $S\times S=7\times 7$ grid. Rule: cell own the beast whose CENTRE fall in it. Ugg warn: CENTRE, not overlap! Horse span 20 cell, only centre-cell answer.

Each cell predict $B=2$ box, each 5 number: $(x,y)$ centre relative to CELL, $(w,h)$ relative to WHOLE picture, $c$ confidence. Plus $C=20$ name-chances SHARED across the 2 box. Output:

$S\times S\times (B\cdot 5+C)=7\times 7\times 30$

Ugg warn: cell give ONE name-list, not one per box! That why YOLO struggle when two kind beast share a cell. Two box why? At train both guess; keep the one with higher IoU to true box as "responsible," push other toward "no object." One specialise tall, one wide.

YOLO loss — sum of squared error over cells, five term, two weight to fix beast/no-beast imbalance: ${\lambda }_{\text{coord}}=5$ (box-place matter MORE), ${\lambda }_{\text{noobj}}=0.5$ (empty cell matter LESS). Ugg warn: no swap them! Five term: box centre $(x,y)$; box size $(\sqrt{w},\sqrt{h})$; confidence on beast-cell (target = IoU); confidence on empty-cell (target 0, weighted small); name on beast-cell. The square-root trick: without it, 10-pixel miss on big box and tiny box count same — but 10 pixel is HALF a tiny beast! Root make small-box miss hurt more. Exam gold.

YOLO three weakness (most-asked!): at most 49 beast ($7\times 7$, one per cell); bad at small clustered beast (flock of birds, many centre one cell); loose boxes (raw regress harder than nudge anchor — v2 add anchor to fix).

Two sharper stones, and the trade

GIoU patch a hole: pure IoU loss give zero pull when boxes no overlap; GIoU minus $|C\setminus (A\cup B)|/|C|$ ($C$ = smallest box hugging both) restore pull, range $[-1,1]$. Focal Loss $=-(1-p_t{)}^{\gamma }\log p_t$ ($\gamma =2$) crush easy background so rare real beast get heard — vital for single-stage where empty anchors swarm.

Trade: two-stage (R-CNN) = careful detective, propose then check — accurate, slow, for medical/security. Single-stage (YOLO) = reflex — real-time, for driving/robot.

Ugg remember

• Detection = WHERE (box) + WHAT (name) for EACH beast; hard because count varies, output mixed, picture big. Boxes are MODAL (visible only).
• IoU $=|A\cap B|/|A\cup B|$ measures box-match; 0.5 is NOT "half overlap." NMS squashes duplicate boxes — always PER CLASS; Soft-NMS decays instead of zeroing.
• mAP = mean area-under-PR over names; VOC@0.5, COCO averages $0.5\to 0.95$ (looks smaller, is stricter). Sort by score first.
• R-CNN → Fast → Faster moves work onto the GPU: per-region CNN → shared backbone + RoI Pool → + RPN with 9 anchors. Speed $47$ s → $0.3$ s → $0.2$ s. R-CNN flaws: slow, post-hoc, multi-stage.
• YOLO: tensor $7\times 7\times 30$, cell owns beast by CENTRE, one shared name-list, ${\lambda }_{\text{coord}}=5,{\lambda }_{\text{noobj}}=0.5$, $\sqrt{w},\sqrt{h}$ trick, cap 49 beast, loose boxes.