Ugg look at world. World just dots.
Ugg open eye, see mammoth — feel EASY. But machine no have eye-magic; it see a picture as just a pile of number-rocks. Computer vision — the art of pull meaning out of flat picture: WHAT thing, WHERE thing, who, when, why, how many. Big secret: picture is JUST numbers. Edge, mammoth, meaning — none come free. Ugg must COMPUTE all of it.
Why hard? Tribe-brain use MORE THAN HALF its head-meat just to see — half of the best thinking-rock, built over 540 million snows (since the Cambrian). Ugg warn — no think "seeing is easy because it FEELS easy." It feels easy only because the head has dedicated, pre-built hunt-hardware. The big brain-budget is PROOF the computing is huge, not that the problem is small.
Old wise-man Marr say the goal
Wise-man Marr (old snow 1982) give one-line hunt-goal:
That is the WHOLE mission. "What" = which thing it is (recognise); "Where" = where it sits, 2D and 3D (locate); "By looking" = from picture only, no cheat with other sense-rock. New tribe add WHEN (action), WHY (intent), HOW MANY (count) — but the spine stay Marr: what and where.
Story of shame: in snow 1966, man Papert wrote a cave-memo — send a few young hunters, they build "a big part of a seeing machine" in ONE summer. Ha! Sixty snows later the tribe STILL hunts this. Lesson: the Summer Vision Project shows computer vision is NOT a summer hunt — it needs deep math-magic. Grader love this fact in exam.
Three hunts of Malik — the Three Rs
Wise-man Malik chop vision into Three Rs. Ugg carve on wall:
Reorganisation — group the dots that belong together into regions, like sort berries into piles. Cut picture into "sky", "road", "person" (segmentation). Edge-finding live here too.
Recognition — match what Ugg see to memory. "That a face. That a cat." Put the NAME on it. Big brain-net hunt here.
Reconstruction — MEASURE the shape and depth of the world. How far the mammoth? How tall the rock? Build 3D back out of flat picture (Structure-from-Motion, Hawk-Eye ball-path).
Ugg warn: NO mix them! Reorganise = group dots. Recognise = give name. Reconstruct = measure geometry. And they NOT separate — real machine use all three at once: the cart-that-drives-self must recognise the walker AND cut out the road AND measure 3D distance to other carts.
Why picture fight Ugg — seven reasons
One: pixels are just numbers. Machine read — no "edge", no "cat" until Ugg compute one. The gap between raw number and real meaning is the semantic gap:
Crossing it is the whole job.
Two: intra-class variation — one class look many ways. "Chair" spans throne, beanbag, office-chair, tree-stump; all look different, so no single picture-template catches every chair.
Three: viewpoint — same rock from a different angle makes a very different flat picture.
Four: illumination — fire-light, shadow, sun all change the number-dots; light is a bad clue for know-what.
Five: occlusion and clutter — occlusion = thing hide behind thing, background messy. Tidy road-picture (KITTI) is easy; busy Indian road with cow, auto, and no lane-line makes a trained hunter fail (distribution shift).
Six: scale — same thing be 10 dots (far walker) or 10,000 dots (face fill frame).
Seven: ambiguity — MANY 3D worlds make the SAME flat picture; a flat painting of a cube looks same as a real cube from one eye-spot. This is the inverse problem of vision: picture-making squashes the world,
and un-squashing is one-to-many. So Ugg MUST use priors — "light comes from above", "rocks are convex" — to pick one good answer. No prior, no recover.
Should machine copy the human eye?
Favourite exam-fight. Ugg argue BOTH sides, never one only.
YES, copy: human eye is the best seeing-system known, and great at recognise, segment, and 3D. Ideas like receptive-field, layer-on-layer feature, and attention came from brain-study and now power the big nets.
NO, not-copy: human eye ALSO fails! Trick-pictures fool it (Adelson checker-shadow, Kitaoka spinning snakes), and inattentional blindness — Simons & Chabris "invisible gorilla": people counting ball-passes MISS a whole gorilla-suit man walking through. Human eye is NOT a faithful camera; attention picks what Ugg see. Machine can also carry sense human no have (LiDAR, infrared). So: take inspiration when handy, no be chained to biology.
Old Gestalt wisdom — how the eye groups
Long before brain-nets, Gestalt wise-folk (1920s) wrote how humans group bits into wholes: "the whole is other than the sum of its parts." Memorise six (seven): Proximity (near dots group), Similarity (same-color dots group), Closure (mind finish a broken shape — WWF panda), Continuation (eye follow smooth path — two crossing dot-lines seen as two lines, not four bits), Common Fate (dots moving together group), Figure-Ground (split thing from background — Rubin's vase), plus Symmetry. These birthed old segmentation magic (mean-shift, normalized cuts) and still whisper in modern attention. Ugg warn: exam ask for four, no give only two!
Why "chair" breaks the hunter — affordance
Beanbag and throne look NOTHING alike, but both let Ugg SIT. Wise-man Gibson call this affordance — a possibility-for-action the thing offers. Chair afford sit, handle afford grab, cup afford hold-water. A class can be named by what it LETS YOU DO, not what it looks like. This is why an appearance-only hunter hits a ceiling on "chair" — the deep answer is that a functional class beats a visual class. Exam ask "why chair hard?" — no only say "many shapes", SAY affordance.
The Bitter Lesson
Last big rule, from wise-man Sutton: the Bitter Lesson — methods that ride raw compute eventually beat methods with hand-carved cleverness. Hand-made features (SIFT, HOG) lost to learned brain-nets (CNN); specialised CNNs now lose ground to more-general transformers. Bet on data + compute + general shape, NOT on baking-in Ugg's clever trick.
Course assume Ugg carry four spears: linear algebra (vector, matrix, eigen, SVD), image/signal processing (filter, edge, Fourier), machine learning (feature, classifier, loss), and programming (Python, NumPy, PyTorch). Signal and ML get recapped next; the other two Ugg carry forward.
Ugg remember
- Marr: vision is "to know what is where, by looking" — from images only.
- Three Rs (Malik): Reorganise (group dots), Recognise (name), Reconstruct (measure 3D). Use all three together — no mix the meanings.
- Seven reasons CV hard: pixels-are-numbers (semantic gap), intra-class variation, viewpoint, illumination, occlusion, scale, ambiguity (inverse problem → need priors).
- Six Gestalt: Proximity, Similarity, Closure, Continuation, Common Fate, Figure-Ground (+ Symmetry). And affordance (Gibson): name a class by what it DOES, not how it looks.
- No think seeing is easy — brain spends half its meat on it. Argue BOTH sides of "copy the human." Bitter Lesson: scale + general beats hand-carved.