01An image is just a grid of numbers

Imagine a sheet of graph paper where every little square is filled in with a shade of gray. A black-and-white photo on a computer is exactly that: a grid of tiny squares called pixels, and each pixel is simply a number that says how bright that spot is. A color photo is the same idea, just with three numbers per pixel — one for red, one for green, one for blue (together, RGB). So a picture the size of your phone screen is really millions of numbers stacked in a grid. That's the whole starting point: to a machine, "seeing" means doing math on that grid of numbers. The hard part is that the same object — a cat in sunlight, a cat in shadow, a cat turned sideways — becomes a wildly different set of numbers each time, so the machine has to learn patterns that survive all that change.

• A pixel is one tiny square of the image, stored as a number (brightness) — or three numbers for color (R, G, B).
• A whole image is a grid: height × width × channels. Vision is math performed on that grid.
• The challenge: the same object maps to different numbers under different light, angle, and position — so meaning lives in patterns, not single pixels.

02Why CNNs work: filters that build up from edges to objects

If meaning lives in patterns, how does a machine find them? The workhorse of vision is the convolutional neural network (CNN). The trick is a small filter — a tiny detector that slides across the whole image looking for one simple thing, like an edge. Because the same filter is reused everywhere, it can spot that pattern no matter where it appears. Stack many layers of filters and something powerful happens: early layers find edges, the next combine edges into textures, then into shapes, and finally into whole objects. Tap each stage to see what it does.

• A filter is a small, reused detector that finds a local pattern anywhere in the image — that reuse (weight sharing) is what makes CNNs efficient.
• Depth builds a hierarchy: edges → textures → shapes → objects. Each layer composes the simpler features below it.

03Watch a model "see": from pixels to a label

Let's put it together. Below is a tiny illustrative image — a grid of pixel numbers. Step through what a model does to it: first it runs edge detection, then it forms feature maps (where useful patterns light up), and finally it settles on a label — that's classification. Then flip the overlay to compare how the same picture is handled by object detection (a bounding box: what and where) versus segmentation (a pixel mask: which pixels belong to the object). Everything here is a simplified illustration, not a real model's output.

• Classification ends at a single label ("what is it"). Detection adds a box ("what + where"). Segmentation marks each pixel ("which pixels").
• The earlier stages (edges, feature maps) are shared groundwork — the same features can feed any of the three tasks.

04The three core tasks: what, what+where, which pixels

Almost everything in computer vision is a version of three questions. They differ in how precisely they pin down an object in the image — from a single word, to a box, to an exact outline. Switch between them to see the idea and a one-line example of each.

05Reusing knowledge, Vision Transformers, and where models still fail

You rarely start from scratch. Training a strong vision model needs a lot of images, so practitioners lean on transfer learning: take a model already trained on a huge image collection, and adapt it to your task. The reusable part — a general feature extractor — is called a pretrained backbone; you attach a small task-specific "head" on top for classification, detection, or segmentation. With only a few hundred labeled images, fine-tuning a backbone almost always beats training a big network cold, which would just memorize the tiny dataset.

For most of the last decade, CNNs were the only game in town. More recently the field has shifted toward the Vision Transformer (ViT): instead of sliding filters, a ViT chops the image into small patches, treats each patch like a word, and uses attention — the same mechanism behind modern language models — to weigh how every patch relates to every other. That lets it capture long-range structure across the whole image at once.

However a model sees, it can still be wrong in revealing ways. Bias: a model inherits the skews of its training images, and can perform unevenly across groups or conditions it rarely saw. Adversarial examples: a tiny, often invisible-to-us change to an image can make a confident model misclassify it entirely. Domain shift: a model trained mostly on clear daytime photos may stumble at night or in fog, because the new images don't match what it learned from. These limits are why computer vision in the real world demands testing, monitoring, and human judgement — not blind trust.

06Check your understanding

07Take it with you & go deeper