Vision & Audio47

How AI recognizes objects in photos, tracks body poses and turns speech into text. CNNs, object detection, segmentation, pose estimation, TTS, STT: many of these you can try with just a webcam and a microphone.

Level
Action Recognition

Watching a run of frames to name what action is happening

Intermediate
Background Removal

Erasing a photo's background and keeping just the subject

Beginner
Bounding Box

The smallest box around an object, and where that box sits

Beginner
CNNConvolutional Neural Network

A neural network built to spot patterns in a photo, layer by layer

Intermediate
Colorization

Adding fitting color to a black-and-white picture

Generative Beginner
Computer Vision

Technology that gets a computer to make sense of photos and video

Basics Beginner
Confidence Score

The number a model attaches to how sure it is about its own answer

Beginner
Convolution

Sliding a small tile across a photo to score it, spot by spot

Intermediate
Denoising

Removing only the flecks mixed in, leaving the original untouched

Generative Intermediate
Depth Estimation

Working out near and far from a single flat photo

Intermediate
Face Detection

Drawing a box around every face in a photo

Beginner
Face Recognition

Comparing faces to work out who someone is

Beginner
Feature Map

A picture that lays scan results out in their original spots

Advanced
Filter / Kernel

A small value tile that carries what to look for

Intermediate
Gaussian Splatting

A 3D scene held as millions of scattered color blobs

Generative Advanced
Gesture Recognition

Reading a hand or body movement as one predefined meaning

Beginner
Hand Tracking

Following hand joint locations from one moment to the next

Beginner
Image Captioning

Looking at a picture and writing what's in it as a sentence

Generative Intermediate
Image Classification

Putting one label on an entire photo

Beginner
Keypoint

A point that pins down one location in an image

Intermediate
Landmark

A numbered reference point at a fixed spot

Intermediate
Lip Sync

Fitting mouth shapes to the beat of a sound

Intermediate
Motion Capture

Turning body movement into a record of point locations

Beginner
Music Generation

Turning a few lines of request into a finished piece of music

Generative Beginner
NeRFNeural Radiance Field

A 3D scene that answers with light for any spot and direction

Generative Advanced
Object Detection

Finding every object in a photo and boxing each one

Beginner
Object Tracking

Keeping the same number attached to the same thing over time

Intermediate
OCR

Reading the text in a picture and turning it into characters

Beginner
Pattern Recognition

Spotting the regularity that keeps repeating

Basics Beginner
Pitch Detection

Pinpointing how high or low a sound is

Beginner
Pooling

Shrinking a picture by keeping one representative value per region

Intermediate
Pose Estimation

Marking joint locations with points to work out a body's pose

Beginner
Real-Time Inference

Finishing a judgment within a fixed window, every single moment

Intermediate
Resolution

The number of dots a picture is made of

Generative Beginner
Segmentation

Putting a label on every single pixel of a photo

Intermediate
Sound Classification

Sticking one label on a short clip of sound

Beginner
Source Separation

Splitting sound that's been mixed together back into its parts

Intermediate
Speaker Recognition

Telling who spoke from the sound of their voice

Intermediate
Spectrogram

A picture drawn so you can see sound with your eyes

Intermediate
Speech Recognition

The technology that transcribes spoken words into text

Basics Beginner
STTSpeech-to-Text

The conversion that turns sound into written words

Beginner
TTSText-to-Speech

The technology that reads text out loud in a voice

Beginner
Upscaling

Enlarging a picture by inventing the detail it never had

Generative Beginner
Vision Transformer

Cutting a photo into pieces and treating it like text

Advanced
Vocoder

The part that turns a sound blueprint into an actual waveform

Advanced
Voice Assistant

An AI you speak commands to, and hear answers from

Tools Beginner
Voice Cloning

Imitating a person's voice from a short recording

Intermediate