Gesture Synth

Gesture Synth is a free, browser-based musical instrument that transforms your webcam into a living chord pad and theremin. It uses MediaPipe Hand Landmarker — a Google open-source machine learning model compiled to WebAssembly — to track your hands entirely on-device. No controller, no install, no account, and no video ever leaves your computer.

Key Features

Two Hands, Two Roles — Your left hand selects chords from the I · vi · IV · V · ii progression (the backbone of countless pop songs), while your right hand controls volume, tone, and dynamics in real time.

• Two Instruments, One Tool — Switch between Gesture mode (continuous chord pad) and Theremin mode (free-form pitch from ~65 Hz to 1200 Hz) without leaving the app.
• Three Sound Scenes — Choose from Pop Pulse, Soft Ballad, or Retro Night to completely change the character of the same gestures, with instant switching and no reload.
• Expressive Gesture Mapping — Finger counts select chords, wrist tilt flips major/minor and sweeps the filter, hand height sets volume, and a fist mutes. Right-hand finger patterns switch between triads, inversions, and 7ths, with an extended thumb dropping the octave.
• Privacy-First Architecture — All hand tracking runs locally as WebAssembly; video frames are processed and discarded immediately. No upload, no telemetry, no cloud.

Use Cases

• Music creation & jamming — Play chord progressions and melodic lines with nothing but your hands and a webcam.
• Learning music theory — Explore the I-vi-IV-V-ii progression and chord voicings intuitively through physical gestures.
• Live performance — Use the theremin mode for expressive, free-form pitch control in a browser-based setup.
• Accessible music making — A zero-install, no-hardware entry point for anyone curious about gesture-based instruments.

Technical Highlights

• Runs on desktop browsers (Chrome and Edge); mobile support is on the roadmap.
• Free forever — no subscription, in-app purchases, or upsells.
• A re-skinned version of an original open-source tool by Ethan and Eric, with legal pages (Privacy, Terms) and contact available.