Gesture Synth is a free, browser-based musical instrument that transforms your webcam into a living chord pad and theremin. It uses MediaPipe Hand Landmarker — a Google open-source machine learning model compiled to WebAssembly — to track your hands entirely on-device. No controller, no install, no account, and no video ever leaves your computer.
Key Features
Two Hands, Two Roles — Your left hand selects chords from the I · vi · IV · V · ii progression (the backbone of countless pop songs), while your right hand controls volume, tone, and dynamics in real time.
- Two Instruments, One Tool — Switch between Gesture mode (continuous chord pad) and Theremin mode (free-form pitch from ~65 Hz to 1200 Hz) without leaving the app.
- Three Sound Scenes — Choose from Pop Pulse, Soft Ballad, or Retro Night to completely change the character of the same gestures, with instant switching and no reload.
- Expressive Gesture Mapping — Finger counts select chords, wrist tilt flips major/minor and sweeps the filter, hand height sets volume, and a fist mutes. Right-hand finger patterns switch between triads, inversions, and 7ths, with an extended thumb dropping the octave.
- Privacy-First Architecture — All hand tracking runs locally as WebAssembly; video frames are processed and discarded immediately. No upload, no telemetry, no cloud.
Use Cases
- Music creation & jamming — Play chord progressions and melodic lines with nothing but your hands and a webcam.
- Learning music theory — Explore the I-vi-IV-V-ii progression and chord voicings intuitively through physical gestures.
- Live performance — Use the theremin mode for expressive, free-form pitch control in a browser-based setup.
- Accessible music making — A zero-install, no-hardware entry point for anyone curious about gesture-based instruments.
Technical Highlights
- Runs on desktop browsers (Chrome and Edge); mobile support is on the roadmap.
- Free forever — no subscription, in-app purchases, or upsells.
- A re-skinned version of an original open-source tool by Ethan and Eric, with legal pages (Privacy, Terms) and contact available.
