NiftyApps
Logo of Gesture Synth

Gesture Synth

A free browser instrument that turns your webcam into a chord pad and theremin — no install, on-device hand tracking.

Introduction

Gesture Synth is a free, browser-based musical instrument that transforms your webcam into a living chord pad and theremin. It uses MediaPipe Hand Landmarker — a Google open-source machine learning model compiled to WebAssembly — to track your hands entirely on-device. No controller, no install, no account, and no video ever leaves your computer.

Key Features

Two Hands, Two Roles — Your left hand selects chords from the I · vi · IV · V · ii progression (the backbone of countless pop songs), while your right hand controls volume, tone, and dynamics in real time.

  • Two Instruments, One Tool — Switch between Gesture mode (continuous chord pad) and Theremin mode (free-form pitch from ~65 Hz to 1200 Hz) without leaving the app.
  • Three Sound Scenes — Choose from Pop Pulse, Soft Ballad, or Retro Night to completely change the character of the same gestures, with instant switching and no reload.
  • Expressive Gesture Mapping — Finger counts select chords, wrist tilt flips major/minor and sweeps the filter, hand height sets volume, and a fist mutes. Right-hand finger patterns switch between triads, inversions, and 7ths, with an extended thumb dropping the octave.
  • Privacy-First Architecture — All hand tracking runs locally as WebAssembly; video frames are processed and discarded immediately. No upload, no telemetry, no cloud.
Use Cases
  • Music creation & jamming — Play chord progressions and melodic lines with nothing but your hands and a webcam.
  • Learning music theory — Explore the I-vi-IV-V-ii progression and chord voicings intuitively through physical gestures.
  • Live performance — Use the theremin mode for expressive, free-form pitch control in a browser-based setup.
  • Accessible music making — A zero-install, no-hardware entry point for anyone curious about gesture-based instruments.
Technical Highlights
  • Runs on desktop browsers (Chrome and Edge); mobile support is on the roadmap.
  • Free forever — no subscription, in-app purchases, or upsells.
  • A re-skinned version of an original open-source tool by Ethan and Eric, with legal pages (Privacy, Terms) and contact available.

Information

Categories

    Tags

    Newsletter

    Join the Community

    Subscribe to our newsletter for the latest news and updates