The hosted edition of Gesture Synth Weld. Play the whole instrument in your browser with your hands — free to play, no download, no plugin. Pro is what takes the work out: realtime MIDI into Ableton & co, 1080p watermark-free takes, a commercial licence and shareable cloud pages. FAQ ↓
Gesture Synth Weld is a two-hand instrument you play in front of a webcam. Your left hand picks the harmony — key, scale degree, major or minor — and your right hand shapes it: chord type, volume, tone, octave. Hand tracking runs locally with MediaPipe, so the camera feed never leaves the machine it is played on.
There are two editions of the same instrument, and this site is the hosted one. Pro is the edition on top of it, built for people who want to keep what they make: realtime MIDI out (bed, melody, glide bends, drums and bass into Ableton & co), 1080p takes without a watermark, a commercial licence, the full graded songbook and cloud work pages you can send to someone. Playing the instrument — including 120-second local recording and two guided pieces — stays free.
The MIT-licensed build lives at gesturesynthweld.com; it is the same instrument without the hosted parts. Pick whichever fits: the free build if you want the source, this one if you want your takes in a DAW.
The free tier is the instrument. Pro is everything that takes the playing out of the browser and into your own work — one plan, everything in (see pricing):
| What | What it gets you |
|---|---|
| 🎹 Realtime MIDI out | bed, melody, glide bends, drums and bass straight into Ableton & co — play the gestures, record the notes |
| 📼 HD recording | 1080p at a higher bitrate, same 120 seconds |
| 🚫 No watermark | clean takes with no brand marks baked in |
| 📜 Commercial licence | takes you make while subscribed are yours to use commercially, permanently — even if you cancel later |
| 📚 Full song library | the whole guided songbook (the free tier has the first two pieces) |
| 🎯 Follow-along scoring | play with the guide and get graded on timing and pitch |
| ☁️ Cloud work pages | long-lived pages you can send to someone; delete one to free its slot |
| 🎵 Songs on your account | your saved setlist follows your account — sign in on another machine and it is already there |
Everything you can reach with your hands is already in key — no wrong notes to hit. The gesture playbook below lists every pose the instrument answers to.
Click "Enable Camera" and grant camera permission. Your video stays on your device — nothing is uploaded.
Choose a key (C–B, default A). 1–5 fingers select scale degrees I–V. Index+Pinky = VI, +Thumb = VII. Default mode is locked to the key’s own chords (Diatonic Major) — switch to Scale+Tilt in settings to control major/minor with wrist tilt.
Height = volume, tilt = tone, thumb out = one octave down. Both hands required for sound — a left fist mutes it. Closing the right hand holds the octave where it was; it does not bring it back.
All gesture combinations for Gesture mode. Both hands required — left hand sets harmony, right hand controls expression. Quick start ↑
Here's how to use Gesture Synth Weld as a hand gesture music synthesizer — every gesture, mapped:
Your left hand picks the chord — each finger count is a different scale degree:
| Fingers | Chord | Gesture | Example (A Major) |
|---|---|---|---|
| 1 | I | Any 1 finger raised | A |
| 2 | ii | Any 2 fingers (not VI combo) | b |
| 3 | iii | Any 3 fingers (not VII combo) | c# |
| 4 | IV | Any 4 fingers raised | D |
| 5 | V | All 5 fingers raised | E |
| VI | vi | Index + Pinky ONLY (middle/ring/thumb must be down) | f# |
| VII | vii° | Index + Pinky + Thumb ONLY (middle/ring must be down) | g#dim |
| 0 (Fist) | — | All fingers curled in | Mute (notes held, resume when fingers return) |
Default is Scale notes only (Diatonic Major) — wrist tilt is off until you switch to Scale+Tilt in Settings. There is a small dead zone around level (±0.05), so a hand held flat does not flip back and forth.
| Wrist Action | Result | Notes |
|---|---|---|
| Tilt Left (tilt < 0) | Minor | All chords become minor (I→i, IV→iv, V→v, etc.) |
| Tilt Right (tilt ≥ 0) | Major | All chords become major (I, IV, V, etc.) |
| Scale notes only (lock mode) | Fixed | Mode locked (default: Diatonic Major) — ignores wrist tilt |
Your right hand shapes the sound — chord type, volume, tone, and octave:
| Fingers | Major Chord Type | Minor Chord Type |
|---|---|---|
| 1 | Major Triad | Minor Triad |
| 2 | Major 1st Inversion | Minor 1st Inversion |
| 3 | Major 7th | Minor 7th |
| 4 | Major 9th | Minor 9th |
| 0 (Fist) | Sound continues — a closed hand is not read as a chord type, and it does not bring the octave back | |
Right Hand Mode is Finger Layout by default. But in camera play the chord type is decided before this table gets a turn: your left hand’s height picks the voicing (triad → 7th → 9th → 13th, low to high), and a Chord Matrix cell can pin a quality per degree. The finger layout above is the fallback for when no height can be read — the keyboard path. To pin one style for a degree instead, use the Chord Matrix (a pinned cell overrides the tier); on the keyboard path, Right Hand Mode = Fixed Chord Style locks one of 12 types: Triad, 7th, 9th, 13th, Major Triad, Major 1st Inv, Minor Triad, Diminished, Sus2, Sus4, Major 7th, Dominant 7th.
Height controls volume; wrist tilt sweeps the tone filter:
| Movement | Controls | How It Works |
|---|---|---|
| Hand Height (Y-axis) | Volume | Hand higher = louder. Hand lower = quieter. |
| Wrist Tilt (left/right) | Filter Sweep / Tone | Tilt right = brighter (filter opens). Tilt left = darker (filter closes). |
Thumb out drops the octave for low bass lines:
| Thumb Position | Result | Status Display |
|---|---|---|
| Thumb curled in (default) | Normal octave | The chord name alone, e.g. A |
| Thumb extended out | Down one octave | An amber 8vb badge appears beside the chord name |
The reading only updates while the hand is open — closing the right hand keeps the octave where it was. Not used in Sculpt or FX, where the thumb is a different control.
Ways to stop or mute the sound:
| Condition | Behavior |
|---|---|
| Left fist (0 fingers) | Music mutes — notes held, resume when fingers return |
| Right fist (0 fingers) | Sound continues and the chord keeps updating — the octave stays where the thumb left it |
| One hand leaves frame | Music holds for 1.8s, then stops — a hand slipping out of view for a moment does not cut the sound |
| Both hands leave frame | Same 1.8s hold. Turn Stage → Hold values when hands leave off to cut it to 80ms instead |
Both hands required — the left hand picks a single note (interval + key), the right hand triggers it and controls volume. Left fist = silence.
| Left Fingers | Interval | Example (Key A) |
|---|---|---|
| 1 | Root note | A4 |
| 2 | Major 3rd | C#5 |
| 3 | Perfect 5th | E5 |
| 4 | Octave | A5 |
| 5 | 9th (octave + 2nd) | B5 |
Both hands required — right hand height = pitch (C3 → C6, logarithmic), left hand height = volume. Raise your right hand for higher notes; raise your left hand for a louder sound.
Everything you can fine-tune in the Settings panel:
| Control | Options | Notes |
|---|---|---|
| Key | 12 keys (C – B) | Toolbar dropdown — transposes every chord |
| Left Hand Mode | Scale+Tilt · Scale notes only | Scale+Tilt: wrist tilt flips major ↔ minor (default: Scale only, Diatonic Major) |
| Right Hand Mode | Finger Layout · Fixed Chord Style | Finger Layout is the default: 1–4 fingers = triad / 1st inv / 7th / 9th — see the note above for when it applies |
| Left Hand Height (Y-axis) | Triad → 7th → 9th → 13th | Raise the left hand to thicken the chord — four spice levels. In camera play this is what picks the chord type |
| Fixed Chord Style | 12 types | Triad · 7th · 9th · 13th · Major Triad · Major 1st Inv · Minor Triad · Diminished · Sus2 · Sus4 · Major 7th · Dominant 7th |
| Chord Voice | pad · keys · hollow · aero | Settings → Sound. Each mode has its own voice underneath: Gesture is a sawtooth pad, Theremin a sine |
| Arpeggiator | Off · On | The ⟿ button in the top toolbar (it has no row in Settings) — sweeps chord notes like a harp. Rate, direction and octaves live in Settings → Sound · Sculpt |
| Auto Bass | Off · On | The ∿ button in the top toolbar — a generated low end that follows your harmony, with its own volume |
| Metronome | BPM 40–240 · 3/4·4/4·5/4·6/8·7/8 · 1–16 bars · Click/Wood/Beep/Hi-hat · TAP · volume | The ♪ button in the bottom bar (it has no row in Settings). Note: the metronome is audible in recordings when it's on |
| Shortcuts | Space = stop all notes · Esc = reset | Work at any time (browser focus) |
Don't have a webcam — or just don't want to use one? Turn on Settings → No camera? Keyboard mode (desktop) — or flip the camera / keyboard switch in the top toolbar — and play the synth with your physical keyboard. No camera permission, no model download, nothing is captured. Flipping the same switch back returns you to hand gestures.
| Key | What it does |
|---|---|
| 1 – 7 | Scale degrees I – VII — hold to play, release to stop (like a keyboard instrument) |
| [ / ] | Minor / Major mode |
| 8 / 9 / 0 / - | Chord style: triad · 1st inversion · 7th · 9th |
| Shift | Octave down (8vb) |
| T | Transpose up a semitone |
| ↑ / ↓ | Volume sweep (hold) |
| ← / → | Filter sweep (hold) |
| Space | Stop all notes |
The first time you enter keyboard mode, an interactive guide shows a real QWERTY keyboard with a demo animation — press the keys yourself and they light up with what each one plays. You can replay it anytime from the ? help panel. Recording in keyboard mode captures audio (including your voice if you enable the mic) — there's no camera feed to record as video.
The default settings out of the box:
| Setting | Default | Description |
|---|---|---|
| Key | A | All chords transposed to A major scale (can change via toolbar dropdown) |
| Left Hand Mode | Scale Only (Diatonic Major) | Fingers pick the scale degree; the mode stays on the key’s own chords |
| Right Hand Mode | Finger Layout | Right hand: height = volume, tilt = tone, thumb = octave |
| Arpeggiator | Off | Toggle ⟿ to sweep chord notes sequentially |
| Auto Bass | Off | Toggle ∿ for automatic low-end root note |
Usually the browser has no camera permission for this site. On Windows that is Settings → Privacy & security → Camera (both "Camera access" and the entry for Chrome/Edge must be on). On macOS it is System Settings → Privacy & Security → Camera, with your browser ticked. On iPhone/iPad the same switch lives in Settings → Privacy & Security → Camera for Safari, and on Android it is Settings → Apps → Chrome → Permissions → Camera. Also close anything else holding the camera (Zoom, Teams, FaceTime) — one app at a time.
A browser remembers a block and stops asking. Re-open it from the address bar: the lock/camera icon → Camera → Allow, then reload. In Firefox the same control is behind the permissions icon next to the address. On iOS Safari, check the per-site settings under the "aA" menu, or Settings → Safari → Camera.
Put light in front of you rather than behind — a bright window at your back turns your hands into silhouettes. Step back until your elbows fit in the frame, and keep both hands visible: this instrument needs two hands to make a sound. Tracking runs on the GPU, so close other GPU-heavy apps and keep the laptop on mains power if it is throttling.
Browsers refuse to start audio until you have interacted with the page — press the start button first. Then check the tab is not muted and the system volume is up. If you are in camera mode, remember both hands have to be in the picture with at least one finger raised; a lone hand is deliberately silent.
Web MIDI exists in Chrome and Edge only — Firefox and Safari have no implementation, so the switch is not shown there. Open Settings, turn MIDI out on, then pick the port in your DAW. If the port list is empty, no virtual MIDI device is installed yet (on macOS create one in Audio MIDI Setup; on Windows install a virtual port). Details per control are in the guide.
Sign in from the account panel — it also has the link that opens the billing portal, where you can change plan or cancel. Cloud work pages take a slot each; deleting one frees it. If something is wrong with a charge or a page, write to support@gesturesynthweld.app.
| Name | What it means | What to do |
|---|---|---|
| NotAllowedError | permission was refused | re-allow it from the address bar, then reload |
| NotFoundError | no camera on this machine | plug one in, or use keyboard mode |
| NotReadableError | another app is holding the camera | quit it, then retry |
| NotSupportedError | the page is not on HTTPS | use https://gesturesynthweld.app |
The same instrument, in its hosted edition. gesturesynthweld.com is the MIT-licensed build you can read and run yourself; this site adds an account, cloud work pages and the optional Pro plan on top of the playing. If you want the source, take the other one; if you want your takes in a DAW, take this one. The MIT-licensed build →
Realtime MIDI out (chord bed, melody, glide bends, drums and bass) straight into your DAW; 1080p takes with no watermark; a commercial licence on anything you record while subscribed; the full guided songbook with follow-along scoring; and cloud work pages you can send to someone. See pricing →
Playing is free: the whole instrument, 120-second local recording, .mid export and the first guided pieces, with no account and no ads. Pro is the optional plan for taking the work out of the browser. See pricing →
Yes — up to 120 seconds, as audio, full video, or a skeleton animation that never shows your camera feed, in 9:16, 16:9 or 1:1. The free tier keeps the file locally; Pro records at 1080p without a watermark. Read the guide →
Yes. Keyboard mode turns your keyboard into the instrument: 1-7 pick scale degrees, [ and ] switch minor/major, 8 9 0 - pick chord styles, the arrow keys ride volume and the filter, Space stops everything. No camera permission and no model download. Read the guide →
From the account panel: it opens the billing portal, where you can switch between monthly and yearly or cancel. Cancelling does not take back what you already exported, and the commercial licence on takes made while subscribed stays valid. Read the guide →
The two-hand split: the left hand picks the harmony (scale degree and quality), the right hand shapes it (chord type, volume, tone, octave), and everything the chord hand can reach is already in key. A webcam replaces the hardware — no gloves, no sensors, nothing to install. Read the guide →