WORKSHOP2D2
A program card for the Music Thing Modular Workshop System Computer that listens to a live voice and improvises its own electronic "droid language" alongside it in real time — continuous chirps, trills, glides and glitches. Not a vocoder, not Auto-Tune, not triggered sample playback.
Built for a theatre production where an actor's live voice needs a synthetic vocal "shadow" performing with them, not a fixed sound-effect cue.
Status: builds clean, algorithm validated against real speech/singing recordings on the bench (see tools/). Not yet flashed and played on hardware.
What it does
SPEECH mode — free-ranging electronic chatter that follows the amplitude envelope of the incoming voice (loud when the voice is loud, quiet in its gaps). A five-state gesture generator (Idle, Stepping, Sweeping, Trilling, Glitching) keeps the texture continuously animated rather than triggering discrete blips. Detected onsets (consonant/syllable attacks) perturb whatever gesture is already running — new target pitch, a glitch burst, a direction change — rather than restarting the sound from silence.
SING mode — same chattering character, but the wandering pitch is attracted to (not locked to) the voice's own detected sung note, snapped to the nearest degree of a selectable scale. The card still glides and glitches around the note instead of doubling it cleanly.
Envelope following, onset detection, and pitch tracking are all computed by the card itself from raw audio on Audio In 1 — no external envelope follower or pitch-to-CV module required.
Controls
- Main — CHARACTER: how extreme each gesture is once it fires (jump size, vibrato depth, glitch harshness).
- X — REGISTER: the pitch range the voice wanders within.
- Y — ACTIVITY: how often something new happens (transition/glitch rate).
- Switch — Middle = SPEECH, Up = SING (the toggle's two stable positions). Down is spring-loaded/momentary and cycles the SING-mode scale (chromatic / major / minor / minor pentatonic) on each press.
- Pulse Out 1 — fires on each detected onset. Pulse Out 2 — fires on each gesture-state change.
- CV Out 1 — current target pitch (nearest MIDI note). CV Out 2 — a copy of the envelope level.
- LEDs — 0 envelope level, 1 pitch voiced/confidence, 2 gesture state, 3 onset flash, 4/5 SPEECH/SING mode.
Signal path
Bring a mic'd voice to Eurorack level and patch it into Audio In 1. The card's generated voice comes out on Audio Out 1/2 (duplicated mono) — mix that live against the real voice downstream.
Building
Requires the Raspberry Pi Pico SDK (see pico_sdk_import.cmake).
mkdir build
cd build
cmake ..
make
Development tools
tools/pitchsim.py and tools/dspsim.py are Python prototypes/regression checks for the fixed-point DSP — the pitch tracker in particular was validated against real voice recordings there before being ported to C++. See their docstrings for what was tested and what bugs that process caught.
License
CC BY 4.0 — see LICENSE.
