Language
Selected project
iOS Guitar Chord Recognition App
An iOS app that recognizes live listening, arpeggiated, and fretboard-entered guitar chords with onset-aware audio analysis and programmatic music theory.
iOS Engineer · Audio DSP & Product Design
Live chord recognition · Part one
Six guitar strings vibrate almost at once, combining fundamentals, harmonics, and room noise into one complex microphone waveform. The challenge is not merely finding frequencies, but turning a short, changing signal into comparable and explainable pitch evidence.
I built an on-device DSP pipeline with SwiftUI, AVFoundation, and Accelerate, then added temporal stability and onset locking so frequent classifier updates do not make the interface flicker.
Signal pipeline
These are not placeholder charts. The in-app developer tools captured every stage from the same E-chord recognition run, preserving inputs, outputs, and intermediate evidence so the algorithm can be inspected rather than treated as a black box.
Inside the pipeline
Move through eight cards to see one waveform transform step by step. Standard signal processing includes formulas; classifier coefficients and stability thresholds that differentiate the product are shown only as design principles.
01 / 08
SIGNALAVAudioEngine receives the microphone signal with a 20 ms I/O buffer, averages input channels to mono, and requests 48 kHz measurement mode. The signed waveform is preserved while RMS provides the level and silence gate.
xₘₒₙₒ[n] = (1 / C) Σ xᶜ[n]
C is the input channel count; this snapshot measured −26.8 dBFS RMS.
02 / 08
SIGNALOne I/O chunk is too short to separate low guitar pitches clearly. Samples accumulate into an 8,192-point analysis window that advances by 2,048 points; 75% overlap balances frequency detail with update speed.
xₘ[n] = x[n + mH], N = 8192, H = 2048
Each window spans about 170.7 ms and produces a result every 42.7 ms.
03 / 08
SIGNALCutting a waveform creates discontinuities at both ends, which an FFT interprets as frequencies that were never played. A Hann window tapers the boundaries toward zero, reducing spectral leakage and exposing the real pitch peaks.
w[n] = ½(1 − cos(2πn / (N − 1)))
The FFT receives the windowed signal xʷ[n] = xₘ[n] · w[n].
04 / 08
SIGNALA vDSP FFT transforms the time waveform into frequency-domain energy. At 48 kHz, an 8,192-point FFT gives roughly 5.86 Hz bin spacing; this E-chord snapshot shows dominant peaks near 123, 164, and 246 Hz.
Xₘ[k] = Σ xʷ[n] · e^(−j2πkn/N)
Magnitude is |X[k]| = (2/N)√(Re² + Im²), ready for local-peak extraction.
05 / 08
MEANINGListeners perceive E notes across octaves as one pitch class. Frequencies are converted to MIDI pitches and folded into the twelve classes from C to B. Only local peaks above the noise floor remain, while high harmonics are gently down-weighted.
p(f) = 69 + 12 log₂(f / 440), class = p mod 12
After L2 normalization, energy at E, G♯, and B forms evidence for E major.
06 / 08
MEANINGA single frame is vulnerable to pick attack and momentary noise. The system keeps the last twelve chroma vectors and gives newer frames more weight, using roughly half a second of context without making the response feel sluggish.
c̄ = (Σᵢ₌₁ᴸ i · cᵢ) / (Σᵢ₌₁ᴸ i), L ≤ 12
E, G♯, and B remain dominant after smoothing before classification.
07 / 08
MEANINGEach possible root is matched against data-driven chord templates. The score considers similarity, out-of-template energy, and missing required tones—preventing a decaying E from becoming Bsus4 merely because both share E and B.
score = similarity − outside-tone penalty − missing-tone penalty
The portfolio explains the scoring model while withholding production weights, coefficients, and tuning values.
Key weights and thresholds omitted
08 / 08
DISPLAYA classifier result never writes directly to the UI. A candidate must remain stable and meet a confidence condition; replacing an established chord also requires a newer note onset, so decay is not mistaken for a new performance.
commit ⇔ stable ∧ sufficient confidence ∧ valid onset
The decision principle remains visible while production thresholds, hold times, and event tuning stay private.
E
Key weights and thresholds omitted
Debugging case
E major contains E–G♯–B; Bsus4 contains B–E–F♯. When G♯ decays sooner than E and B, those shared tones can make frame-by-frame classification drift. Raising the confidence threshold alone is not enough because a wrong candidate can also persist.
Beyond template similarity, the score penalizes missing required tones and out-of-template energy. Bsus4 lacks convincing F♯ evidence, so E and B resonance alone should not let it win.
The classifier may keep updating candidates, while the primary chord still needs sustained stability, sufficient confidence, and readable display time before the interface changes.
Each valid note attack creates a new generation. An established chord can only be replaced by a different name after a newer attack, so its tail is not mistaken for a new performance.
I added an automated test asserting that a chord cannot be replaced during decay without a new onset. Stability is not simply longer UI latency; it gives recognition updates a clear causal relationship to what the musician played.
Arpeggio recognition · Part two
A strum exposes several notes in one spectrum, while an arpeggio reveals them over time. The previous string may still resonate when the next one enters, so no single analysis frame contains the complete chord.
I reframed recognition as an event pipeline: detect a new pluck, estimate the pitch introduced by that event, then let the musician finish explicitly before classifying the notes collected across the phrase.
01
Track level rises and spectrum changes to divide continuous audio into musically meaningful note events.
02
Compare spectra before and after an onset so ringing notes are not mistaken for the newly plucked string.
03
Keep pitch and octave for readable feedback, then fold the sequence into twelve pitch classes for chord analysis.
Event-based pipeline
Microphone capture, frame buffering, and STFT reuse the live pipeline. This chapter expands only event detection, pitch selection, and note accumulation while keeping production thresholds and tuning coefficients private.
01 / 06
EVENTThe system continuously compares the current level with the recent past. Only events satisfying energy, rise, and timing conditions open a pitch-capture window, preventing room noise and one pluck's oscillation from being counted repeatedly.
onset ⇔ energy rise ∧ valid interval
The event model is visible; production energy thresholds, ratios, and debounce time remain private.
02 / 06
EVENTAfter each onset, a YIN-style periodic-difference analysis finds plausible fundamentals. This is more resilient to guitar harmonics than selecting the largest spectrum peak; candidates also pass tuning and confidence checks.
d(τ) = Σₙ (x[n] − x[n + τ])²
Lag τ represents a period, so pitch is approximately sampleRate / τ; production candidate thresholds are omitted.
03 / 06
PITCHThe hard part is not finding every sounding note, but isolating the note that just entered. The spectrum immediately before onset becomes a background snapshot, leaving only positive energy added afterward.
novelty[k] = max(0, current[k] − background[k])
Blue is current, gray is background, and red is new energy; the production background weight remains private.
04 / 06
PITCHEach candidate checks new energy near its fundamental and harmonics, then combines that evidence with pitch confidence. The selected note is not simply the loudest—it is the one that best explains this onset.
selected = arg max candidate novelty
The selection principle is public; harmonic count, weighting, and stability conditions remain implementation details.
05 / 06
CHORDEvery confirmed pitch joins the sequence in performance order with its MIDI pitch and octave intact, showing what the system heard. A mistaken note can be removed and restored without replaying the whole phrase.
sequence = [n₁, n₂, …, nₘ]
The test arpeggio collected E2, B2, E3, G♯3, B3, and E4 in order.
06 / 06
CHORDWhen the musician finishes, notes with the same name across octaves collapse into twelve pitch classes. E2, E3, and E4 all become E; repetitions cannot dominate the result before the shared chord classifier runs.
chroma[pitchClass(n)] = 1
E, G♯, and B form E major; chord-template and scoring weights remain in the protected shared core.
Product interaction
The current flow uses explicit controls to start and finish collection instead of guessing that a longer pause means the performance ended. Notes can be removed and restored during recognition; classification happens only after completion, keeping live and arpeggio recognition as clear product boundaries.
Current scope
This page now covers live strumming and event-based arpeggio recognition. The interactive fretboard, hands-free interaction details, and a fuller account of testing and limitations will follow in the next phase.
Next project