PROJECT / 11 FEATURE ARTICLE
Local VoiceAssistant
A completed local voice experiment built around a single turn-based speech loop.
01 Local inference loop
Speech in, speech out
- 01Record
- 02Whisper
- 03Qwen
- 04Kokoro
- 05Playback
02 Project overview
A completed turn-based macOS experiment
Local Voice Assistant was a single-user macOS experiment that connected microphone input, local transcription, a local language model, and speech playback in one turn-based loop. Pressing Return started recording; pressing it again ended the utterance.
The terminal printed the recognized speech and the assistant response, then played the synthesized reply. An earlier retained browser/server experiment explored a lower-latency direction; the archived formal entrypoint is a local command-line application.
03 Recorded implementation
Local model runtimes joined by a Python CLI
The archived CLI recorded 16 kHz mono audio with sounddevice and wrote temporary audio under the project directory. mlx-whisper transcribed it locally with a Whisper small model. llama-cpp-python then ran a local Qwen2.5-3B GGUF model using the system instruction and current utterance.
MLX-Audio/Kokoro synthesized the reply. The app selected a Mandarin voice when Chinese characters appeared and an English voice otherwise, then played the generated audio with afplay. If Kokoro failed, it fell back to macOS say.
The inference path ran locally once model assets were available; setup could download those assets when needed. This one-turn experiment did not implement conversation history, durable memory, voice activity detection, interruption handling, or realtime audio streaming.
04 One interaction turn
Record, transcribe, respond, play
- 01
Start recording
Pressing Return in the terminal started recording from the microphone.
- 02
Stop recording
Pressing Return again finished the utterance and saved temporary local audio.
- 03
Transcribe and respond
Local Whisper produced text, then the local Qwen model received that utterance and returned a concise reply.
- 04
Synthesize and play
Kokoro produced an English or Mandarin spoken response for macOS playback; say was available if synthesis failed.
05 Observable output
A terminal trace and spoken reply
Recognized speech was printed in the terminal after local transcription.
The one-turn Qwen response was shown in the terminal before it was spoken.
Kokoro synthesis played through macOS audio, choosing a Mandarin or English voice from the reply text.
macOS say was available when Kokoro synthesis was unavailable or failed.
Recordings and generated response audio stayed in the project-local temp directory.
The archived experiment was a local terminal interaction, without a public web interface or persistent conversation view.