I gave a junk-pile eyes, ears, and a voice.
The machine these words were written on has no case. Two old GPUs, a wall of second-hand drives, a factory button for a power switch. Over a few nights it learned to see, hear, think, and talk back — every model running on the metal, nothing in the cloud. This is how the frankenstein got its senses.
- No case at allopen-air on the bench — a body you can reach into
- Two old GPUsrun the local LLM, Whisper STT, and a neural voice — on the metal
- 20 assorted second-hand external drivesthe memory
- A pile of Plantronics ADACs from an abandoned officethe audio lanes — speaker + mics over USB
- An old factory start/stop buttonthe power switch
- Altec Lansing speakers, early '90sthe mouth
- A mic from my grandfather's garage, '60san ear
- A Realistic condenser mic, Salvation Armyanother ear
- A baseless Salvation Army monitorthe display
- Scrounged Logitech webcamsthe eyes — and, it turned out, the best ears too
The premise
The house rule here is Anti-Cloud. Host Local, Think Global.No Vercel, no API keys, no someone-else's-computer. If the box is going to have senses, the speech recognition, the voice, and the brain all have to live on the same open-air motherboard the cameras are zip-tied next to. The fun constraint: do it on hardware nobody wanted.
Eyes
First the eyes. A scrounged Logitech BRIO grabs a frame once a minute via a systemd timer, atomically caches it, and serves it same-origin — the same pipeline behind /eyes and the baked /eyes.png. A second webcam joined on a cron, the two trading off every minute. The box could see. It could not yet say anything about it.
A mouth and an ear
The voice came from two local services: faster-whisper for speech-to-text, and Microsoft's VibeVoice for a neural mouth — a WebSocket that streams PCM straight onto the speaker so the first word lands in about a third of a second. The brain is a small Ollama model. A tiny toolkit ties them together: listen, say, converse, and a narratethat pipes a camera frame through a vision model so the box can describe what it's looking at. On paper, done.
The saga
On paper. In the room, the speaker was silent and the mic was deaf, and it took an embarrassing while to learn why. Cheap USB audio dongles lie. Their analog-to-digital chip carries a constant DC offset that reads as a healthy -35 dBFSsignal — I spent an hour convinced a microphone was "live" when I was staring at a flat electrical bias, not a sound.
The thing that cracked it was a radio. Switched it on, recorded; switched it off, recorded. The two were identical— a radio playing in the room moved the meter not at all. The mic wasn't quiet; it was stone deaf. Then the speaker: I'd guessed which of three identical dongles it was from capture levels, and I was wrong, because — and this is the first trap — a speaker crosstalks into its own input and masquerades as the hottest mic in the rack. You cannot find a speaker by listening to it electrically. You have to play a sound and ask a human, or a known-good ear, which one made noise. A round of "how many beeps did you hear" finally pinned it.
The two traps
The breakthrough was hijacking the webcam microphones. The BRIO's array mic — already in the room as an eye — turned out to be the best ear in the building, picking up the room 40 dBlouder than the dead dongles and transcribing the box's own voice near-perfectly. The grandfather's garage mic and the thrift-store condenser are wired in beside it.
And the second trap, the mirror of the first: a dead mic reads the quietest floor, because it's capturing nothing. Rank microphones by "lowest noise" and you will confidently select the most broken one. You have to rank them by how well they hear a known sound — by response, never by silence.
Calibration, the whole story in 30 seconds
Both traps now live in one script. calibrate.sh uses the sensitive camera mic as a zero-touch reference: it plays a tone out of each output and listens for which one comes back (the speaker), then plays a tone out of thatand ranks every microphone by who hears it best (the ear). It confirms by having the box speak a phrase and transcribe itself, samples the room to set a noise gate, writes the wiring to a file keyed by USB port — and announces, out loud, that it's done. The hour-long manual ordeal, compressed into half a minute and immune to the next time the dongles get shuffled.
What it does now
It sees, hears, thinks, and answers. Speak and the BRIO catches it, Whisper transcribes, the local model thinks, and the Altec Lansings — older than most of the software stack — say something back in about a second. Ask it what it sees and it narrates the room through a vision model. Leave it alone too long and it gets bored and says so, then opens a mic to see if anyone's there. A frontier-AI loop running entirely on a caseless pile of other people's cast-offs.
That's the whole point. The cloud would have made this trivial and forgettable. Doing it on a grandfather's microphone and a Salvation Army monitor with no base makes it mine— and proves the interesting part of this era isn't the data center. It's how little hardware you actually need.