Parley, Your AI Companion

The idea behind Parley was pretty simple. I wanted something I could talk to while I worked, something that could see me walk in the door, recognize me, remember things that matter, and actually help me get stuff done. If it could remember and grow with me over time, maybe it could start to feel less like a software void and more like a little productive, unconditional friend that does all my work for me. I also wanted to challenge myself and see how far I could take an idea at the edge of what was possible.

Once I had the vision in mind, I worked backward from what it needed to do. I used Claude and other AI tools to research the hardware and software together, checking model support, cameras, drivers, and firmware before ordering anything. The first build used an NVIDIA Jetson Orin Nano Super and an IMX708 camera. The camera did not work on JetPack 6.2 (turns out, minor versions do matter!) and a solution didn’t exist, so I figured out the kernel, device-tree, and color fixes and that’s how nvidia-jetson-toolkit was created.

The same runtime now works fully local on Apple Silicon too, so Parley no longer depends on its original hardware rig. The whole project has basically been me following the idea wherever it goes, learning new parts of the stack, and trying to build the little friend I imagined in the first place.

Most AI assistants are a chat box in a browser tab talking to somebody else's server. A companion you can look at has to hear you, answer out loud, watch the room, remember you, call tools, and animate a face in real time, all while running on the machine in front of you without a person babysitting the process list.

An event-driven Python runtime coordinates local speech recognition, reasoning, speech synthesis, wake word, vision, emotion recognition, memory, and tool calls over a Redis bus. The avatar is a C++ renderer on Filament, drawing through Metal on Mac and Vulkan on Jetson, with lip sync, face tracking, gaze, posture, and speech-driven body motion. Memory is a local vector store that a conversation summarizer writes facts back into. A macOS menu-bar app handles avatars end to end: drop in a model, it validates and bakes it, then swaps it into the running renderer. Startup runs a health gate that tears the whole stack back down if any service fails to come up, and the renderer is built and run under address and thread sanitizers.