AIC (AI Companion)
A local-first multimodal agent runtime combining voice, vision, memory, tools, and an embodied real-time interface across Apple Silicon and NVIDIA Jetson.
Overview
AIC is a local-first multimodal agent runtime I designed and built for real-time, embodied interaction. It coordinates voice, vision, persistent memory, local or cloud-optional reasoning, and bounded tool use through typed events and a central conversational state machine. The runtime has a fully offline Apple Silicon path and a separate NVIDIA Jetson deployment profile rather than treating cloud inference as a hidden requirement.
The Problem
Most AI assistants are still browser-bound cloud chat surfaces. Building a companion that can perceive, speak, remember, use tools, animate in real time, and survive day-to-day operation requires a coherent runtime across models, native rendering, edge hardware, state, supervision, and failure recovery.
The Solution
Built an event-driven Python runtime around Redis, local ASR and LLM inference, TTS, wake word, vision, emotion recognition, memory, and tool execution. A hardware-accelerated C++ and Filament interface runs on Metal or Vulkan with lip sync, face tracking, gaze, posture, and speech-reactive animation. Health telemetry, process supervision, deployment automation, deterministic asset validation, and end-to-end tests make the system operable rather than a one-off demo.
Key Results
Fully offline Apple Silicon path
Separate NVIDIA Jetson profile
Typed Redis event contracts
Hardware-accelerated embodied interface