Session Wrap-Up — Vision Kit Live Demos & Stage 1 Complete

Quick-reference summary so nothing needs to be re-learned next time

The short version

• Reassembly survived — camera still detected fine after being put back in the enclosure.
• SSH access briefly broke over .local hostname resolution, not the Pi itself — USB gadget mode and connecting via raw IP both worked as fallbacks.
• Face detection and joy scoring run live on-device at ~8–10 fps, correctly tracking a real-time joy score.
• Object detection works, but only on a downscaled image — a full 8MP photo reliably triggers the Linux OOM-killer on the Pi Zero's 512MB RAM.
• Phase 1 (Voice + Vision foundations) is complete; Phase 2 (SO-ARM101 arm) is next, with no blockers.

After the camera fix (new 8MP V2 camera + gpu_mem bump + disabling the background joy-detection service), this session was about actually running the Vision Kit's live demos.

1. Reassembly survived ✅

Put everything back in the cardboard box. Camera still detected fine afterward:

vcgencmd get_camera
supported=1 detected=1

Confirms the earlier fix was solid, not a fluke.

2. Reconnection hiccup (network, not hardware)

Lost SSH access after being away for a while — .local hostname resolution stopped working (Unknown host), even though the Pi was genuinely online with a valid WiFi IP. Root cause was never fully pinned down (likely laptop/mDNS flakiness after network changes — the Pi even briefly appeared on a different subnet, 192.168.1.x instead of 192.168.2.x).

Practical fixes that worked:

• USB gadget mode as a reliable fallback — plug the Pi Zero's USB (not PWR IN) port into the laptop, wait ~30–60 sec, connect via ssh pi@aiy-vision.local over that link. Works even while normal power stays plugged in.
• Connecting via raw IP (ssh pi@192.168.1.85) when .local refuses to resolve, even over plain WiFi.

Still unresolved / worth revisiting sometime: why .local mDNS resolution intermittently fails on this network. Not urgent — the two workarounds above are reliable enough for now.

3. Face detection — working ✅

cd ~/AIY-projects-python/src/examples/vision
python3 face_detection_camera.py

Live output, ~8–10 fps, correctly detected 1 face and tracked a real-time "joy score" — score jumped from ~0.02 (neutral) to 0.97–0.99 when smiling. Genuinely well-calibrated, running entirely on the Bonnet's own chip.

4. Object detection — working, with a memory gotcha ✅

python3 object_detection.py --input <file> --output <file> --threshold 0.2

• On the full 8MP photo: process got killed by the Linux OOM-killer (dmesg confirmed it — Pi Zero only has 512MB RAM, and 256MB is already reserved for GPU, leaving little headroom for decoding a large image in Python).
• Fix: capture a smaller image first (raspistill -w 640 -h 480 -o small.jpg), then run detection on that instead. Completed fast, correctly drew a bounding box around a laptop in frame.
• Takeaway for future testing: always feed object_detection.py a low-resolution image on this board — full 8MP photos will reliably crash it.

Current project status

✅ Phase 1 — Foundations: COMPLETE

• Voice Kit: assembled, WiFi working, Google Cloud speech recognition working (cloudspeech_demo.py transcribes live speech)
• Vision Kit: assembled, WiFi working (own hostname aiy-vision), camera replaced (original sensor died after 9 years), face detection + joy scoring + object detection all confirmed working locally on-device

🔜 Phase 2 — Arm Basics: NOT STARTED YET

Plan: assemble the SO-ARM101 leader/follower pair over the coming weekend. No blockers — parts are on hand, this phase was designed to be independent and can start anytime.

⏳ Later phases (unchanged from original roadmap)

1. Record leader→follower demonstrations
2. Add vision during demonstrations
3. Add voice as a trigger
4. (Stretch) train an imitation-learning/VLA policy off-device

Reference documents so far

• AIY Voice Kit setup guide
• AIY Vision Kit — initial troubleshooting log (WiFi, hostname collision, USB gadget mode)
• AIY Vision Kit — camera replacement & fix (dead camera, GPU memory, background service conflict)
• Voice + Vision + Arm roadmap (phased plan, parts list, hardware notes) — see the full roadmap
• Final build report — full status and honest retrospective
• This document — live demo session + status check-in