Working on multimodal embodied AI forces to you learn a lot, about a lot of stuff. Focusing on the "hearing" subsystem for #BozBox right now, so having to spend a lot of time on signal processing stuff that I don't know much about. Learning everything from ALSA, JavaSound API, etc. through audio file formats (PCM, WAV, etc.), to spectral analysis, FFT, DFT, and so on. And that's just to front-end the work on actual speech recognition. Wheeew!!!

#norestforthewicked

One other angle I may pursue for #BozBox is giving it the ability to sense the state of its onboard power supply (when not plugged into the wall) so it will have something on which I can base a notion of something analogous to the "hunger" that we humans experience.

Will that prove to be important, or even useful? No idea, but I'm leaning towards implementing it nonetheless.

So far #BozBox captures motion data using a 9DOF accelerometer/magnetometer/gyroscope board, receives coarser grained motion and location data from a GPS receiver, and now is just about to start capturing audio.

Vision will be next after that, and then some random things that I might or might not do: light sensors, barometric pressure, temperature, etc.

So, what *is* this #BozBox thing anyway, you might be asking.

Simply put, it's a "box of sensors" that I can carry around that will experience the physical world with those various sensors, and save multi-modal data about its experiences, so I can play with ideas about how an AI (or possibly a human) learns a "naive epistemology of the world", "intuitive physics of the world", language usage, naive epistemology, etc.

Making progress on the #BozBox project, in terms of receiving audio input. Starting to understand how #ALSA works. That's something I *never* thought I'd say!

The tricky part is working with two microphones so we can have stereo hearing, analogous to human hearing.

The second tricky part is getting the sound into #Java code. Use JNI? Or maybe use plain old arecord and a shared memory-mapped file? Or "other"? Hmm...

#Linux
#AI

What's going on today? Got about 8 miles in on the mountain bike at Carolina North Forest, now sitting at Barnes & Noble in Durham working on #BozBox our "box of sensors" project for experimenting with #EmbodiedAI.