OrCam Hear

OrCam Hear is a pair of AI earbuds for people with mild hearing loss. They isolate the voices you choose and suppress background noise in real time. People control them from an iPhone app in the middle of a conversation, so the app has to work at a glance.

My part

I was the developer of the iOS app and also worked on the software on the earbuds' embedded device.

• Figma to Swift. Every screen built pixel-accurate, with motion, timing and edge cases worked out with the designers. Each UI component is a shared part in Swift, so new screens are assembled from the same pieces as the design library.
• Audio-driven animation. Animations react to live audio from the device, so users see who is being heard and that the earbuds are working. Fast, predictable gestures move a person into or out of the conversation.
• State and data. The app's state, and the data layer that talks to the device: enrolled voices, who is active and device status.

On the device

• Real-time C++ for the embedded device: multi-process, multi-threaded, under tight compute and power limits.
• Owned parts of the real-time audio-video flow: components that capture, process and deliver sound and image data.
• Integrated ML algorithms; built performance optimizations, usage analytics and internal tools.
• Coordinated development and testing with the vision, DSP, algorithms, embedded, mobile and QA teams.

What I built from

The interaction model: the room holds the people you're listening to, the balcony holds enrolled people who are waiting, and strangers stay outside. My job was to make that model feel immediate in the app.

Screens I built in Swift. Design by the UX and UI designers.

App icons

Alongside the apps, I designed app icon explorations for two OrCam products, OrCam Hear and OrCam MyMe: an eye and an ear merged into one mark, in a few directions and colorways. These were internal and did not ship.

My own designs, internal explorations.

Team: UX lead, UI designer, product manager, and me on the iOS app and device software.