Brilliant Sole

Local AI Assistant on Smart Glasses

Using our wearables SDK to take a picture, speak a prompt with Whisper’s speech to text, and use chrome’s Prompt API to generate a response to be displayed back on the glasses’ display

Spotify Player with Lyrics on Smart Glasses

Using our Wearables SDK to experiment with a music player on compatible smartglasses (in this case the Brilliant Labs’ Frame running custom firmware)

Also using a Mudra Band and TapXR for hand controls

Realtime maps in your smart glasses

Using our wearables SDK to display a simple map overlay, using @openstreetmap.bsky.social data to retrieve nearby data

Recognize people with smart glasses

Using our wearables SDK to look for certain people, and displaying relevant information when someone is recognized

Works with compatible smartglasses (in this case the Brilliant Labs’ Frame running custom firmware)

Experimenting with hand controls on smart glasses

With our Wearables SDK, we can both display a 3d wireframe scene on compatible smartglasses, as well as control a cursor by strapping one of our motion modules to our wrist

We’re able to detect pinches by running an @edgeimpulse.com model

View yourself in the third person with smart glasses

Using our Wearables SDK, we’re able to visualize MediaPipe tracking data (body, hands, and face) on compatible smartglasses (in this case the Brilliant Labs Frame running our custom firmware)

Great for yoga, working out, dancing, etc

Translation on Smart Glasses

Added foreign language support to our Wearables SDK, allowing us to display languages like Chinese, Japanese, and Korean

In this case we’re using @hf.co models for both realtime transcription and translation - all done locally (no cloud required)

Boxing on smart glasses

Added wireframe support to our Wearables SDK, allowing us to display realtime 3d graphics (like A-Frame and @threejs.org) on compatible smart glasses (in this case the Brilliant Labs Frame)

Realtime Subtitles on Smart Glasses, using our Wearables SDK

We used @hf.co’s Whisper model to transcribe microphone data locally, and then display the transcription on the smart glasses’ display (in this case the Brilliant Labs’ Frame) - no cloud required

Added sprite support to our wearables SDK

Great for stuff like emulating 3d graphics via billboarding

Added Font support to our Brilliant Sole Wearables’ JavaScript SDK

Added Bitmap support to our Brilliant Sole Wearables’ JavaScript SDK

Our wearables’ JavaScript SDK works great with react-like environments

This is a simple example using @nextjs.org, @supabase.com, and @tailwindcss.com to make a basic data collection website that can both record and view sensor data

Detect head gestures on your smart glasses

Made a simple @edgeimpulse.com model to detect nods and head shakes

Smart glasses webcam

Created a virtual webcam using OBS and our SwiftUI camera demo

Express yourself with smart glasses

Using MediaPipe’s face tracking to visualize a user’s eyes and eyebrows on a smart glasses display (in this case the Brilliant Labs’ Frame running our custom firmware)

Our smart glasses display simulator also works in WebXR, allowing us to preview ar glasses applications on a Quest 3

Added a smart glasses display simulator to our JavaScript SDK

Preview what your ar glasses applications will look like using your webcam, a video file, or even an interactive 3D scene

Added display support for smart glasses to our JavaScript SDK

We created a Canvas-like interface for drawing primitives onscreen for smartglasses with displays, like the Brilliant Labs Frame

Easily draw rectangles, circles, rounded rectangles, ellipses, polygons, and segments

Running speech recognition on smart glasses 👓

As we port our firmware to the Brilliant Labs’ Frame, we added microphone support to our SDK’s, allowing us to run @hf.co’s Whisper Web to transcribe speech in the browser

Hand, face, and object tracking on smart glasses

Running MediaPipe and @hf.co models on the Brilliant Labs Frame camera data

Streaming camera data from smart glasses to a smart watch via Bluetooth

As we port our firmware to the Brilliant Labs’ Frame, we added camera support to our SDK’s

Ported our firmware to the Brilliant Labs Frame 👓, making it compatible with our SDK’s

Kicking a virtual soccer ball in WebXR using our motion modules

Kick, Stomp, and toss green shells in WebXR

By wrapping our motion module around our ankle, we can add kick/stomp detection running an @edgeimpulse.com ML model

Adding haptic feedback to create an immersive WebXR experience on the Quest 3

Here we vibrate our motion modules differently based on the gesture (pet, punch, grab, and release)

We added WiFi support to our JavaScript, allowing us to connect directly to our WiFi-enabled hardware via WebSockets

Using our motion modules to detect punches, providing haptic feedback

Experimenting with basic wrist controls on our modular Ukaton hardware

You can remove the insole+shoe clip, attach a loop clip, and strap it around your wrist

Made a simple @edgeimpulse.com model to detect pinches, and using the gyro to move the cursor, similar to @doublepoint.bsky.social

Rewrote Ukaton’s firmware so their hardware is 100% compatible with our SDK’s

We recently merged with Ukaton and now have 2 implementations for smart insoles

Interfacing with our smart insoles on an Apple TV in a SwiftUI app using our Swift Package