Affordable Smart Glasses for the Blind and Visually Impaired
The Project
Affordable Smart Glasses for the Blind and Visually Impaired are designed to help with independence in performing day-to-day tasks, such as navigating in unfamiliar places, finding objects, and reading signs and books, with a focus on affordability with low cost components addressing particular needs of the blind in rural communities. A few years back when I was visiting India, my grandfather took me to see a rural school for blind people he works with. I saw how people with blindness lived their daily lives, navigating their disability all while living in an area with low infrastructure and limited access to money and resources. I was learning about the superpowers of computer vision and AI which inspired me to start working on a pair of affordable smart glasses for blind people with a camera that can use AI to tell the wearer what it sees through a bone conduction speaker near the ear for added privacy. The glasses have a 3D printed frame, wide angle camera, touch input, bone conduction transducer and a battery, and use a Raspberry Pi Zero 2W, Google Coral AI accelerator, and audio and power supply add-on boards. I wrote the code using Python, with OpenCV and dlib for CNN based face recognition, TensorFlow Lite with COCO dataset for object detection, Flite for text-to-speech and Tesseract for optical character recognition. While all these models run on the device itself for low latency and offline capabilities, I also added online capabilities using OpenAI’s API with their GPT-4o and TTS-1 module to generate voice-over description from the camera feed, and for reading text. I faced many technical difficulties when programming the app especially in making concepts work in the real world and in such a small prototype. For example, it was very difficult to connect to the audio on the Raspberry Pi through alsamixer. In another case, the AI models caused the camera to run at an extremely low speed since the Raspberry Pi wasn’t able to handle all the processing all the time. I managed to solve this using a Google Coral AI accelerator, and processing the picture every 5 seconds instead of every frame. I also made the speaking asynchronous so that the camera wouldn’t have to wait for the glasses to finish speaking. Size and usability was a recurring challenge. Originally, I planned to use a Raspberry Pi Pico since it was the smallest and cheapest microcontroller I could find. I switched to a Raspberry Pi Zero 2W, with better processing power and high-res camera support. Since most AI models were not built to run on Pi Zero, I had to learn to adapt and teach myself how it works. Cost was also a huge challenge, since many of the hardware or AI models came with a high cost and wouldn’t work if I were to make the glasses affordable, so I had to do extensive research to find out how to make everything as low cost as possible.
