Most of the AI demos I come across are hand gestures to control a game, face filters, object detection drawing boxes around a coffee mug on a webcam feed. They look great. I kept getting stuck on the same question though: who is this actually for?
The Idea
A browser-based assistive tool for visually impaired people. Real-time object detection, audio feedback, and possibly haptic feedback, to help someone move through the space around them.
Where It Came From
At DU there is a professor who is blind. Watching how he works with technology, mostly through accessibility tools that are either limited or clunky, made me wonder what would happen if the same browser APIs I mess around with for fun were pointed at an actual problem.
That is the whole origin. Nothing more dramatic than that.
What It Should Do
The app runs in the browser and lets the user:
- Point their device camera at their surroundings
- Detect objects in real time (cars, people, potholes, signs)
- Hear what is in front of them through the Web Speech API
- Optionally feel vibration patterns instead of, or alongside, the audio
- Choose during setup whether they want voice, vibration, or both
The whole thing has to stay simple. Anything I add that needs explaining is probably a mistake.
Tech Stack
| Feature | Tech / API |
|---|---|
| Object Detection | TensorFlow.js + COCO-SSD |
| Camera Access | getUserMedia() Web API |
| Voice Instructions | Web Speech API |
| Vibration Feedback | Vibration API (mobile-supported) |
| UI | Next.js |
| Accessibility Enhancements | ARIA roles, voice commands (future) |
Later, If This Works
- Pothole detection, which would need a custom model trained on Indian roads
- QR code recognition for reading signs and information
- Face and person detection with alerts
- Offline support through PWA capabilities
- Voice-based interaction so the app can be operated hands-free
Why Not OpenCV
OpenCV is more powerful, no argument there. But most projects built on it need a local setup, are written as demos rather than products, and assume the user is technical.
I want something that opens in a browser tab. No install, no setup. If someone can open Chrome, they should be able to use this.
What I Expect to Learn
- TensorFlow.js for real-time detection
- Building UI where accessibility is the requirement, not a checklist item at the end
- Using browser APIs in something meant for actual use
- Deploying models and keeping them fast enough to be useful
- Possibly training a custom model for potholes and other things specific to Indian streets
If You Are Working on Something Similar
Accessibility, browser-based ML, open source assistive tech. I would like to talk to you.
You can find me on LinkedIn: @sidharthsangelia