The frontend for the Smart Visionary application, designed to assist the visually impaired and blind. This Flutter application connects to the backend and facilitates user interaction via voice commands and taps.
- Voice Commands - Activate features using specific voice commands.
- Double-tap Capturing - Double-tap on the screen to capture an image.
- Firebase Messaging - For authentication and device token management.
- Audio Feedback - Results from the backend are played back as audio.
After choosing a model or feature to use, the user double-taps on the screen to capture an image. This image is sent to the backend server through the specified API for the chosen feature. The backend processes the image and returns the result as text within a JSON file. The Flutter application then converts this text into audio and plays it to the user.
- Speak the trigger word to select a feature.
- Double-tap to capture an image.
- The app processes the image using the backend API and provides results accordingly.
The app is able to recognize any sentence containing any of these trigger words
- Read - Activates the reading mode used to read any text in either languages English or Arabic.
- Face - Used to recognize familiar faces that have been saved before (friends or family of the user).
- Describe - Activates the image captioning mode used to describe the scene in front of the user.
- Money - Used to recognize Egyptian currency and calculate the sum.
- Introduction - Used to display a how-to-use audio introduction.
- Arabi - Used to translate the most recent audio output that has been displayed to the Arabic language.
- Skip - Used when there is a stranger (a new face has been detected) and the user doesn't want to add him to the familiar faces list.