BabyFace

🔧 Dependencies

google-cloud-aiplatform==1.36.4
redis==5.0.1
twilio==8.10.2

Vertex AI

Install the Vertex AI SDK for Python with this link. Ensure that the Python version (Python 3.10 or 3.11 as of 11/19/23) for the virtual environment fits the python requirement for Vertex AI SDK. Then, set up the Application Default Credentials for your Google Cloud Services. Finally, ensure to do source ~/.bashrc at the end.

Redis

Install Redis for Python with this link or using pip install redis . Please refer to the Redis Documentation for more help.

Twilio

Install and setup Twilio for Python using this link or using pip install twilio.

📖 Implementation

Diagram

Redis Database

Create library to use in other files to create the same Redis instance

Conversationer

Initiates conversation with Vertex AI and sends/receives messages from a Redis Database.

Initializes Google API Platform
Initializes Redis Database
Initializes a Vertex AI ChatModel using a pre-trained model named "chat-bison"
Retrieves context and input message from the Redis database
Starts a chat with the Vertex AI model using the provided context
Sends the input message to the model and receives a response
Tries to establish a connection to another Redis database to verify connectivity.
Stores the model's response back into the Redis database for the user

Image Question and Answer

Use Vertex AI image text model to answer a series of questions about baby’s situation.

Initialize Google Cloud and Vertex AI Model
Load the input image 
Generate the answers given the series of questions 
Initialize the Redis database 
Formulate and publish the answers as the context for 
	the Conversationer

Twilio App

Contains SMS interactions with Parent

Initializes Redis Database 
Create a Twilio Client 
Retrieve and publish the Parent's Message 
	for Conversationer
Retrieve Conversationer's Message and send to the Parent

Image Capturer

Takes image from laptops' webcam

Open a connection to camera
Once every 10 second has passed since the last capture, 
	capture a new image

🚀 Launch

Navigate to first_demo folder
Ensure that Google Application Default Credentials are set up
Confirm that Twilio’s Account SID, Authentication Code, and Phone Numbers are correct
Learn the virtual environment that you have created with the main dependencies
Execute capture_image.py on a terminal
Execute ngrok http 5000 on a different terminal and copy ngrok URL into Twilio Webhook
1. ⚠️MacOS needs to turn off Airplay Receiver in Settings
Execute Twilio Flask App twilio_app.py on another terminal, resulting in three different running terminals
Text to Your Designated Baby Face AI!

🔮 Future Implementations

Our vision extends to integrating long-term video monitoring devices like cameras or specialized baby monitoring devices. We aim to refine our AI model, providing more tailored solutions and fostering more engaging conversations to better serve parents and their babies.

Kulkoen / BabyFaceAI