Contents:
- Blood Pressure Recorder Mobile App
In this repo, I developed a mobile APP to recognize picture of blood pressure monitor and store the data to sqlite3 database. The use can view, add, edit the records.
In my experience, taking a picture is the easiest way to record important information, such as recording blood pressure, recording logs on the running machine.
About the files:
📦Blood Pressure Recorder Mobile App
┣ 📂example_img
┃ ┣ 📜IMG_20210904_180521.jpg
┃ ┣ 📜pick2.jpg
┃ ┗ 📜pick5.jpg
┣ 📜BP.db
┣ 📜bp_img.png
┣ 📜kv_str.py
┣ 📜line_img.jpg
┣ 📜main.py
┣ 📜recog.py
┣ 📜seg_image.ipynb
┗ 📜sql_op.py
The fron-end is designed with kivy, kiviMD, Plyer, back-end is implemented with python, OpenCV, numpy.
There are three screens:
- Home screen: show line map the records
- Adding screen, user can add records by filling out form, or uploading file, or taking a picture
- Records list screen, list all records, user can deleting selected record
In home screen, all records are shown in scatter plot
When new record is inserted or deleted, the figure will be updated.
The third creen is Viewing and editting screen, it lists all records ordered by date and time. The user can delete record by clicking deletion button.
In the below picture, the record with id 15 is deleted.
left: before deletion right: after deletion
Below is the base interface of the middle screen.
- left one is the base interface
- right one the the camera screen when clicking camera icon
Explaination of the icons:
black picture: showing the photo chosen or taken by userupload icon: choosing photo from local foldercamera icon: when clicking, it will switch to camera screen, by clicking theleft arrow icon, it will switch back to the previous screenHigh, Low, Date, Time: text fieldscalendar icon: Choosing date from pop up windowtime icon: choosing timme from pop up windowcancel icon: cancel this try, and clear all text fieldsadd icon: adding records to database, and clear all text fields
Below is the date picker window(left) and time picker window(right).
The user can add record by filling out the text files.
In the following pictures:
- left one shows the filled text fields
- middle one shows the result after clicking add icon
- right one shows the added result in list
The user can add record by choosing photo from local folder.
By clicking the upload icon, a folder explorer window will pop up, user can choose photo. Below are some results:
Below pictures show the second added results in scatter plot and list.
The App recognizes the pressure value correctly, it also extracts the date and time information from the file with the help of library exifread.
The user can add record by taking photo with camera.
By clicking the camera icon, it will switch to camera screen, by clicking the left arrow icon, it will switch back to the previous screen.
Below are the results:
- Left one is the the camera screen
- Right one is the adding record screen
Note: By using iVCam, I can use my cellphone's camera when coding on PC.
We can see that the App recognizes the pressure value correctly, and sets current time as it's date and time.
The back-end code is in files:
recog.pysql_op.py
The digit on the blood pressure monitor is seven segment digits, which are not always easy to recognize.
I tried EasyOCR and EAST text detection model.
Both of them can detect the words in the photo except the large digits in the middle, which are the blood pressure values.
Below are the text detection results, left is result of EasyOCR, right is result of EAST.
Both of them are powerful tool in text detection and/or recognition. But in my case, they don't work.
After trying some deep-learning based methods, I didn't get a better solution. So my decided to segment the digits and recognize them by their seven segment area.
DIGITS_LOOKUP = {
(1, 1, 1, 0, 1, 1, 1): 0,
(0, 0, 1, 0, 0, 1, 0): 1,
(1, 0, 1, 1, 1, 0, 1): 2,
(1, 0, 1, 1, 0, 1, 1): 3,
(0, 1, 1, 1, 0, 1, 0): 4,
(1, 1, 0, 1, 0, 1, 1): 5,
(1, 1, 0, 1, 1, 1, 1): 6,
(1, 0, 1, 0, 0, 1, 0): 7,
(1, 1, 1, 1, 1, 1, 1): 8,
(1, 1, 1, 1, 0, 1, 1): 9,
(0, 0, 0, 0, 0, 0, 0): 0
} #the last zero is for empty at the left-most side of valuesThe whole process go in this way:
- Read image, apply
Gaussian Blurto clear out noise, applyCannymethod to get the contours in the image. Usedcv2.GaussianBlur, cv2.Canny.
- Sort the contours by length, the first four contours shoulb be the outline of the LED screen. Used
cv2.findContours, cv2.arcLength, cv2.approxPolyDP.
- Convert the image to white-black picture, used
cv2.threshold
- Apply
Openingmethod to decrease noise again, usedcv2.getStructuringElement,cv2.morphologyEx - Setmenting the digits horizontally and vertically
- Extracting each digit by checking each segment's area
Take number 2 as example,its seven segments matrix is [1, 0, 1, 1, 1, 0, 1], by checking the above
DIGITS_LOOKUPmatrix, it's recognized as 2. The seven recognized digits are as follows:1,2,8, ,9,7
The details are in the file seg_image.ipynb.
There are very basic SQL operations in this file.
import sqlite3
import time
def connect():
conn = sqlite3.connect("BP.db") #BP means blood pressure
cur = conn.cursor()
#time is text datatype, https://tableplus.com/blog/2018/07/sqlite-how-to-use-datetime-value.html
cur.execute("CREATE TABLE IF NOT EXISTS bp_table (id INTEGER PRIMARY KEY, time int, high int, low int)")
conn.commit()
conn.close()
def insert(time, high, low):
conn = sqlite3.connect("BP.db")
cur = conn.cursor()
cur.execute("INSERT INTO bp_table VALUES (NULL,?,?,?)", (time, high, low))
conn.commit()
conn.close()
def view():
conn = sqlite3.connect("BP.db")
cur = conn.cursor()
cur.execute("SELECT * FROM bp_table")
rows = cur.fetchall()
conn.close()
return rows
def delete(id):
conn = sqlite3.connect("BP.db")
cur = conn.cursor()
cur.execute("DELETE FROM bp_table WHERE id=?", (id,))
conn.commit()
conn.close()
def update(id, time, high, low):
conn = sqlite3.connect("BP.db")
cur = conn.cursor()
cur.execute("UPDATE book SET time=?, high=?, low=? WHERE id=?", (time, high, low, id))
conn.commit()
conn.close()- Gneralize the recognition method, deep-based method is better, or train a model on blood picture pictures
- The UI design
- More interactive function
- More functions: such as recognizing the log on running machine, log on bandsports bracelets, mileage of my car, gallon and price after fueling, etc.
- Start up speed