amitsou / Multimodal-User-Monitoring

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

APM Generic badge

About The Project

This repository contains the thesis code entitled: 'Workplace recording in order to record fatigue from work sources'

This dissertation takes place within the interdepartmental program of postgraduate studies in 'Data Science' which is co-organized by the Department of Informatics and Telecommunications of the University of the Peloponnese and the Institute of Informatics and Telecommunications of the National Center for Scientific Research Demokritos (NCSRD). The purpose of this work is to create a pipeline in order to collect data from sensors which include:

  • Resistance-power sensors mounted on a chair
  • A common web camera with a mounted microphone
  • Peripherals such as the keyborad and the mouse

After the data collection process we create the Feature Vectors in order to create a Machine Learning Pipeline. Techniques from the fields of machine learning, vision computer, audio analysis and text analysis will then be used, in order to identify how much fatigue, stress and anxiety an employee is experiencing.

Getting Started

To use the existing code you must use force resistance sensors mounted on a chair. These sensors should be connected with an Arduino UNO in order to send the data via the serial monitor (through the USB port). Don't worry if you don't own any FSR sensors. You could easily use any of the remaining code files due to the reason that every file can be executed individually.
Example: If you wish to use only one modality, f.i. say you want to run a webcamlogger in order to get the user's facial expressions every minute, it can be achieved by executing './webcamlogger.sh.'
Keep in mind that there's also an implementation of the following tree structure in order to organize your recordings:

├───Data
    ├───Mouse
        ├───Raw 
        ├───Edited_logs
        ├───CSV
    ├───Keyboard
        ├───Raw 
        ├───Edited_logs
        ├───CSV
    ├───Chair
        ├───Raw 
        ├───Edited_logs
        ├───CSV
    ├───Webcam
    ├───Features

If you wish to create the aforementioned tree structure execute the following script:

$cd /Loggers
$python3 initialize_dirs.py

Prerequisites

In order to execute the code in this repository you'll need the following installations.

  • FFMPEG for Linux version: 4.2.4-1ubuntu0.1

    $sudo apt install ffmpeg  
    
  • Video for linux utilities v4l-utils

    $sudo apt install v4l-utils
    
  • In order to execute the webcamlogger you'll need to check for the installed devices on your PC

    $v4l2-ctl --list-devices   
    
  • pynput for python3: https://pypi.org/project/pynput/

📱 Contact

amitsou95@gmail.com
Link

About

License:MIT License


Languages

Language:Python 94.8%Language:Shell 3.4%Language:C++ 1.8%