VU Amsterdam - CLTL - Robot Framework

Repository for Robot Applications created as part of the Computational Lexicology & Terminology Lab (CLTL) at the Vrije Universiteit, Amsterdam.

This is a Python 3 version of the Leolani platform and is not intended for the robot use. It does not support the NAOqi backend! It has been tested on Windows, Linux and Mac.

Features

A framework for creating interactive Robot Applications using Python, to enable:
- Human-Robot conversation using Speech-to-Text and Text-to-Speech
- Recognising friends by face and learning about them and the world through conversation
- Recognising and positioning the people and objects in its enviroment.
Natural Language Understanding through Syntax Trees (Grammars)
Knowledge Representation of all learned facts through a RDF Graph: Pepper's Brain!
Curiosity based on Knowledge Gaps and Conflicts resulting from learned facts
Realtime visualisation in web browser

Getting started

Check out our (old) WIKI for information on how it works.

Part of the WIKI still references the Python 2.7 version of Leolani!

More information

More information on the Pepper project at CLTL can be found on http://makerobotstalk.nl

Features

A framework for creating interactive Robot Applications using Python, to enable:
Human-Robot conversation using Speech-to-Text and Text-to-Speech
Recognising friends by face and learning about them and the world through conversation
Recognising and positioning the people and objects in its enviroment.
Natural Language Understanding through Syntax Trees (Grammars)
Knowledge Representation of all learned facts through a RDF Graph: Pepper's Brain!
Curiosity based on Knowledge Gaps and Conflicts resulting from learned facts
Realtime visualisation in web browser

Prerequisites

Python 3.7.9 environment
An application on Wolfram Alpha with for API access
A project on the Google Cloud Platform supporting Text-To-Speech and Speech-To-Text APIs
Docker Engine

How to start

We assume that the following steps are executed in in the directory where this README.md is located.

Clone this repo by running

git clone https://github.com/leolani/pepper.git
Start from a python 3.7.9 environment (highly recommended). To create an isolated environment for the project run

python -m venv venv
source venv/bin/activate

To deactivate the environment after use type

deactivate
Create a JSON key file for the service account used in your Google Cloud project and copy it to config/google_cloud_key.json.
Setup an application at WolframAlpha and create a text file named config/credentials.config with content:
```
[credentials]
wolfram: MY-APPLICATION-ID
```
Install Java, and on Linux install portaudio by running

sudo apt-get install portaudio19-dev default-jdk
Install the required python modules by running

pip install -r linux_requirements.txt

on Linux or

pip install -r mac_requirements.txt

on OS X or

pip install -r windows_requirements.txt

on Windows
Clone the repo pepper_tensorflow into a separate workspace by running

cd .. && git clone https://github.com/leolani/pepper_tensorflow.git

Keep in mind that this changes your current directory.
Build the pepper_tensorflow docker image

cd pepper_tensorflow && docker build -t cltl/pepper_tensorflow .
Return to the root directory of this repository and download GraphDB (You have to register your email and the link will be sent to your mailbox. Check the spam folder as well. Download the standalone server version). Move the zipfile to the setup/graphdb-docker/lib folder. In the setup/docker-compose.yml file adjust the GraphDB version to the version you just downloaded.

If you use a local installation of GraphDB remove the GraphDB entry from setup/docker-compose.yml before starting the Docker images in the next step.
Run the Docker images needed by Leolani by typing

cd setup && docker-compose up && cd ..

This will start:
- bamos/openface
- GraphDB
- pepper_tensorflow
Data used by these containers is stored in setup/data and can be reset by removing all subfolders of that directory. Depending on your Docker setup make sure the Docker containers have enough memory and CPU available.
Make sure there is a GraphDB repository named leolani. If not, run

./setup/setup-graphdb-repo.sh.

If this script fails, you may create the repository manually by accessing http://localhost:7200/webapi on your browser. Under Repository Management, use the POST endpoint to upload the /setup/repo-config.ttl file.
The directory pepper has to be in the PYTHONPATH. So everytime you run an example, be sure to write PYTHONPATH="." before starting an example file.

For example,

PYTHONPATH="." python apps/examples/greeting.py

If you use an IDE to run the applications make sure that the workspace root is added to the python path, that the working directory for execution is the workspace root and not the directory containing the application script and, if a virtual environment was setup in step 1, it is used.
Configurations for the application are set in config/default.config and individual items can be overriden in config/pepper.config. Choose the backend right backend in config/pepper.config (only system is supported for the Python 3 version).
Start other applications in pepper/apps/

For example,

PYTHONPATH="." python apps/hmk.py
Start/Stop Docker by running cd setup && docker-compose start/stop && cd .. when stopping/restarting the application.

Usage

See How to start

Examples

See the last numbered lines in How to start

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

Fork the Project
Create your Feature Branch (git checkout -b feature/AmazingFeature)
Commit your Changes (git commit -m 'Add some AmazingFeature')
Push to the Branch (git push origin feature/AmazingFeature)
Open a Pull Request

License

MIT

leolani / pepper

VU Amsterdam - CLTL - Robot Framework

Features

Getting started

More information

Features

Prerequisites

How to start

Usage

Examples

Contributing

License

About

Languages