Giters

nhminh0701 / pychat

webchat via WebSockets/WebRTC that allows messaging/video call/screen sharing

Home Page:https://pychat.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Docker Cloud Build Status Docker Cloud Build Status Scrutinizer Code Quality Codacy Badge

Live demo: pychat.org, video

Table of contents

About

This is free web (browser) chat, that features:

  • Sending instant text messages via websockets.
  • Sending images, smiles, anchors, embedded youtube, giphy, code highlight
  • Making calls and video conference using Peer to peer WebRTC.
  • Sharing screen during call or conference
  • Sending files directly to another PC (p2p) using WebRTC + FileSystem Api (up to 50MByte/s, limited by RTCDataChannel speed)
  • Editing images with integrated painter
  • Login in with facebook/google oauth.
  • Sending offline messages with Firebase push notifications
  • Responsive interface (bs like)+ themes

When should I use pychat:

Pychat Slack Skype Telegram Viber
Open Source + - - - -
Free + +/- +/- + +/-
Screen sharing + + - - -
Syntax highlight + - - - -
Unlimited history + +/- + + +
Audio/Video conference + + + + -
Can run on your server + - - - -
Audio/Video messages + - - + +
P2P file sharing + - - - -
Desktop client + + + + +/-
Mobile client + + + + +
GCM phone call - - +/- - +/-
3rd-party integration - + - - -

Run test docker image

Please don't use this build for production, as it uses debug ssl certificate, lacks a few features and all files are located inside of container, meaning you will lose all data on container destroy.

  • Download and run image:
docker run -tp 443:443 deathangel908/pychat-test

Run prod docker image

  • You need create ssl certificates: server.key and certificate.crt. For example
openssl req -nodes -new -x509 -keyout server.key -out certificate.crt -days 3650
docker volume create pychat_data
containerid=`docker container create --name dummy -v pychat_data:/data hello-world`
docker cp settings.py dummy:/data/settings.py
docker cp production.json dummy:/data/production.json
docker cp certificate.crt dummy:/data/certificate.crt
docker cp server.key dummy:/data/server.key
docker rm dummy

If you need to edit files inside container you can use

docker run -i -t -v pychat_data:/tmp -it alpine /bin/sh
  • Run image with:
docker run -t -v pychat_data:/data -p 443:443 deathangel908/pychat

Native setup

If you don't or unable to run docker you can alway do the setup w/o it. You definitely spend more time, so I would recommend to use docker if possible. But if you're still sure, here's the setup for cent-os/archlinux based system:

  • For production I would recommend to clone repository to /srv/http/pychat. If you want to close the project into a different directory, replace all absolute paths in config files. You can use download_content.sh rename_root_directory to do that.
  • For archlinux follow Install OS packages, add add these ones: pacman -S postfix gcc jansson.
  • For centos use add alias yum="python2 $(which yum)" to /etc/bashrc if you use python3. And then install that packages yum install python34u, python34u-pip, redis, mysql-server, mysql-devel, postfix, mailx
  • If you want to use native file-uploader (nginx_upload_module written in C) instead of python uploader (which is a lot slower) you should build nginx yourself. For archlinux setup requires pacman -S python-lxml gd make geoip. To build nginx with this module run from the root user: bash download_content.sh build_nginx 1.15.3 2.3.0. And create dir + user useradd nginx; install -d -m 0500 -o http -g http /var/cache/nginx/. If you don't, just install nginx with your package manager: pacman -S nginx or yum install nginx
  • Follow Bootstrap files flow.
  • I preconfigued native setup for domain pychat.org, you want to replace all occurrences of pychat.org in rootfs directory for your domain. To simplify replacing use my script: ./download_content.sh rename_domain your.new.domain.com. Also check rootfs/etc/nginx/sites-enabled/pychat.conf if server_name section is correct after renaming.
  • HTTPS is required for webrtc calls so you need to enable ssl:
    • Either create your certificates and put according to pychat.conf ssl_certificate/ssl_certificate_key.
    • Either use something like certbot
  • Copy config files to rootfs with sh download_content.sh copy_root_fs.
  • Don't forget to change the owner of current (project) directory to http user: chown -R http:http. And reload systemd config systemctl daemon-reload.
  • Generate postfix files: install -d -m 0555 -o postfix -g postfix /etc/postfix/virtual; postmap /etc/postfix/virtual; newaliases; touch /etc/postfix/virtual-regexp; echo 'root postmaster' > /etc/aliases
  • For archlinux start this services: packages=( mysqld redis systemctl tornado@8888 nginx postfix ) ; for package in "${packages[@]}" ; do systemctl enable $package; done;
  • For centos start that services: packages=( redis-server nginx postfix mysqld tornado@8888) ; for package in "${packages[@]}" ; do service $package start; done;
  • If you want to enable autostart (after reboot) for archlinux: packages=( redis nginx postfix mysqld tornado) ; for package in "${packages[@]}" ; do systemctl start $package; done;
  • If you want to enable autostart (after reboot) for centos: chkconfig mysqld on; chkconfig on; chkconfig tornado on; chkconfig redis on; chkconfig postfix on
  • Follow the Frontend steps
  • Open in browser https://your.domain.com. Note that by default nginx accepts request by domain.name rather than ip.
  • If something doesn't work you want to check pychat/logs directory. If there's no logs in directory you may want to check service stdout: sudo journalctl -u YOUR_SERVICE. Check that user http has access to you project directory.

Frontend

  • cd fe; nvm use 12.10
  • yarn install
  • Create production.json based on Frontend config
  • Run yarn run prod. This generates static files in fe/dist directory.

Desktop app

Pychat uses websql and built the way so it renders everything possible w/o network. You have 2 options:

Natifier

Use nativifier to create a client (replace pychat.org for your server): npx run nativifier pychat.org

Electron

  • Create production.json based on Frontend config
  • Run cd fe; yarn run electronProd.

Android app

This is harsh. If you're not familiar with android SDK I would recommend doing the steps below from AndroidStudio:

  • Install android sdk, android platform tools. accept license
  • Create production.json based on Frontend config
  • bash download_content.sh android

Development setup

The flow is the following

  • Install OS packages depending on your OS type
  • Bootstrap files
  • Follow the Frontend steps
  • Start services and check if it works

Install OS packages

This section depends on the OS you use. I tested full install on Windows/Ubuntu/CentOs/MacOS/Archlinux/Archlinux(rpi3 armv7). pychat.org currently runs on Archlinux Raspberry Pi 3.

Windows:

  1. Install python with pip. only Python 3.6-3.8 is supported.
  2. Add pip and python to PATH variable.
  3. Install redis. Get the newest version or at least 2.8.
  4. Install mysql. You basically need mysql server and python connector.
  5. You also need to install python's mysqlclient. If you want to compile one yourself you need to vs2015 tools. You can download visual-studio and install Common Tools for Visual C++ 2015. You need to run setup as administrator. The only connector can be found here. The wheel (already compiled) connectors can be also found here Mysqlclient. Use pip to install them.
  6. Add bash commands to PATH variable. Cygwin or git's will do find.(for example if you use only git PATH=C:\Program Files\Git\usr\bin;C:\Program Files\Git\bin).

Ubuntu:

  1. Install required packages: apt-get install python pip mysql-server (python should be 3.6-3.8) If pip is missing check python-pip.
  2. Install redis database: add-apt-repository -y ppa:rwky/redis; apt-get install -y redis-server

Archlinux:

  1. Install system packages: pacman -S unzip python python-pip redis mariadb python-mysqlclient nvm.
  2. If you just installed mariadb you need to initialize it: mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql.

MacOS

  1. Install packages: brew install mysql redis python3
  2. Start services brew services run mysql redis
  3. Install mysqlclient pip install mysqlclient

Bootstrap files:

  1. I use 2 git repos in 2 project directory. So you probably need to rename excludeMAINfile to .gitignoreor create link to exclude. ln -rsf .excludeMAIN .git/info/exclude
  2. Rename chat/settings_example.py to chat/settings.py. Modify file according to the comments in it.
  3. Create virtualEnv python3 -m venv --system-site-packages .venv and activate it: source .venv/bin/activate
  4. Install python packages with pip install -r requirements.txt.
  5. From root user create the database: echo "create database pychat CHARACTER SET utf8 COLLATE utf8_general_ci; CREATE USER 'pychat'@'localhost' identified by 'pypass'; GRANT ALL PRIVILEGES ON pychat.* TO 'pychat'@'localhost';" | mysql -u root. If you also need remote access do the same with '192.168.1.0/255.255.255.0';
  6. Fill database with tables: bash download_content.sh create_django_tables

Follow the Frontend steps

Configure IDEs if you use it:

Pycharm

  1. Enable django support. Go to Settings -> Django -> Enable django support.
  • Django project root: root directory of your project. Where .git asides.
  • Put Settings: to chat/settings.py
  1. Settings -> Project pychat -> Project Interpreter -> Cogs in right top -> 'Add' -> Virtual Environment -> Existing environment -> Interpereter = pychatdir/.venv/bin/python. Click ok. In previous menu on top 'Project interpreter` select the interpriter you just added.
  2. Settings -> Project: pychat -> Project structure
  • You might want to exclude: .idea
  • mark templates directory as Template Folder
  1. Add tornado script: Run -> Edit configuration -> Django server -> Checkbox Custom run command start_tornado. Remove port value.

Linting

Current linting supports:

Webstorm

Set template

  1. New
  2. Edit files templates...
  3. Vue single file component
<template>
    <div>#[[$END$]]#</div>
</template>

<script lang="ts">
  import {State} from '@/utils/store';
  import {Component, Prop, Vue, Watch, Ref} from 'vue-property-decorator';

  @Component
  export default class ${COMPONENT_NAME} extends Vue {
   
  }
</script>
<style lang="sass" scoped>

</style>

Change linting settings

Disable tslint, since it's not used, and enable eslint:

  1. Settings
  2. Typescript
  3. Tslint
  4. Disable tslint

Enable aliases for webpack

  • to resolve absolute path for webpack webstorm requires webpack.config.js. Go to settings -> javascript -> webpack -> Webpack config file

Build frontend

I would recommend to use version node 12.10, nvm use 12.10. You can install nvm with archlinux ubuntu windows.

  • To get started install dependencies first: yarn install # or use npm if you're old and cranky
  • Take a look at copy [development.json](fe/development.json]`. The description is at Frontend config
  • Webpack-dev-server is used for development purposes with hot reloading, every time you save the file it will automatically apply. This doesn't affect node running files, only watching files. So files like builder.js or development.json aren't affected. Take a look at development.json. To run dev-server use yarn run dev. You can navigate to http://localhost:8080
  • To build android use yarn run android -- 192.168.1.55 where 55 is your bridge ip address
  • To run electron use yarn run electronDev. This will start electron dev. and generate /tmp/electron.html and /tmp/electron.js

Start services and run:

  • Start mysql server if it's not started.
  • Start session holder: redis-server
  • Start webSocket listener: python manage.py start_tornado
  • Open in browser https://127.0.0.1:8080.
  • Add self signed ssl certificate provided by django-sslserver to browser exception. For chrome you can enable invalid certificates for localohost in chrome://flags/#allow-insecure-localhost. Or for others open https://localhost:8888 and https://localhost:8000. Where 8888 comes from start_tornado.py

Contribution guide

Description

Pychat is written in Python and typescript. For handling realtime messages WebSockets are used: browser support on client part and asynchronous framework Tornado on server part. For ORM django was used with MySql backend. Messages are being broadcast by means of redis pub/sub feature using tornado-redis backend. Redis is also used as django session backend and for storing current users online. For video call WebRTC technology was used with stun server to make a connection, which means you will always get the lowest ping and the best possible connection channel. Client part is written with progressive js framework VueJs which means that pychat is SPA, so even if user navigates across different pages websocket connection doesn't break. Pychat also supports OAuth2 login standard via FaceBook/Google. Css is compiled from sass. Server side can be run on any platform Windows, Linux, Mac. Client (users) can use Pychat from any browser with websocket support: IE11, Edge, Chrome, Firefox, Android, Opera, Safari...

Shell helper

Execute bash download_content.sh it will show you help.

Frontend logging

By default each user has turned off browser (console) logs. You can turn them on in /#/profile page (logs checkbox). All logs are logged with window.logger object, for ex: window.logger('message')(). Note that logger returns a function which is binded to params, that kind of binding shows corrent lines in browser, especially it's handy when all source comes w/o libraries/webpack or other things that transpiles or overhead it. You can also inspect ws messages here for chromium. You can play with window.wsHandler.handleMessage(object) and window.wsHandler.handle(string) methods in debug with messages from log to see what's going on

Icons

Chat uses fontello and its api for icons. The decision is based on requirements for different icons that come from different fonts and ability to add custom assets. Thus the fonts should be generated (.wolf etc). W/o this chat would need to download a lot of different fonts which would slow down the loading process. You can easily edit fonts via your browser, just execute bash download_content.sh post_fontello_conf. Make your changes and hit "Save session". Then execute bash download_content.sh download_fontello. If you did everything right new icons should appear under fe/src/assets/demo.html

Sustaining online protocol

Server pings clients every PING_INTERVAL miliseconds. If client doesn't respond with pong in PING_CLOSE_JS_DELAY, server closes the connection. If ther're multiple tornado processes if can specify port for main process with MAIN_TORNADO_PROCESS_PORT. In turn the client expects to be pinged by the server, if client doesn't receive ping event it will close the connection as well. As well page has window listens for focus and sends ping event when it receives it, this is handy for situation when pc suspends from ram.

Database migrations

Pychat uses standard django migrations tools. So if you updated your branch from my repository and database has changed you need to ./manage.py makemigration and ./manage.py migrate. If automatic migration didn't work I also store migrations in migration. So you might take a look if required migration is there before executing commands. If you found required migration in my repo don't forget to change Migration.dependencies[] and rename the file.

Screen sharing for Chrome v71 or less

ScreenShare available for Chrome starting from v71. For chrome v31+ you should install an extension. It uses chrome.desktopCapture feature that is available only via extension. The extension folder is located under screen_cast_extension`. If you want to locally test it:

  • Open chrome://extensions/ url in chrome and verify that developer mode checkbox is checked.
  • In the same tab click on load unpacked extension... button and select screen_cast_extension directory.
  • Note that in order to background.js be able to receive messages from webpage you need to add your host to externally_connectable section in manifest.json

Tp publish extension:

  • If you want to update existing extension don't forget to increment version in manifest.json.
  • Zip screen_cast_extension directory into e.g. bash download_content.sh zip_extension
  • Upload archive extension.zip to chrome webstore (Note, you need to have a developer account, that's 5$ worth atm).

WebRTC connection establishment

The successful connection produces logs below in console

Sender:

rsok33GN CallHandler initialized
rsok33GN:0005:EJAd Created CallSenderPeerConnection
rsok33GN:0005:EJAd Creating RTCPeerConnection
rsok33GN:0005:EJAd Creating offer...
rsok33GN:0005:EJAd Created offer, setting local description
rsok33GN:0005:EJAd Sending offer to remote
rsok33GN:0005:EJAd onsendRtcData
rsok33GN:0005:EJAd answer received
rsok33GN:0005:EJAd onaddstream
rsok33GN:0005:EJAd onsendRtcData

Receiver:

rsok33GN CallHandler initialized
rsok33GN:0004:oIc5 Created CallReceiverPeerConnection
rsok33GN:0004:oIc5 Creating RTCPeerConnection
rsok33GN:0004:oIc5 onsendRtcData
rsok33GN:0004:oIc5 Creating answer
rsok33GN:0004:oIc5 onaddstream
rsok33GN:0004:oIc5 Sending answer
rsok33GN:0004:oIc5 onsendRtcData
rsok33GN:0004:oIc5 onsendRtcData

The string rsok33GN:0005:EJAd describes:

  • rsok33GN is ID of CallHandler
  • 0005 is Id of user
  • EJAd id of connection (TornadoHandler.id)

Frontend Stack

The technologies stack used in project:

  • Typescript
  • Vue, Vuex, VueRouter, lines-logger
  • Vuex-module-decorators, Vue-property-decorator
  • Webpack and loaders
  • Sass

builder.js is used to build project. Take a look at it to understand how source files are being processed. Its start point is entry: ['./src/main.ts']. Everything is imported in this files are being processed by section loaders.

Every vue component has injected .$logger object, to log something to console use this.logger.log('Hello {}', {1:'world'})(); Note calling function again in the end. Logger is disabled for production. For more info visit lines-logger

This project uses vue-property-decorator (that's has a dependency vue-class-component) vuex-module-decorators. You should write your component as the following:

import { Vue, Component, Prop, Watch, Emit, Ref } from 'vue-property-decorator'
import {userModule, State} from '@/utils/storeHolder'; // vuex module example


@Component
export class MyComp extends Vue {
  
  @Ref
  button: HTMLInputElement;

  @Prop readonly propA!: number;
  
  @State
  public readonly users!: User[];

  @Watch('child')
  onChildChanged(val: string, oldVal: string) { }

  @Emit() 
  changedProps() {}

  async created() {
    userModule.setUsers(await this.$api.getUsers());
  }
}

Frontend config

development.json and production.json have the following format:

{
  "BACKEND_ADDRESS": "e.g. pychat.org:443, protocol shouldn't be there, note there's no trailing slash, you can specify '{}' to use the same host as files served with",
  "IS_DEBUG": "set true for development and debug mode enabled",
  "UGLIFY": "true/false uglifies js/css so it has less weight, set this for production.json only, when you're sue you don't need to debug the output",
  "GOOGLE_OAUTH_2_CLIENT_ID" : "check chat/settings_example.py",
  "FACEBOOK_APP_ID": "check chat/settings_example.py",
  "MANIFEST": "manifest path for firebase push notifications e.g.`/manifest.json`",
  "RECAPTCHA_PUBLIC_KEY": "check chat/settings_example.py RECAPTCHA_SITE_KEY",
  "AUTO_REGISTRATION": "if set to true, for non loggined user registration page will be skipped with loggining with random generated username. Don't use RECAPTCHA with this key",
  "PUBLIC_PATH": "Set this path if you have different domains/IPs for index.html and other static assets, e.g. I serve index.html directly from my server and all sttatic assets like main.js from CDN, so in my case it's 'https://static.pychat.org/' note ending slash"
}

TODO

About

webchat via WebSockets/WebRTC that allows messaging/video call/screen sharing

https://pychat.org

License:MIT License

Languages

Language:Java 28.3%Language:TypeScript 24.1%Language:Vue 18.8%Language:Python 13.7%Language:JavaScript 4.3%Language:HTML 3.4%Language:Shell 2.4%Language:CSS 2.1%Language:Batchfile 1.3%Language:CMake 1.1%Language:Dockerfile 0.5%Language:C++ 0.0%