Live demo: pychat.org, video
This is free web (browser) chat, that features:
- Sending instant text messages via websockets.
- Sending images, smiles, anchors, embedded youtube, giphy, code highlight
- Making calls and video conference using Peer to peer WebRTC.
- Sharing screen during call or conference
- Sending files directly to another PC (p2p) using WebRTC + FileSystem Api (up to 50MByte/s, limited by RTCDataChannel speed)
- Editing images with integrated painter
- Login in with facebook/google oauth.
- Sending offline messages with Firebase push notifications
- Responsive interface (bs like)+ themes
Pychat | Slack | Skype | Telegram | Viber | |
---|---|---|---|---|---|
Open Source | + | - | - | - | - |
Free | + | +/- | +/- | + | +/- |
Screen sharing | + | + | - | - | - |
Syntax highlight | + | - | - | - | - |
Unlimited history | + | +/- | + | + | + |
Audio/Video conference | + | + | + | + | - |
Can run on your server | + | - | - | - | - |
Audio/Video messages | + | - | - | + | + |
P2P file sharing | + | - | - | - | - |
Desktop client | + | + | + | + | +/- |
Mobile client | + | + | + | + | + |
GCM phone call | - | - | +/- | - | +/- |
3rd-party integration | - | + | - | - | - |
Please don't use this build for production, as it uses debug ssl certificate, lacks a few features and all files are located inside of container, meaning you will lose all data on container destroy.
- Download and run image:
docker run -tp 443:443 deathangel908/pychat-test
- Open https://localhost
- You need create ssl certificates:
server.key
andcertificate.crt
. For example
openssl req -nodes -new -x509 -keyout server.key -out certificate.crt -days 3650
- Download settings.py and edit it according comments in it.
- Download production.json and edit it according wiki
- Create volume and copy files there
docker volume create pychat_data
containerid=`docker container create --name dummy -v pychat_data:/data hello-world`
docker cp settings.py dummy:/data/settings.py
docker cp production.json dummy:/data/production.json
docker cp certificate.crt dummy:/data/certificate.crt
docker cp server.key dummy:/data/server.key
docker rm dummy
If you need to edit files inside container you can use
docker run -i -t -v pychat_data:/tmp -it alpine /bin/sh
- Run image with:
docker run -t -v pychat_data:/data -p 443:443 deathangel908/pychat
- Open https://localhost and enjoy it!
If you don't or unable to run docker you can alway do the setup w/o it. You definitely spend more time, so I would recommend to use docker if possible. But if you're still sure, here's the setup for cent-os/archlinux based system:
- For production I would recommend to clone repository to
/srv/http/pychat
. If you want to close the project into a different directory, replace all absolute paths in config files. You can usedownload_content.sh rename_root_directory
to do that. - For archlinux follow Install OS packages, add add these ones:
pacman -S postfix gcc jansson
. - For centos use add
alias yum="python2 $(which yum)"
to /etc/bashrc if you use python3. And then install that packagesyum install python34u, python34u-pip, redis, mysql-server, mysql-devel, postfix, mailx
- If you want to use native file-uploader (nginx_upload_module written in
C
) instead of python uploader (which is a lot slower) you should build nginx yourself. For archlinux setup requirespacman -S python-lxml gd make geoip
. To build nginx with this module run from the root user:bash download_content.sh build_nginx 1.15.3 2.3.0
. And create dir + useruseradd nginx; install -d -m 0500 -o http -g http /var/cache/nginx/
. If you don't, just install nginx with your package manager:pacman -S nginx
oryum install nginx
- Follow Bootstrap files flow.
- I preconfigued native setup for domain
pychat.org
, you want to replace all occurrences ofpychat.org
in rootfs directory for your domain. To simplify replacing use my script:./download_content.sh rename_domain your.new.domain.com
. Also checkrootfs/etc/nginx/sites-enabled/pychat.conf
ifserver_name
section is correct after renaming. - HTTPS is required for webrtc calls so you need to enable ssl:
- Either create your certificates and put according to pychat.conf ssl_certificate/ssl_certificate_key.
- Either use something like certbot
- Copy config files to rootfs with
sh download_content.sh copy_root_fs
. - Don't forget to change the owner of current (project) directory to
http
user:chown -R http:http
. And reload systemd configsystemctl daemon-reload
. - Generate postfix files:
install -d -m 0555 -o postfix -g postfix /etc/postfix/virtual; postmap /etc/postfix/virtual; newaliases; touch /etc/postfix/virtual-regexp; echo 'root postmaster' > /etc/aliases
- For archlinux start this services:
packages=( mysqld redis systemctl tornado@8888 nginx postfix ) ; for package in "${packages[@]}" ; do systemctl enable $package; done;
- For centos start that services:
packages=( redis-server nginx postfix mysqld tornado@8888) ; for package in "${packages[@]}" ; do service $package start; done;
- If you want to enable autostart (after reboot) for archlinux:
packages=( redis nginx postfix mysqld tornado) ; for package in "${packages[@]}" ; do systemctl start $package; done;
- If you want to enable autostart (after reboot) for centos:
chkconfig mysqld on; chkconfig on; chkconfig tornado on; chkconfig redis on; chkconfig postfix on
- Follow the Frontend steps
- Open in browser https://your.domain.com. Note that by default nginx accepts request by domain.name rather than ip.
- If something doesn't work you want to check
pychat/logs
directory. If there's no logs in directory you may want to check service stdout:sudo journalctl -u YOUR_SERVICE
. Check that userhttp
has access to you project directory.
cd fe; nvm use 12.10
yarn install
- Create production.json based on Frontend config
- Run
yarn run prod
. This generates static files infe/dist
directory.
Pychat uses websql and built the way so it renders everything possible w/o network. You have 2 options:
Use nativifier to create a client (replace pychat.org for your server): npx run nativifier pychat.org
- Create production.json based on Frontend config
- Run
cd fe; yarn run electronProd
.
This is harsh. If you're not familiar with android SDK I would recommend doing the steps below from AndroidStudio:
- Install android sdk, android platform tools. accept license
- Create production.json based on Frontend config
bash download_content.sh android
The flow is the following
- Install OS packages depending on your OS type
- Bootstrap files
- Follow the Frontend steps
- Start services and check if it works
This section depends on the OS you use. I tested full install on Windows/Ubuntu/CentOs/MacOS/Archlinux/Archlinux(rpi3 armv7). pychat.org currently runs on Archlinux Raspberry Pi 3.
- Install python with pip. only Python 3.6-3.8 is supported.
- Add pip and python to
PATH
variable. - Install redis. Get the newest version or at least 2.8.
- Install mysql. You basically need mysql server and python connector.
- You also need to install python's mysqlclient. If you want to compile one yourself you need to vs2015 tools. You can download visual-studio and install Common Tools for Visual C++ 2015. You need to run setup as administrator. The only connector can be found here. The wheel (already compiled) connectors can be also found here Mysqlclient. Use
pip
to install them. - Add bash commands to
PATH
variable. Cygwin or git's will do find.(for example if you use only git PATH=C:\Program Files\Git\usr\bin;C:\Program Files\Git\bin
).
- Install required packages:
apt-get install python pip mysql-server
(python should be 3.6-3.8) If pip is missing checkpython-pip
. - Install redis database:
add-apt-repository -y ppa:rwky/redis; apt-get install -y redis-server
- Install system packages:
pacman -S unzip python python-pip redis mariadb python-mysqlclient nvm
. - If you just installed mariadb you need to initialize it:
mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql
.
- Install packages:
brew install mysql redis python3
- Start services
brew services run mysql redis
- Install mysqlclient
pip install mysqlclient
- I use 2 git repos in 2 project directory. So you probably need to rename
excludeMAIN
file to.gitignore
or create link to exclude.ln -rsf .excludeMAIN .git/info/exclude
- Rename chat/settings_example.py to
chat/settings.py
. Modify file according to the comments in it. - Create virtualEnv
python3 -m venv --system-site-packages .venv
and activate it:source .venv/bin/activate
- Install python packages with
pip install -r requirements.txt
. - From root user create the database:
echo "create database pychat CHARACTER SET utf8 COLLATE utf8_general_ci; CREATE USER 'pychat'@'localhost' identified by 'pypass'; GRANT ALL PRIVILEGES ON pychat.* TO 'pychat'@'localhost';" | mysql -u root
. If you also need remote access do the same with'192.168.1.0/255.255.255.0';
- Fill database with tables:
bash download_content.sh create_django_tables
Follow the Frontend steps
- Enable django support. Go to Settings -> Django -> Enable django support.
- Django project root: root directory of your project. Where .git asides.
- Put
Settings:
tochat/settings.py
Settings
->Project pychat
->Project Interpreter
->Cogs in right top
-> 'Add' ->Virtual Environment
->Existing environment
->Interpereter
=pychatdir/.venv/bin/python
. Click ok. In previous menu on top 'Project interpreter` select the interpriter you just added.Settings
->Project: pychat
->Project structure
- You might want to exclude:
.idea
- mark
templates
directory asTemplate Folder
- Add tornado script:
Run
->Edit configuration
->Django server
-> CheckboxCustom run command
start_tornado
. Remove port value.
- atm frontend linting is only available, so
cd fe
- https://eslint.vuejs.org/rules/this-in-template.html
Current linting supports:
- Sass is linted with stylelint configured with .stylelintrc
- Typescript are linted with eslint, along with .vue files eslint-plugin-vue configured with .eslintrc.json
- New
- Edit files templates...
- Vue single file component
<template>
<div>#[[$END$]]#</div>
</template>
<script lang="ts">
import {State} from '@/utils/store';
import {Component, Prop, Vue, Watch, Ref} from 'vue-property-decorator';
@Component
export default class ${COMPONENT_NAME} extends Vue {
}
</script>
<style lang="sass" scoped>
</style>
Disable tslint, since it's not used, and enable eslint:
- Settings
- Typescript
- Tslint
- Disable tslint
- to resolve absolute path for webpack webstorm requires webpack.config.js. Go to settings -> javascript -> webpack -> Webpack config file
I would recommend to use version node 12.10, nvm use 12.10
. You can install nvm with archlinux ubuntu windows.
- To get started install dependencies first:
yarn install
# or use npm if you're old and cranky - Take a look at copy [development.json](fe/development.json]`. The description is at Frontend config
- Webpack-dev-server is used for development purposes with hot reloading, every time you save the file it will automatically apply. This doesn't affect node running files, only watching files. So files like builder.js or development.json aren't affected. Take a look at development.json. To run dev-server use
yarn run dev
. You can navigate to http://localhost:8080 - To build android use
yarn run android -- 192.168.1.55
where 55 is your bridge ip address - To run electron use
yarn run electronDev
. This will start electron dev. and generate/tmp/electron.html
and/tmp/electron.js
- Start
mysql
server if it's not started. - Start session holder:
redis-server
- Start webSocket listener:
python manage.py start_tornado
- Open in browser https://127.0.0.1:8080.
- Add self signed ssl certificate provided by django-sslserver to browser exception. For chrome you can enable invalid certificates for localohost in chrome://flags/#allow-insecure-localhost. Or for others open https://localhost:8888 and https://localhost:8000. Where
8888
comes fromstart_tornado.py
Pychat is written in Python and typescript. For handling realtime messages WebSockets are used: browser support on client part and asynchronous framework Tornado on server part. For ORM django was used with MySql backend. Messages are being broadcast by means of redis pub/sub feature using tornado-redis backend. Redis is also used as django session backend and for storing current users online. For video call WebRTC technology was used with stun server to make a connection, which means you will always get the lowest ping and the best possible connection channel. Client part is written with progressive js framework VueJs which means that pychat is SPA, so even if user navigates across different pages websocket connection doesn't break. Pychat also supports OAuth2 login standard via FaceBook/Google. Css is compiled from sass. Server side can be run on any platform Windows, Linux, Mac. Client (users) can use Pychat from any browser with websocket support: IE11, Edge, Chrome, Firefox, Android, Opera, Safari...
Execute bash download_content.sh
it will show you help.
By default each user has turned off browser (console) logs. You can turn them on in /#/profile page (logs
checkbox). All logs are logged with window.logger
object, for ex: window.logger('message')()
. Note that logger returns a function which is binded to params, that kind of binding shows corrent lines in browser, especially it's handy when all source comes w/o libraries/webpack or other things that transpiles or overhead it. You can also inspect ws messages here for chromium. You can play with window.wsHandler.handleMessage(object)
and window.wsHandler.handle(string)
methods in debug with messages from log to see what's going on
Chat uses fontello and its api for icons. The decision is based on requirements for different icons that come from different fonts and ability to add custom assets. Thus the fonts should be generated (.wolf
etc). W/o this chat would need to download a lot of different fonts which would slow down the loading process. You can easily edit fonts via your browser, just execute bash download_content.sh post_fontello_conf
. Make your changes and hit "Save session". Then execute bash download_content.sh download_fontello
. If you did everything right new icons should appear under fe/src/assets/demo.html
Server pings clients every PING_INTERVAL miliseconds. If client doesn't respond with pong in PING_CLOSE_JS_DELAY, server closes the connection. If ther're multiple tornado processes if can specify port for main process with MAIN_TORNADO_PROCESS_PORT. In turn the client expects to be pinged by the server, if client doesn't receive ping event it will close the connection as well. As well page has window listens for focus and sends ping event when it receives it, this is handy for situation when pc suspends from ram.
Pychat uses standard django migrations tools. So if you updated your branch from my repository and database has changed you need to ./manage.py makemigration
and ./manage.py migrate
. If automatic migration didn't work I also store migrations in migration. So you might take a look if required migration is there before executing commands. If you found required migration in my repo don't forget to change Migration.dependencies[]
and rename the file.
ScreenShare available for Chrome starting from v71. For chrome v31+ you should install an extension. It uses chrome.desktopCapture
feature that is available only via extension. The extension folder is located under screen_cast_extension`.
If you want to locally test it:
- Open
chrome://extensions/
url in chrome and verify thatdeveloper mode
checkbox is checked. - In the same tab click on
load unpacked extension...
button and select screen_cast_extension directory. - Note that in order to
background.js
be able to receive messages from webpage you need to add your host toexternally_connectable
section in manifest.json
Tp publish extension:
- If you want to update existing extension don't forget to increment
version
in manifest.json. - Zip screen_cast_extension directory into e.g.
bash download_content.sh zip_extension
- Upload archive
extension.zip
to chrome webstore (Note, you need to have a developer account, that's 5$ worth atm).
The successful connection produces logs below in console
Sender:
rsok33GN CallHandler initialized
rsok33GN:0005:EJAd Created CallSenderPeerConnection
rsok33GN:0005:EJAd Creating RTCPeerConnection
rsok33GN:0005:EJAd Creating offer...
rsok33GN:0005:EJAd Created offer, setting local description
rsok33GN:0005:EJAd Sending offer to remote
rsok33GN:0005:EJAd onsendRtcData
rsok33GN:0005:EJAd answer received
rsok33GN:0005:EJAd onaddstream
rsok33GN:0005:EJAd onsendRtcData
Receiver:
rsok33GN CallHandler initialized
rsok33GN:0004:oIc5 Created CallReceiverPeerConnection
rsok33GN:0004:oIc5 Creating RTCPeerConnection
rsok33GN:0004:oIc5 onsendRtcData
rsok33GN:0004:oIc5 Creating answer
rsok33GN:0004:oIc5 onaddstream
rsok33GN:0004:oIc5 Sending answer
rsok33GN:0004:oIc5 onsendRtcData
rsok33GN:0004:oIc5 onsendRtcData
The string rsok33GN:0005:EJAd
describes:
rsok33GN
is ID of CallHandler0005
is Id of userEJAd
id of connection (TornadoHandler.id
)
The technologies stack used in project:
- Typescript
- Vue, Vuex, VueRouter, lines-logger
- Vuex-module-decorators, Vue-property-decorator
- Webpack and loaders
- Sass
builder.js is used to build project. Take a look at it to understand how source files are being processed. Its start point is entry: ['./src/main.ts']
. Everything is imported in this files are being processed by section loaders
.
Every vue component has injected .$logger
object, to log something to console use this.logger.log('Hello {}', {1:'world'})();
Note calling function again in the end. Logger is disabled for production. For more info visit lines-logger
This project uses vue-property-decorator (that's has a dependency vue-class-component) vuex-module-decorators. You should write your component as the following:
import { Vue, Component, Prop, Watch, Emit, Ref } from 'vue-property-decorator'
import {userModule, State} from '@/utils/storeHolder'; // vuex module example
@Component
export class MyComp extends Vue {
@Ref
button: HTMLInputElement;
@Prop readonly propA!: number;
@State
public readonly users!: User[];
@Watch('child')
onChildChanged(val: string, oldVal: string) { }
@Emit()
changedProps() {}
async created() {
userModule.setUsers(await this.$api.getUsers());
}
}
development.json and production.json have the following format:
{
"BACKEND_ADDRESS": "e.g. pychat.org:443, protocol shouldn't be there, note there's no trailing slash, you can specify '{}' to use the same host as files served with",
"IS_DEBUG": "set true for development and debug mode enabled",
"UGLIFY": "true/false uglifies js/css so it has less weight, set this for production.json only, when you're sue you don't need to debug the output",
"GOOGLE_OAUTH_2_CLIENT_ID" : "check chat/settings_example.py",
"FACEBOOK_APP_ID": "check chat/settings_example.py",
"MANIFEST": "manifest path for firebase push notifications e.g.`/manifest.json`",
"RECAPTCHA_PUBLIC_KEY": "check chat/settings_example.py RECAPTCHA_SITE_KEY",
"AUTO_REGISTRATION": "if set to true, for non loggined user registration page will be skipped with loggining with random generated username. Don't use RECAPTCHA with this key",
"PUBLIC_PATH": "Set this path if you have different domains/IPs for index.html and other static assets, e.g. I serve index.html directly from my server and all sttatic assets like main.js from CDN, so in my case it's 'https://static.pychat.org/' note ending slash"
}
- ctrl+c on builder leaves the process in foreground
- .stylelintignore @for loops doesn't work in linter AleshaOleg/postcss-sass#53
- https://www.infoworld.com/article/3443039/typescript-37-beta-debuts-with-optional-chaining.html
- get\s+(\w+)():\s+((\w|[|])+)\s+{\s+return\s+store.\w+;?\s+};? @State\n public readonly $1!: $2;
- purge all callbacks to async code
- move backend to be directory, fe to fe, docker to docker and etc
- npm run stats
- Add google and fb auth via iframe
- compile to bytenode for electron https://github.com/OsamaAbbas/bytenode
- Add codepart to live code
- https://static.pychat.org/main.js?5db3927a045ba970fade:17390:17 Uncaught Error: setLocalStreamSrc roomDict {}, {"id":1,"state":null} OBJ: Error: setLocalStreamSrc roomDict {}, {"id":1,"state":null}
- "unable to begin transaction (3850 disk I/O error)" when 2 tabs are opened
- Add search for roomname in rooms list and username for user in direct messages and user in room
- Save message upon typing in localstorage and restore it on page load, be aware of pasted files
- Add linter badges for typescript, test badges for tornado and backend, code coverage etc
- Update to tornado 6.0 and detect blocking loops https://stackoverflow.com/a/26638397/3872976
- https://stackoverflow.com/questions/33170016/how-to-use-django-1-8-5-orm-without-creating-a-django-project
- tornado uses blocking operation like django orm or sync_redis (strict redis). While this operations are executed main thread awaits IO and prevents new messages and connections from execution. async_redis create cb wrapper. django orm: https://docs.djangoproject.com/en/dev/releases/3.0/#asgi-support
- Replace email login to google_user_id and fb_user_id so we could detach oauth2 account
- Add smile reaction to the message below it, like in slack
- https://www.w3.org/TR/css-scrollbars-1/
- Search add user to room should container user icon
- add webhooks, move giphy to webhooks, add help command
- https://www.nginx.com/resources/wiki/modules/upload/
- Store image in filstream api
- add srcset and minify uploaded image
- Update to tornado 6.0 and detect blocking loops https://stackoverflow.com/a/26638397/3872976
- https://stackoverflow.com/questions/33170016/how-to-use-django-1-8-5-orm-without-creating-a-django-project
- tornado uses blocking operation like django orm or sync_redis (strict redis). While this operations are executed main thread awaits IO and prevents new messages and connections from execution. async_redis create cb wrapper. django orm: https://docs.djangoproject.com/en/dev/releases/3.0/#asgi-support
- autoupdate pychat.org from github webhook and expose port http to build
- If self assigned certificate was used, mb add user an option to click on iframe or smth?
- Giphy: The gif-picture won't change after editing and leaving it's name. But there are tons of other gifs under every tag.
- If user A was online in Brower BA and he didn't have any history, when he joins online from browser B and send the message, it won't appear on browser BA when he opens ba.
- RoomUsers should have disabled instead of Room, so when user leaves direct messages, another one doesn't exit it. But in case of new message, user just doesn't receive any... Mb we can make them hidden in UI
- Merge base.js into chat.js so 1 request less
- tornadoweb/tornado#2243
- Add sound/video messages like in telegram
- Add webrtc peer to peer secure chats (like telegrams)
- output logs to kibana
- Store userOnline in a single list, refactor All channel for online storing
- Add "last seen" feature and status afk/online/dnd
- blink icon in title on new message
- Add message to favorite
- Ability to quote any code
- Paint errors
- Add ability to show growls messages to channel from ADMIN
- gitb don't backup files larger than 10MB
- Add "last seen" feature
- Add video/voice record to chat like in telegram
- https://static.pychat.org/photo/uEXCJWJH_image.png
- Add go down button if scroll is not in the botom for chatbox
- Firefox doesn't google support fcm push
- Transfer file should be inside of chatbox instead of being a separate window
- Messages should appear in chat instantly with automatic resend when connection is up
- signup verification emails is sent to admin instead of current user.
- Add avatar to notifications and users
- update service worker if its version changed with registration.update()
- setTimeot stops working after 30min in chrome background, it has been changed to setInterval, check if it works
- giphy search should return random image
- Add payback to firebase
- Fix all broken painter event in mobile
- https://static.pychat.org/photo/xE9bSyvC_image.png
- https://developers.google.com/web/updates/2015/12/background-sync
- Added bad code stub for: Wrong message order, that prevents of successful webrtc connection: leporo/tornado-redis#106 https://stackoverflow.com/questions/47496922/tornado-redis-garantee-order-of-published-messages
- No sound in call https://bugs.chromium.org/p/chromium/issues/detail?id=604523
- paste event doesn't fire at all most of the times on painter canvasHolder, mb try to move it to
- Replaced email oauth with fb\google id and add them to profile
- Add applying zoom to method that trigger via keyboard in canvas
- add queued messaged to wsHandler, if ws is offline messages goes to array. userMessage input clears after we press enter and we don't restore its state we just put message to queue. When webrtc is disconnected we send reconnect event to this ws.queue
- Just a note https://codepen.io/techslides/pen/zowLd , i guess transform: scale is better https://stackoverflow.com/questions/11332608/understanding-html-5-canvas-scale-and-translate-order https://stackoverflow.com/questions/16687023/bug-with-transform-scale-and-overflow-hidden-in-chrome
- remove setHeaderTest, highlight current page icos. Always display username in right top
- add timeout to call. (finish after timeout) Display busy if calling to SAME chanel otherwise it will show multiple videos
- file transfer - add ability to click on user on receivehandler popup (draggable)
- add message queue if socketed is currently disconnected ???
- Add link to gihub in console
- Add title for room.
- TODO if someone offers a new call till establishing connection for a call self.call_receiver_channel would be set to wrong
- !!!IMPORTANT Debug call dialog by switching channels while calling and no.
- shape-inside for contentteditable
- Add multi-language support.
- remember if user has camera/mic and autoset values after second call
- android play() can only be initiated by a user gesture.
- add 404page
- https://code.djangoproject.com/ticket/25489
- http://stackoverflow.com/a/18843553/3872976
- add antispam system
- move loading messages on startup to single function?
- add antiflood settings to nginx
- tornado redis connection reset prevents user from deleting its entry in online_users
- add media query for register and usersettings to adjust for phone's width
- file upload http://stackoverflow.com/a/14605593/3872976
- add pictures preview if user post an url that's content-type =image
- SELECT_SELF_ROOM https://github.com/Deathangel908/pychat/blob/master/chat/settings.py#L292-L303 doesnt work with mariadb engine 10.1
- also admin email wasn't triggered while SELECT_SELF_ROOM has failed
- Remove django server and leave only tornado
- send image to chat if error on server or inet goes down while uploading - we don't have an option to retry
- Add linters like on webpack-vue-typescript
- ADD ability to change theme during registration
- add ability to cancel filetransfer on sender side
- add aliases to webpack
- add test
- add tslint
- add sass-lint
- resolve sw.ts imports doesn't work with ts-loader + file-loaders