Improve logs visualization
steve-chavez opened this issue · comments
I mostly use tmux for developing in terminals, when doing: sub0 console screen is runned inside tmux, screen conflicts with tmux prefix "ctrl+.." so I can't choose any screen split for scrolling the logs and also I can't quit screen.
I think it would be better not to break any user workflow nor force them to learn screen to navigate the logs.
I propose to visualize the logs with blessed, a nodejs terminal ui lib, some examples:
Webpack dashboard
Nodejs dashboard
With that we have scrolling working without terminal multiplexers and also the text widgets collapse so it may be better for smaller screen sizes.
WOW ... nice.
Feel free to start implementing this and also change whatever script you want.
What i want to see in this screen/console in the beginning is:
- openresty/db/rabbitmq logs
- the output of the watch command
- maybe later a few buttons (if they are possible) to "restart containers", "reset db", "recreate containers", "purge logs"
One thing to note about the "devtools" container. It's setup in such a way that it actually connects to the docker engine running on the host and reads the log from there, and you can also execute docker commands in the scripts that will have the same effect as if you run them manually in your terminal on the host.
(Very nice suggestion)
When implementing this, keep in mind that it should also work for a postgrest project, so in that case, there are no rabbitmq logs, but there are openresty/db logs (i am thinking, with the kickstart project for postgrest we also push users to a setup similar to subzero for development, everything in docker)
Let's try and have this interface
Ideally you would move all the code to nodejs (no shell scripts) so that people can do just npm install but if there are not ready made libs to colorize the logs we can still go with a docker image. For ppl that do not want to install node, we will have it installed in a docker
also check this repo if it helps https://github.com/yaronn/blessed-contrib
Syntax highlighting works through highlight.js, an example with two docker logs:
Though the lib for the terminal I'm using, cli-highlighting, requires a bugfix to avoid a workaround I'm doing now for highlighting the http logs, I'll do a PR later.
I'll work on the reloading next, with that all the shell scripts can be replaced, also for the distribution issue I've tried to create a nodejs binary through nexe.js but blessed uses some native modules that this lib doesn't support, there's other stuff to try like embedding the native modules into the binary, once finished maybe I could try that.
I'll be committing on this branch https://github.com/steve-chavez/sub0_devtools/tree/nodejs-version.
Reloading is working now though the reset in case of error is not finished.
I had some initial problems due to the fact that the current devtools downloads his own psql copy, and does the reloading with psql \i meta-command and there's no client library that supports meta-commands, so I thought the best solution was just to use the already existing psql in the sub0 postgres container so I'm taking advantage of that through a docker exec {COMPOSE_PROJECT_NAME}_db_1 psql - c \i file.
The thing with that is that now we need to add DB_DIR=docker-entrypoint-initdb.d/ to the .env file so I can do psql \i to reload the .sql files contained in the docker managed volume, this is done here.
I'm using psql from the PostgreSQL container defined here in sub0 , not from postgREST, psql it's already included in the official PostgreSQL image, why that is not a good solution?
So right now everything is working except the Purge Log button:
You can rotate through the logs with the left and right arrow keys. I've left instructions so you can test in on your local machine, if the build is successful you should only need to run one command(sub0_devtools) when in your sub0_kickstart directory.
Purging the docker logs on Linux is a one liner but on OS X seems that docker-machine needs to be used, don't know about Windows, I would have to query the platform and choose the right command.
Blessed doesn't support tabs like the ones you drew and I haven't seen an example implementation, I've noticed the Blessed API has issues while working with it, I think that it'll better to keep it simple, I've seen the nodejitsu-ui do it this way.
The code needs to be improved and right now I think the dashboard isn't that robust but hopefully you can test all the current button actions.
very nice progress 👍
tabs are not important, it was just an idea. The code looks good, simple, easy for other ppl to submit PRs and ideas in the future.
I have not booted it up yet, out of office but i will soon and this looks 90% done, the only work that will probably be needed is a bit of helpful hints for other on how to use this, split components into files (since that is the style of js echosystem, but 1 file does not bother me), maybe package it in different ways for release and automate the process, i'll think about it.
Add whatever bits you are still working on then let's leave it as it is for a short while untill i test it and think of how to package it, and also while i rearange a bet the kickstart project to fit a postgrest project. I'll ping you on slack abut a small feature for postgrest
Already pushed the branch to https://github.com/ruslantalpa/sub0_devtools/tree/nodejs-version.
To summarize the discussion on chat, the new tasks are:
- Change watcher to listen to
*.sql,*.conf,*.luafile changes on any dir level and ignoretestsdir - Show list of containers on top highlighting the active one.
- Acommodate layout according to available containers(be it postgREST + db or sub0)
- Expand and shrink watcher log with up and down arrow keys.
- Add spinner to watcher while shrinked if it's reloading some file.
- Add Purge action and clear log
- Refactor and modularize the code.
Also, you mentioned Restart this container as a new task, but it's already implemented, by clicking the button with the same name or pressing "2" it restarts the container that corresponds to the logs shown, then a message like Restarting sub sub0boilerplate_postgrest_1 in shown the logs, is it showing an error for you?
Tried to make the layout you requested, but the blessed library is brittle and couldn't make it work. I have an alternate layout to propose:
With that, is more obvious that you can iterate over the containers, and also it shows the one enabled in green, also included the watcher there and the spinner shows to it's right, by selecting the Watcher will split the space of the container log in half.
If you agree with that I'll finish the features on that layout, also maybe add a border to the buttons container in the bottom.
Aside from that, I was able to create an executable using https://github.com/zeit/pkg, so that will help with distribution.
From what i understand the problem was with making the watcher expand at the bottom. That is ok to change.
Still, the container names at the left is not ok. Have them at the top, just like the (1,2,3) is at the bottom and the Watcher is just another window.
push what you have today, even if it does not compile/work, i want to add to it some stuff too.
The problem had more to do with layout, the items at the top plus having buttons at the bottom didn't look good, tried to make a horizontal separator line having your first suggestion at the top, and got weird bugs, like:
Don't know why by having a single line it rendered multiple lines, tried removing padding, and other stuff, didn't work. So I tried other options, figured why not to do it like the jitsu-ui:
What's wrong with having the containers on the left, too much space wasted?
Also I deleted previous layout iterations, the code I have is just for the layout with the containers on the left, I think it'll be better if I commit a completed task tomorrow.
i'll look over it today and think about it. commit as it is
all the tasks in this issue are implemented, closing