yet another storm ui based on storm ui rest api release with 0.9.2 or later
Origin storm ui is not handy for some cases. Thanks for the 0.9.2 release brings out the rest api which is the footstone of this project.
This project is trying to build yet another ui for storm.
Licence: Apache License 2
Original storm UI is not customizable. So
- Dangerous operations are exposed to all users, such as "Kill topology"
- ACL, SSO can not be added (this is not support in this project either but it can be done based on this easily)
- Other data can not be added such as host ip, host load, etc.
And from the usability part
- Need to open lots of tabs to check a topology status
- Runtime status on a host is not provided
This project can helps if these are what you want.
- Provide a Host perspective to show runtime status of topologies on hosts.
- Provide one-page web application experience
- Other enhancements...
This is the UI at first glance:
If interested, this section will help to run this web application.
- Need storm 0.9.2 or 0.9.3. Start storm ui using "storm ui"
- Need Maven 3 to build
- Developed using Java 7 (Should work on Java 6)
- Run in Tomcat (Any Java EE container should works)
- Tested and support Chrome and Firefox. IE is not supported (Not a web ui expert, don't know why and has no plan to support IE (-: )
Get code from github. Goes into directory "yet-another-storm-ui/yet-another-storm-ui-server", run "maven clean pacakge install". Then goes to "yet-another-storm-ui/yet-another-storm-ui-server/target", there should be a "yasu.war".
Copy the "yasu.war" to the Tomcat's webapp dir ( NOTE: Please don't rename the war file, if rename is needed, please refer to the last paragraph "Touble Shooting -> YASU not works if rename the war file" of this section) and this webapp will run in subdomain "yasu"
Open the link "http://TOMCAT_SERVER_IP:PORT/yasu" in browser, like "http://127.0.0.1:8080/yasu". You might see a page like this:
It is almost there. YASU now need a configuration to know the Storm UI url so that YASU can get storm cluster status data.
Click the Settings tab, specifiy the Storm UI url as follow:
Click "Update Storm URL" button. If update succeeded, there will be a tooltip says "Storm url updated to AAA.BBB.CCC.DDD:EEEEE" and all topologies should be listed as tabs after Host tab, as follow:
If Update failed, there will be a tooltip as follow:
Then please check the storm url and update again.
After the Storm URL is updated succeeded, all topologies should be listed as tabs after Host tab. But for firefox, this is not done automatically. Then please press F5 to refresh the whole page.
If rename the war file is needed, a file need to be modified:
- yet-another-storm-ui/app/scripts/app.js: "return $http.get("/yasu/" + restPath);" should be changed to "return $http.get("/THE_WAR_FILE_NAME/" + restPath);" . For example: return $http.get("/myyasu/" + restPath);
The rest steps are the same ( and surely open the link with the new subdomain name)
Following are details about the UI.
The new storm ui is still web-based and trying to provide one-page web application experience. It will use tabs to list top-level content. There are "Overview" tab, "Settings" tab, "Host" tab and topology tab(s).
Tabs are listed on the right-top of the page. The following is details about the tabs.
Overview tab is quite the same like the storm origin UI, including Cluster Summary, Topology Summary, Supervisor Summary and Cluster Config.
One small enhancement it will show the IP together with the host name.
Here is the snapshot
Settings tab now has one setting only. In case there are more than one storm cluster, yasu supports switch storm restful URL in this tab. Specify a new URL and click the "Update Storm URL" button, the new URL will be stored in cookie and sent to server. And the UI's data will comes from the new URL.
Host tab is new to storm origin ui. It list status start from host perspective.
This tab shows host status in the following order:
- Host
- Slots status on the Host (Process)
- Executors status (Thread in the same Process) on the Slot
In this tab, it is easy to find out if memory-critical components(Spout/Bolt) run on the same slot or if CPU-critical components run on the same host. These can be useful for trouble-shooting.
One new column planning to be added is the restart times of an component. Storm will manage to restart a bad component automantically, it is hard to know how many times a component restarted. (Only knows how long it is up. It might restart one or more times prior to the last startup).
Here is the snapshot
Each topology is listed as a tab.
There are six parts on topology tab
First two is Topology summary and Topology stats:
Theses are the same as storm default.
Then the spout part:
In this part, some key columns uses different background color. And there is a detail button to show the details of the spout:
Basically this part comes from spout page of origin storm ui but tighter.
- Shows status of 10 minutes only. Mostly that is what should be cared about for a real-time system.
- Skip the empty part. For example, if there is no error, there will be no error table rather than show an empty one.
- Add host IP of executor.
- Shorten system component name. For example, metrics component's name is full class name which is kind of long. Now it is shorten to "__" plus "class short name".
Then the bolt part. The changes are quite the same as spout:
Finally comes the config parts. It is separated to two parts, "Different configs from default" and "Same configs with default".
PS: Topology actions are not supported for now. This might be supported together with ACL.
There are two project. One is called yet-another-storm-ui which is only front-end based on angularjs. Another is yet-another-storm-ui-server which provide yet-another-storm-ui data with rest api too.
user send request to yet-another-storm-ui, yet-another-storm-ui request data from yet-another-storm-ui-server and yet-another-storm-ui-server pull data from storm rest api.
yet-another-storm-ui-server is quite a simple java restful app. it uses annotations to add servlet. There is client wrap to communicate with storm rest api. And there is a service to do more works with raw data returned from storm api (adding caches maybe). Also some beans is created to store storm api data for convenience.
yet-another-storm-i is created using yeoman, using AngularJS as framework. if you want to use grunt, please run "bower install & npm install" for the first time.
There are more features to deliver, such as
- ACL, especially for kill topology.
- JMX integration.
- More monitoring&management.
- Add Host perspective. Show slots and executors running on the host
- Show topology config as two parts: the same with default ande different with default
- Integrate topology components details into the same page
- Show IP together with hostname