critocrito / ncube

Data exploration and verification for the human rights.

Ncube

Enter the hypercube.

Installation • Documentation • Ncube Discovery • Development

---

Ncube supports human rights defenders and investigative journalists to conduct data based investigations. It helps to develop compelling stories which expose the misuse of power and human rights abuses. It is a graphical desktop and cross-platform application that turns quantitative data into qualitative data. All the features of Ncube have one purpose: produce a set of verified data. Ncube's features around the preservation, exploration and verification of data all serve this single goal.

Read more about Ncube in its introduction blog post. To get help about Ncube and data investigation methodologies, or to suggest new use cases and functionality, feel free to post on the Sugarcube Tools community forum.

This project is in an early stage. Some features are not implemented yet.

Installation

You can find installation instructions for Linux and macOS on the Sugarcube Tools website.

From Source

The following prerequesites are required to build Ncube from source:

• A recent version of Rust. Rustup is a great way to do so. The minimum supported version is 1.40.0+.
• NodeJS to build and bundle the UI. Version 12 of NodeJS is tested, but it might work with other versions as well.
• NodeJS comes with npm as a package manager. This project prefers to use yarn. While it is possible to use npm all the build scripts assume yarn. Installation is quite simple.
• Builds are orcehstrated using make. macOS and most Linux distributions have make installed out of the box.
• For OSX, the Apple's Xcode Command Line Tools is also required and it may be installed with xcode-select --install.

Verify that all build dependencies are satisfied.

$ cargo --version
rustc 1.42.0 (b8cedc004 2020-03-09)

$ node --version
v12.16.1

$ yarn --version
1.22.0

$ make --version
GNU Make 3.81

and if you are using OSX, verify xcode-select as well.

$ xcode-select --version
xcode-select version 2373.

Once all the dependencies are in place you can choose one of the following build targets. The build target that you most likely want can be simply build by running make:

• make :: Build a standalone binary of the desktop version of Ncube. The binary can be found in target/release/ncube.

There are more specialized build targets as well:

• make server :: Build the server version of Ncube. This produces two binaries target/release/ncubed and target/release/ncubectl.
• make pkg-dmg :: Build a DMG installer package of the desktop version of Ncube for macOS. The package can be found in pkgs.
• make pkg-deb :: Build a DEB package package of the desktop version of Ncube for Linux. The package can be found in pkgs.
• make pkg-deb-ncubed :: Build a DEB package of the server version of Ncube for Linux. The package can be found in pkgs.
• make pkg-web-ext :: Build a ZIP package of the Discovery Browser plugin. The package can be found in pkgs.

If something goes wrong or you want to make a clean build from scratch clean the old builds first:

• make clean :: Remove all previous build assets.

Documentation

All documentation can be found in the doc directory.

The HTTP endpoints of ncubed are described in the HTTP API documentation.

The account authorization for remote workspaces is described in a dedicated document.

The high level architecture of Ncube is described with a series of diagrams. To re-generate the architecture diagrams install fc4 and regenerate the images:

fc4 -fsr doc/diagrams

Ncube Discovery

Ncube is accompanied by a browser extension for the Firefox browser and for the Google Chrome browser for a smoother investigation workflow. The browser extension allows to store URL from the browser as sources directly in Ncube.

Installation

The browser extension communicates with the Ncube application that has to run on your local computer. Before using the browser extension see above for ways to install the desktop version of Ncube. The discovery plugin only works while Ncube is running.

TODO: Publish extension to AMO and provide a download link.

Development

The following instructions are to test and develop the browser extension locally. Make sure to install all required dependencies.

yarn install

To build the full extension and package it up simply run:

make pkg-web-ext

This will produce the package in the pkgs directory.

To develop locally run compilation of the web extension in one terminal:

yarn web-ext:watch

In another terminal start a development version of Firefox that contains the browser extension:

yarn web-ext

The source code of the browser extension and Ncube is shared. The entry point for the discovery source code can be found in src/ts/discovery.tsx.

Development

Ncube consists of several parts:

• ncubed is the backend of Ncube that exposes all functionality of Ncube.
• ncubectl is a CLI tool to manage server installations of Ncube.
• The frontend UI is a single page web app that communicates to ncubed.
• ncube is the full desktop app including ncubed and wrapping the frontend UI in a local browser window.

See the installation from source section to setup all the build dependencies.

Since the backend delivers the frontend using it's own HTTP server, the frontend assets must be available when compiling the backend. there is an open issue to resolve this but until then before compiling the backend you need to run the following once, and everytime you clean the target directory.:

make ui
make workspace

Open a terminal in the project root and start the backend:

cargo run --bin ncubed

In a different terminal start the webpack development server to build the UI:

yarn start

This will open a browser at http://localhost:8080 that provides the UI.

Tests

The tests can be run by executing the following command:

make test

License

All code is copyrighted by [e-mail] and licensed under the GPL3.

Support

Ncube was developed with the support of: