willfrew / gemini

Utility for regression testing of web pages using screenshots

Gemini

Gemini is a utility for regression testing the visual appearance of web pages.

Gemini allows you to:

• Work with different browsers:

  • Google Chrome (tested in latest version)
  • Mozilla Firefox (tested in latest version)
  • IE8+
  • Opera 12+

• Test separate sections of a web page

• Include the box-shadow and outline properties when calculating element position and size

• Ignore some special case differences between images (rendering artifacts, text caret, etc.)

• Gather CSS test coverage statistics

Gemini was created at Yandex and is especially useful to UI library developers.

Quick start

Installing

npm install -g gemini
npm install -g selenium-standalone
selenium-standalone install

Configuring

Put the .gemini.js file in the root of your project:

module.exports = {
    rootUrl: 'http://yandex.ru',
    gridUrl: 'http://127.0.0.1:4444/wd/hub',

    browsers: {
        chrome: {
            desiredCapabilities: {
                browserName: 'chrome'
            }
        }
    }
};

Writing tests

Write a test and put it in the gemini folder in the root of your project:

gemini.suite('yandex-search', (suite) => {
    suite.setUrl('/')
        .setCaptureElements('.home-logo')
        .capture('plain');
});

Running tests

Start selenium-standalone in a separate tab before running the tests:

selenium-standalone start

Run gemini tests with flat reporter:

gemini test --reporter flat

Dependencies

Required software:

1. WebDriver server implementation. There are several options:

   • Selenium Server — for testing in different browsers. Launch with the selenium-standalone start command.

   • ChromeDriver — for testing in Google Chrome. Launch with the chromedriver --port=4444 --url-base=wd/hub command.

   • PhantomJS — launch with the phantomjs --webdriver=4444 command.

   • Cloud WebDriver services, such as SauceLabs or BrowserStack

2. Compiler with support for C++11 (GCC@4.6 or higher). This is a png-img requirement. Compiling on Windows machines requires the node-gyp prerequisites.

Installing

To install the utility, use the npm install command:

npm install -g gemini

Global installation is used for launching commands.

Configuring

Gemini is configured using a config file at the root of the project. Gemini can use one of the following files:

• .gemini.conf.js
• .gemini.conf.json
• .gemini.conf.yml
• .gemini.js
• .gemini.json
• .gemini.yml

Let's say we want to run our tests only in the locally installed PhantomJS.

In this case, the minimal configuration file will only need to have the root URL of your web app and the WebDriver capabilities of PhantomJS: For example,

rootUrl: http://yandex.com
browsers:
  PhantomJS:
    desiredCapabilities:
      browserName: phantomjs

Also, you need to run PhantomJS manually in WebDriver mode:

phantomjs --webdriver=4444

If you are using a remote WebDriver server, you can specify its URL with the gridUrl option:

rootUrl: http://yandex.com
gridUrl: http://selenium.example.com:4444/wd/hub

browsers:
  chrome:
    desiredCapabilities:
      browserName: chrome
      version: "45.0"

  firefox:
    desiredCapabilities:
      browserName: firefox
      version: "39.0"

You can also set up each browser to have its own node:

rootUrl: http://yandex.com

browsers:
  chrome:
    gridUrl: http://chrome-node.example.com:4444/wd/hub
    desiredCapabilities:
      browserName: chrome
      version: "45.0"

  firefox:
    gridUrl: http://firefox-node.example.com:4444/wd/hub
    desiredCapabilities:
      browserName: firefox
      version: "39.0"

Other configuration options

See the details of the config file structure and available options.

Writing tests

Each of the blocks that are being tested may be in one of the determined states. States are tested with the help of chains of step-by-step actions declared in a block's test suites.

For example, let's write a test for a search block at yandex.com:

gemini.suite('yandex-search', function(suite) {
    suite.setUrl('/')
        .setCaptureElements('.main-table')
        .capture('plain')
        .capture('with text', function(actions, find) {
            actions.sendKeys(find('.input__control'), 'hello gemini');
        });
});

We are creating a new test suite yandex-search, assuming that we will capture the .main-table element from the root URL http://yandex.com. We know that the block has two states:

• plain — right after the page is loaded
• with text — with the hello gemini text inserted into .input__control

States are executed one after another in the order in which they are defined, without the browser reloading in between.

See the details of test creation methods.

Using CLI

To complete the test creation procedure, you need to take reference shots using the following command:

gemini update [paths to test suites]

To launch a test (to compare the current state of a block with a reference shot), use the command:

gemini test [paths to test suites]

See the details of interaction with CLI and available options.

GUI

You can use the Gemini graphical user interface instead of the command line. It is located in the gemini-gui package and must be installed additionally:

npm install -g gemini-gui

GUI advantages:

• Handy preview of reference shots

• Clear real-time demonstration of the differences between a reference shot and the current state of a block

• Easy to update reference shots

Plugins

Gemini can be extended with plugins. You can choose from the existing plugins or write your own. To use a plugin, install and enable it in your .gemini.yml:

system:
  plugins:
    some-awesome-plugin:
      plugin-option: value

HTML report

To see the difference between the current state of a block and a reference picture more clearly, use the HTML reporter - plugin for gemini. This plugin produces HTML report, which displays reference image, current image and differences between them, for each state in each browser. When all tests are completed, you will see a link to HTML report.

Programmatic API

To use Gemini in your scripts or build tools, you can use the experimental programmatic API.

Events

To learn more about all events in Gemini, see the events documentation.