NOTICE: Human Framework is still in the Proof-of-Concept stage
CONTRIBUTE! I am releasing Human Framework under GPLv3 which makes it open source and free.
Contribute by:
- Localization of training data (translation)
- Adding more intents and actions
- Reporting bugs
- Suggesting new features
NOTICE: I am working on a Rasa NLU support
Human Framework is a test automation framework designed for testers without software programming background by allowing them to write test steps or instructions in their own language like English.
Human Framework uses Natural Language Understanding (NLU) for intent classification and entity extraction to be able to perform specific actions. It currently depend on Microsoft LUIS (Language Understanding).
Below is an example of test instructions that Human Framework can understand.
To get an idea of what specific sentences are supported, check the the content of trials
and tests
folder.
Open chrome browser https://devpost.com
Page title should be "Devpost - The home for hackathons"
Page should contain link "About"
Click link "About"
Page title should be "About us · Devpost"
Close browser
LUIS (and other NLU tools) makes it possible to write test cases on any human language. Current version of Human Framework only supports English.
Human Framework was inspired by Robot Framework but leans towards Natural Language Processing (NLP) for writing test cases.
"Human Framework is the Siri or Alexa of automation testing." (minus the voice, but it is possible in the future)
The architecture of Human Framework can be simplified into 2 steps- Intent Classification and Action Execution:
- Human Framework reads the test case files and then uses an Intent Classifier (e.g. LUIS.ai) to identify intents and entities. For example, the sentence "open chrome browser" will be identified with the intent "web.open_browser" and entity "chrome" as the browser type.
- Human Framework then executes the action specific for the classified intent. For example, the intent "web.open_browser" will use Selenium to automate opening a chrome browser.
-
Go to Azure portal and create an account.
-
On the Azure portal dashboard, click
Create a resource
, look forCognitive Services
and clickCreate
-
Fill up form and click
Create
button
-
Download the Human Framework training data from https://raw.githubusercontent.com/roniemartinez/HumanFramework/master/train/en.json and save it with
.json
extension -
Go to LUIS.ai and create an account
-
Click
Import new app
-
Click
Choose app file (JSON format) ...
and select the downloaded Human Framework training data -
Optionally, type your desired app name
-
Click
Done
-
Click on the application you created.
-
On the application dashboard, click
Train
to train your LUIS.ai app
-
Click
MANAGE
>Azure Resources
>Add prediction resource
-
Fill form
-
Click
Publish
, selectProduction
and then click thePublish
button -
Go back to
Azure Resources
. If you published your application toStaging
environment, useChange query parameters
to switch between environments. -
Copy the endpoint assigned resource in #2
NOTE: After copying the endpoint, enter it to a browser's address bar to make sure that it is accessible., Otherwise, repeat the steps or use the Starter_Key
which will only give you 1000 API calls
Human Framework is written in Python 3. Start by downloading/installing Python from python.org.
To install Human Framework, enter the following command into the terminal/cmd:
pip install -e git+git@github.com:roniemartinez/HumanFramework.git#egg=humanframework
pip install humanframework
Use the copied endpoint from LUIS.ai and setup Human Framework. Paste the endpoint enclosed in double quotes.
human config --luis-endpoint "<endpoint>"
Human Framework depends on Selenium for testing web applications. Download your desired webdriver and extract them to your local working directory.
Write a text file (test_*.txt) containing your tests and save it to local working directory where you placed the Selenium Drivers.
For example, if we have a file named test_web.txt
:
open chrome https://python.org
page title should be "Welcome to Python.org"
close browser
To run the tests in test_web.txt
, enter the following on the terminal/cmd:
human --test test_web.txt
Files inside the trials
folder that starts with the text test_
are automatically executed.
trials
|- test_simple.txt
|- test_simple_2.txt
|- ...
With this structure, you can simply type human
to run the test cases.
- Integrated Editor - While testing was greatly simplified because Human Framework will just execute sentences, using the terminal to install Python and several libraries is still more developer-centric. To remove this technical requirement, another major goal of the Human Framework project is to release a one-click installer which should include new features like an integrated IDE/Editor. This will greatly increase tester's productivity.
- Language Support - Not all testers use the English language. Supporting most languages will benefit more testers around the world.
- NLU Options - While LUIS.ai is very simple to use and accurate compared to other Intent Classifiers, it is not cheap. To support more testers, a free option like Rasa NLU can be used as an alternative.
I only implemented a few intents and actions for the proof-of-concept stage.
To know what are the possible tests that you can do with Human Framework, check the contents of trials
and tests
folders.