code-dot-org / dance-stress-test

Run Dance Party against real mobile devices in the cloud

Dance stress tests

Runs Dance Party against real iOS and Android devices using AWS Device Farm.

Running on AWS Device Farm

Based on this setup guide so check there if you get lost.

System Dependencies

1. Install Python 2

2. Install pip

   For Ubuntu use the python-pip package:

   sudo apt install python-pip

3. Install python virtualenv

   pip install virtualenv

Project setup

1. Clone this repo

   git clone https://github.com/code-dot-org/dance-stress-test.git
   cd dance-stress-test

2. Create a virtualenv workspace

   virtualenv .

3. Active your workspace in your current terminal

   You'll need to do this each time you open a new interactive terminal session for this project.

   source bin/activate

4. Install additional project dependencies into your workspace

   pip install -r project_requirements.txt

5. Verify you can load and list tests

   py.test --collect-only tests

Running tests on Device Farm

1. Run the buildzip script to generate a package for upload to Device Farm

   ./buildzip
   # Expect output to end with:
   # 2.0M	test_bundle.zip

2. Use AWS Device Farm to run the tests.

   Sign into AWS Console

   click here to open our AWS Device Farm project.

   Click "Create a new run"

   At "Choose your application" pick "Test a web application." The run name is just a label.

   At "Configure" pick "Appium Python" and upload the test_bundle.zip you just generated. After the zip file is processed more settings will pop up - use "Run your test in a custom environment" and the default YAML spec.

   At "Select devices" use whatever device pool you like - I've been using iPhone 5s and Samsung Galaxy S4.

   At "Review and start run" use a 30 minute timeout.

Note

Due to how many tests there are, it might be easiest to run a subset of tests on each Device Farm run.

Local testing

Not solved yet! This is more difficult than it looks