testimio / testim-analyzer

[ Unofficial ] Analyzes testim results and provides insights, downloads standalone reports with screenshots and videos.

Testim-Analyzer

This is a fully responsive, interactive, predictive, mobile friendly, emailable, auto generated website that has the ability to analyze failed tests results and provide meaningful insights, create individual testim reports with screenshots & video.

Disclaimer: This project is provided "AS IS". It is not officially supported by Testim.

Table of contents:

• Testim-Analyzer
• Pre-requisite:
  • Windows:
  • Mac:
• Steps to execute:
  • Mac:
  • Windows:
    
• Analyze Test Runs:
  • via downloaded Test Runs csv:
    • in-depth analysis:
    • high level analysis:
  • via Testim API:
    • in-depth analysis:
      
    • high level analysis:
      • dates:
      • execution id/ run id:
      • execution name:
      • result label:
  • Advanced Customizations:
• Generate standalone reports:
  • without downloaded screenshots & video:
    • for both passed & failed tests:
      • via report-file from testim CLI/ docker:
      • via downloaded test runs csv:
      • via API result id:
    • for failed tests only via API:
      • dates:
      • execution name:
      • execution id/ run id:
  • with downloaded screenshots & video:

Pre-requisite:

• Install docker.
  
• Download/ clone this github project.
  

Windows:

• Install git
  

Mac:

• Open terminal navigate to folder where run.sh is present & run the below command:
  

chmod +x ./run.sh

Steps to execute:

• Enter the following commands in terminal/ command prompt & then choose your preferred parameters from the following sections to analyze test runs/ download single reports with screenshots & videos.

Mac:

./run.sh "<PROJECT_NAME>" "<PROJECT_ID>" "<API_KEY>" 

Windows:

sh run.sh "<PROJECT_NAME>" "<PROJECT_ID>" "<API_KEY>" 

notes:

• 1st argument: Replace <PROJECT_NAME> with your testim project name.
• 2nd argument: Replace <PROJECT_ID> with your testim project id.
• 3rd argument: Replace <API_KEY> with Testim's API Key.

Analyze Test Runs:

via downloaded Test Runs csv:

Pre-requisite: Download the required test runs csv file to the current working directory.

in-depth analysis:

"csvFile=on|trends=on|email=on|deep_analytics=on|excel=on|showIssues=on"

high level analysis:

"csvFile=on"

notes:

• 4th argument: csvFile=on will analyze any test runs csv file present in the current working directory.
• There are other configurable filters/ options to enable advanced analysis.

via Testim API:

Note: Change branch as applicable.

in-depth analysis:

Note: Append |trends=on|email=on|deep_analytics=on|excel=on|showIssues=on with any of the below options. E.g. (Dates)

"startDate=2022-06-23|endDate=2022-06-23|branch=master|trends=on|email=on|deep_analytics=on|excel=on|showIssues=on"

high level analysis:

dates:

"startDate=2022-07-12|endDate=2022-07-12|branch=master"

execution id/ run id:

"runId=hKqljO1q6WMi8KJn|startDate=2022-06-23|endDate=2022-06-23|branch=master"

notes:

• RunId can be retrieved from single execution details view url (after suites/)
  
• Supports more than one runId seperated by semicolon.
  
  

execution name:

"executionName=pass, fail & skip|startDate=2022-06-23|endDate=2022-06-23|branch=master"

notes:

• executionName can be retrieved from Testim's Executions page
  
• Supports more than one executionName seperated by semicolon.
  
  

result label:

"resultLabel=pass, fail & skip|startDate=2022-06-23|endDate=2022-06-23|branch=master"

notes:

• Refer here to add result labels to your tests.
• resultLabel can be retrieved from Testim's Executions page
  
• Supports more than one resultLabel seperated by semicolon.
  
  

Screenshot:

Advanced Customizations:

Change the background color, debug with logs, customize logo & custom version number in heading by editing the run.sh after -a params with the below bold bold text:

`testim-analyzer:latest -n "${NAME}" -a "${PARAMS}"` **-b white -d true -l www.wikipedia.org -v 1.0**

Generate standalone reports:

• Create standalone test reports with test info, test params, steps info, filter option, option to download screenshots & create a standalone report with screenshots & stitched together video.
• Each Screenshot can be zoomed in and clicked to view full screen.
• Table colors change based on Test status & Groups.
  

without downloaded screenshots & video:

for both passed & failed tests:

via report-file from testim CLI/ docker:

Prerequisite: Run testim CLI or docker with an additional parameter: -r ~/report.xml

"singleReport=on|reportFile=./report.xml"

notes:

• reportFile value corresponds to the report path mentioned with -r of testim CLI or docker
  

via downloaded test runs csv:

Pre-requisite: Download the required test runs csv file to the current working directory.

"csvFile=on|singleReport=on"

Note:

• Uses direct screenshot urls with a default validity of 30 days.
  
  

via API result id:

"singleReport=on|resultId=bIXYH4A51vScEg2i"

notes:

• Use this only when you want to download individual reports. Prefer to generate via report-file if executed from CLI/CI.
• resultId can be retrieved from single report url (after result-id=).
  
• Supports more than one resultId seperated by semicolon.
  
  

for failed tests only via API:

Note: Change branch as applicable.

dates:

"singleReport=on|startDate=2022-07-12|endDate=2022-07-12|branch=master"

execution name:

"singleReport=on|executionName=pass, fail & skip|startDate=2022-06-23|endDate=2022-06-23|branch=master"

notes:

• executionName can be retrieved from Testim's Executions page
  
• Supports more than one executionName seperated by semicolon.
  
  

execution id/ run id:

"singleReport=on|runId=iGsJuXvLxXwHU3HL|startDate=2022-06-23|endDate=2022-06-23|branch=master"

notes:

• runId can be retrieved from single execution details view url (after suites/)
  
• Supports more than one runId seperated by semicolon.
  
  

with downloaded screenshots & video:

Append downloadArtifacts=on to any of the generate standalone report options if you wish to download screenshots & video. [ Default - off ]
E.g (report-file):

"singleReport=on|reportFile=./report.xml|downloadArtifacts=on"

Sample Screenshot: