Running tests/suites

You can start test run in Copado Robotic Testing in three ways:

  • Manually by clicking the run button

  • Scheduling them with scheduler

  • Triggering runs from external systems using API key

Running tests manually

To Run test cases manually, just select the suite in Copado Robotic Testing UI and click Run button [4]. Prior to starting run, you can select if test runs will be recorded as a video file [1] and if test run will be streamed live [2]. You can also select if the run is considered "regression" test run or "development test run" [3].

suites run manual

Recording videos and streaming

You can record a video of:

  • All test cases

  • Only failed test cases

  • None (turns recording off)

Recorded videos can be found after run ends from "Test case summary" section:

2902

When streaming is enabled, a separate browser window will pop-up and display the test execution live:

suites run streaming
Recording videos is currently available for cloud robots running web or mobile test automation.
Streaming run is currently available for cloud robots running web test automation.

Regression and Development runs

You can select if run is considered Regression run (default) or Development run.

Reression runs are "normal" runs and all results from these runs are available for metrics and dashboards. Development runs are meant to be used when testing new changes to test cases themselves; you use Development run while validating that the test case works correctly. Development runs are not counted towards metric calculations or show up in metrics.

Development runs are only available for newer Kubernetes based projects.

Scheduling test runs

To schedule test runs to be run automatically, for example every day at 12:00, you can use Copado Robotic Testing’s built-in scheduler.

Open the built-in scheduler

Click the test suite name you want to schedule to be run. Then click "Scheduler" button under test suite name:

suites scheduler icon

Using the scheduler

Select frequency [1] (daily, every x minutes, monthly etc.). and time [2] and [3] of test run using the scheduler dialog. Note that additional hours and minutes fields [2,3] depend on frequency selection. For example, if you select "Monthly" as frequency, then your additional selections will be date and time on every month.

Optionally, you can also directly give the schedule using the Crontab format [4].

You can record videos [5] of test runs with the exact same rules as with manual runs.

When you click Save [6], the schedule will be applied. With Delete [7] button you can remove existing schedule.

suites scheduler dialog

Setting up notifications

You can also setup Copado Robotic Testing to send notifications after test run. Notifications can be sent as an email or as a Slack message.

Click Notifications button

Click the test suite name you want to schedule to be run. Then click "Notifications" button under test suite name:

suites notifications icon

Add a new email notification

Click the (+) icon in order to add a new notification:

suites notifications add

Select Email as Type.

Select which trigger to use for your notification (test execution starts, test execution finalizes and there are failures etc.)

Fill in the recipients who will receive the email notification using Add users/groups -button.

suites notifications email

Add a new Slack notification

Click the (+) icon in order to add a new notification:

suites notifications add

Select Slack as Type.

Select which trigger to use for your notification (test execution starts, test execution finalizes and there are failures etc.)

Fill in the Slack Webhook URL and generate Slack application to your organizations Slack instance. More information about this can be found from Slack’s documentation.

suites notifications slack

Triggering runs from external sources

Triggering runs using API key is meant to be used, if you want to hook the test robot and suite into your Continuous Integration pipeline or other external scripts. Once Copado Robotic Testing’s api is called with generated API key, it will trigger the run for suite corresponding to API key and return results.

Create an API key for a suite

To get started, select the suite you want to create an API key to. You can do this by clicking the suites name.

After the suite is selected, click the "API options" icon:

suites running api key icon

The next screen allows you to Generate and API key for the suite, or remove/revoke one if you had already created an API key earlier:

suites running api key generate
suites running api key created
You need to have Editor rights to robot in order to be able to generate API key.
At this phase, remember to copy the generated API key. Store it securely, since it provides access for running your test suite. You will need to include the API key to your script calling the Copado Robotic Testing API.

Use the API key in scripts

To trigger runs using API, use the generated API key in your scripts. You can do this in basically any language that can call REST APIs, and most of our customers have their own favorite language.

We will provide an bash example script here, since it’s easy to use this in linux based continuous integration setups.

You will need to provide the following information to API:

  • PACE_API_URL

  • PACE_API_KEY

    • The API key you just generated for your suite

  • inputParameters (optional)

    • Additional parameters/variables for your run (see the example below)

#!/bin/bash
PACE_API_URL=https://pace.qentinel.com/pace/v4/builds
PACE_API_KEY=[YOUR_API_KEY_HERE]
# Start the build
BUILD=$(curl -sS -d '{"key": "'${PACE_API_KEY}'", "inputParameters": [{"key": "BROWSER", "value": "firefox"}]}' -H "Content-Type: application/json" -X POST ${PACE_API_URL})
echo "${BUILD}"
BUILD_ID=$(echo "${BUILD}" | grep -Po '"id":\K[0-9]+')
if [ -z "${BUILD_ID}" ]; then
  exit 1
fi
echo -n "Executing tests "
STATUS='"executing"'
# Poll every 10 seconds until the build is finished
while [ "${STATUS}" == '"executing"' ]; do
  sleep 10
  RESULTS=$(curl -sS ${PACE_API_URL}/${BUILD_ID}?key=${PACE_API_KEY})
  STATUS=$(echo "${RESULTS}" | grep -Po '"status": *\K"[^"]*"' | head -1)
  echo -n "."
done
echo " done!"
FAILURES=$(echo ${RESULTS} | grep -Po '"failures":\K[0-9]+')
LOG_REPORT_URL=$(echo "${RESULTS}" | grep -Po '"logReportUrl": *\K"[^"]*"')
echo "Report URL: ${LOG_REPORT_URL}"