Note: If you haven't checked out how to set up a living documentation article, you can find it here.

Sometimes you need to run your tests against different resources or combinations of resources, or indifferent environments (Testing, Staging, Production). To achieve this, multiple jobs (often called Matrix Jobs or Builds) are created in your CI pipeline to trigger Cucumber with these different combinations.

Behave Pro can display the pass and fail results for each of the different Job runs of Cucumber but the results need to be published to Behave Pro for each job.

To tell Behave Pro which combination of properties a test report belongs to, you need to include one or more parameters using the X-PARAM header in the HTTP request as shown below in the example.

Curl:

curl -L -X PUT https://test-reports.behave.pro/REST/1.0/bdd/report \

-H "X-API-KEY: your_api_key” \

-H "X-COMMIT-ID: your_commit_sha" \

-H "X-PARAM: Os=Linux" \

-H "X-PARAM: JVM=Java 1.8" \

--data-binary @./target/cucumber-reports/cucumber.json

Powershell:

Invoke-WebRequest https://test-reports.behave.pro/REST/1.0/bdd/report `

-Method PUT `

-Headers @{'X-COMMIT-ID'='your_commit_sha';'X-API-KEY'='your_api_key';'X-PARAM'='Os=Linux, JVM=Java 1.8'} `

-Infile ./target/cucumber-reports/cucumber.json

Note: If you are using Jira Server or Jira Data Center, the URL to upload the test results is the domain name of your Jira followed by /rest/behavepro/1.0/bdd

For example: https://jira.example.com/rest/behavepro/1.0/bdd

In the screenshot below, you can see that the test results are separated by the Java version and grouped together by the OS type. This is purely for example purposes, you don't have to use OS and Java parameters, you can specify any parameter you want to.

Note: Please note that the order of the parameters matters how are they going to be displayed. In the example above the parameter that specifies the OS type has to be the first one.

These parameters are provided via the X-PARAM header and the header value must be a key-value pair, for example OS=Linux. Please note that the order of these parameters matters. In the curl example above, the Os=Linux parameter is treated with higher priority than the JVM=Java 1.8 parameter. If your tool doesn't support multiple header values within the same header name, you can use a comma to delimiter the parameters within the same header.

Example: X-PARAM: Os=Linux, JVM=Java 1.8.

You can also create a deeper structure with the matrix build test results by providing additional parameter with the X-PARAM header. The following example was created with the following headers via a curl command:

-H "X-PARAM: Os=Mac OS" -H "X-PARAM: JVM=Java 1.8"\

-H "X-PARAM: Test=World"

You can send different parameters for different tests. To achieve that you need to send 2 parameters within the HTTP request.
​
As an example, browser testing for matrix job can be below:

curl -L -X PUT https://test-reports.behave.pro/REST/1.0/bdd/report \

-H "X-API-KEY: your_api_key” \

-H "X-COMMIT-ID: your_commit_sha" \

-H "X-PARAM: TestType=Browser" \

-H "X-PARAM: Browser=Chrome” \

--data-binary @./target/cucumber-reports/cucumber.json

Or for the mobile tests and if you use Android example curl can be:

curl -L -X PUT https://test-reports.behave.pro/REST/1.0/bdd/report \

-H "X-API-KEY: your_api_key” \

-H "X-COMMIT-ID: your_commit_sha" \

-H "X-PARAM: TestType=Mobile" \

-H "X-PARAM: Os=Android” \

--data-binary @./target/cucumber-reports/cucumber.json