:tocdepth: 0
.. _/plugins/report:
Report Plugins
~~~~~~~~~~~~~~
.. include:: report-header.inc.rst
.. _/plugins/report/common-keys:
Common Keys
-----------
The following keys are accepted by all plugins of the ``report`` step.
name
The name of the step phase.
Environment variable: ``TMT_PLUGIN_REPORT_DISPLAY_NAME``
In plan metadata:
.. code-block:: yaml
name:
On command-line:
.. code-block:: shell
--name ...
export TMT_PLUGIN_REPORT_DISPLAY_NAME=...
order
Order in which the phase should be handled.
Default: ``50``
Environment variable: ``TMT_PLUGIN_REPORT_DISPLAY_ORDER``
In plan metadata:
.. code-block:: yaml
order:
On command-line:
.. code-block:: shell
--order ...
export TMT_PLUGIN_REPORT_DISPLAY_ORDER=...
summary
Concise summary describing purpose of the phase.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_DISPLAY_SUMMARY``
In plan metadata:
.. code-block:: yaml
summary:
On command-line:
.. code-block:: shell
--summary ...
export TMT_PLUGIN_REPORT_DISPLAY_SUMMARY=...
when
If specified, phase is run only if any rule matches plan context.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_DISPLAY_WHEN``
In plan metadata:
.. code-block:: yaml
when: RULE
On command-line:
.. code-block:: shell
--when RULE
export TMT_PLUGIN_REPORT_DISPLAY_WHEN=RULE
.. _/plugins/report/display:
display
-------
.. warning::
Please, be aware that the documentation below is a work in progress. We are
working on fixing it, adding missing bits and generally making it better.
Also, it was originally used for command line help only, therefore the
formatting is often suboptimal.
Show results on the terminal.
Give a concise summary of results produced by tests and ``prepare``
phases, directly on the terminal.
Allows to select the desired level of verbosity:
.. code-block:: yaml
tmt run -l report # overall summary only
tmt run -l report -v # individual test results
tmt run -l report -vv # show full paths to logs
tmt run -l report -vvv # provide complete test output
Configuration
^^^^^^^^^^^^^
See also :ref:`Common Keys` accepted by the plugin.
display-guest
When to display full guest name in report: when more than a single guest was involved
(default), always, or never.
Default: ``auto``
Environment variable: ``TMT_PLUGIN_REPORT_DISPLAY_DISPLAY_GUEST``
In plan metadata:
.. code-block:: yaml
display-guest: auto|always|never
On command-line:
.. code-block:: shell
--display-guest 'auto|always|never'
export TMT_PLUGIN_REPORT_DISPLAY_DISPLAY_GUEST='auto|always|never'
----
.. _/plugins/report/html:
html
----
.. warning::
Please, be aware that the documentation below is a work in progress. We are
working on fixing it, adding missing bits and generally making it better.
Also, it was originally used for command line help only, therefore the
formatting is often suboptimal.
Format test results into an HTML report.
Create a local ``html`` file with test results arranged in
a table. Optionally open the page in the default browser.
Example config:
.. code-block:: yaml
# Enable html report from the command line
tmt run --all report --how html
tmt run --all report --how html --open
tmt run -l report -h html -o
.. code-block:: yaml
# Use html as the default report for given plan
report:
how: html
open: true
Configuration
^^^^^^^^^^^^^
See also :ref:`Common Keys` accepted by the plugin.
absolute-paths
Make paths absolute rather than relative to working directory.
Default: ``false``
Environment variable: ``TMT_PLUGIN_REPORT_HTML_ABSOLUTE_PATHS``
In plan metadata:
.. code-block:: yaml
absolute-paths: true|false
On command-line:
.. code-block:: shell
--absolute-paths
export TMT_PLUGIN_REPORT_HTML_ABSOLUTE_PATHS=1|0
display-guest
When to display full guest name in report: when more than a single guest was involved
(default), always, or never.
Default: ``auto``
Environment variable: ``TMT_PLUGIN_REPORT_HTML_DISPLAY_GUEST``
In plan metadata:
.. code-block:: yaml
display-guest: auto|always|never
On command-line:
.. code-block:: shell
--display-guest 'auto|always|never'
export TMT_PLUGIN_REPORT_HTML_DISPLAY_GUEST='auto|always|never'
file
Path to the file to store HTML report to.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_HTML_FILE``
In plan metadata:
.. code-block:: yaml
file: PATH
On command-line:
.. code-block:: shell
--file PATH
export TMT_PLUGIN_REPORT_HTML_FILE=PATH
open
Open results in your preferred web browser.
Default: ``false``
Environment variable: ``TMT_PLUGIN_REPORT_HTML_OPEN``
In plan metadata:
.. code-block:: yaml
open: true|false
On command-line:
.. code-block:: shell
--open
export TMT_PLUGIN_REPORT_HTML_OPEN=1|0
----
.. _/plugins/report/junit:
junit
-----
.. warning::
Please, be aware that the documentation below is a work in progress. We are
working on fixing it, adding missing bits and generally making it better.
Also, it was originally used for command line help only, therefore the
formatting is often suboptimal.
Save test results in chosen JUnit flavor format.
When flavor is set to custom, the ``template-path`` with a path to a custom template must be
provided.
When ``file`` is not specified, output is written into a file named ``junit.xml`` located in
the current workdir.
.. code-block:: yaml
# Enable junit report from the command line
tmt run --all report --how junit
tmt run --all report --how junit --file test.xml
.. code-block:: yaml
# Use junit as the default report for given plan
report:
how: junit
file: test.xml
Possible test results and their mapping to junit tags:
* ``pass`` - no specific tag, test has succeeded
* ``fail`` - ```` tag
* ``skip`` or ``info`` - ```` tag
* ``error``, ``warn``, or ``pending`` - ```` tag
.. note::
For neater JUnit XML and XML validation against the XSD, ``lxml`` package is required
by the ``report/junit`` plugin.
To quickly test ``lxml`` presence, you can try running ``python -c 'import lxml'``.
* Users who installed tmt from system repositories should install ``tmt+report-junit``
package.
* Users who installed tmt from PyPI should install ``tmt[report-junit]`` extra.
Configuration
^^^^^^^^^^^^^
See also :ref:`Common Keys` accepted by the plugin.
file
Path to the file to store JUnit to.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_JUNIT_FILE``
In plan metadata:
.. code-block:: yaml
file: PATH
On command-line:
.. code-block:: shell
--file PATH
export TMT_PLUGIN_REPORT_JUNIT_FILE=PATH
flavor
Name of a JUnit flavor to generate.
Default: ``default``
Environment variable: ``TMT_PLUGIN_REPORT_JUNIT_FLAVOR``
In plan metadata:
.. code-block:: yaml
flavor: default|custom|subresults
On command-line:
.. code-block:: shell
--flavor 'default|custom|subresults'
export TMT_PLUGIN_REPORT_JUNIT_FLAVOR='default|custom|subresults'
include-output-log
Include full standard output in resulting xml file.
Default: ``true``
Environment variable: ``TMT_PLUGIN_REPORT_JUNIT_INCLUDE_OUTPUT_LOG``
In plan metadata:
.. code-block:: yaml
include-output-log: true|false
On command-line:
.. code-block:: shell
--include-output-log
export TMT_PLUGIN_REPORT_JUNIT_INCLUDE_OUTPUT_LOG=1|0
prettify
Enable the XML pretty print for generated JUnit file. This option is always disabled
for 'custom' template flavor.
Default: ``true``
Environment variable: ``TMT_PLUGIN_REPORT_JUNIT_PRETTIFY``
In plan metadata:
.. code-block:: yaml
prettify: true|false
On command-line:
.. code-block:: shell
--prettify
export TMT_PLUGIN_REPORT_JUNIT_PRETTIFY=1|0
template-path
Path to a custom template file to use for JUnit creation.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_JUNIT_TEMPLATE_PATH``
In plan metadata:
.. code-block:: yaml
template-path: TEMPLATE_PATH
On command-line:
.. code-block:: shell
--template-path TEMPLATE_PATH
export TMT_PLUGIN_REPORT_JUNIT_TEMPLATE_PATH=TEMPLATE_PATH
----
.. _/plugins/report/polarion:
polarion
--------
.. warning::
Please, be aware that the documentation below is a work in progress. We are
working on fixing it, adding missing bits and generally making it better.
Also, it was originally used for command line help only, therefore the
formatting is often suboptimal.
Write test results into an xUnit file and upload to Polarion.
In order to get quickly started create a pylero config
file ``~/.pylero`` in your home directory with the
following content:
.. code-block:: ini
[webservice]
url=https://{your polarion web URL}/polarion
svn_repo=https://{your polarion web URL}/repo
default_project={your project name}
user={your username}
password={your password}
See the ``Pylero Documentation`` for more details on how
to configure the ``pylero`` module.
https://github.com/RedHatQE/pylero
.. note::
For Polarion report to export correctly you need to
use password authentication, since exporting the
report happens through Polarion XUnit importer which
does not support using tokens. You can still
authenticate with token to only generate the report
using ``--no-upload`` argument.
.. note::
Your Polarion project might need a custom value format
for the ``arch``, ``planned-in`` and other fields. The
format of these fields might differ across Polarion
projects, for example, ``x8664`` can be used instead
of ``x86_64`` for the architecture.
Examples:
.. code-block:: yaml
# Enable polarion report from the command line
tmt run --all report --how polarion --project-id tmt
tmt run --all report --how polarion --project-id tmt --no-upload --file test.xml
.. code-block:: yaml
# Use polarion as the default report for given plan
report:
how: polarion
file: test.xml
project-id: tmt
title: tests_that_pass
planned-in: RHEL-9.1.0
pool-team: sst_tmt
Configuration
^^^^^^^^^^^^^
See also :ref:`Common Keys` accepted by the plugin.
arch
Which architecture was this run executed on,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_ARCH.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_ARCH``
In plan metadata:
.. code-block:: yaml
arch: ARCH
On command-line:
.. code-block:: shell
--arch ARCH
export TMT_PLUGIN_REPORT_POLARION_ARCH=ARCH
assignee
Who is responsible for this test run,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_ASSIGNEE.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_ASSIGNEE``
In plan metadata:
.. code-block:: yaml
assignee: ASSIGNEE
On command-line:
.. code-block:: shell
--assignee ASSIGNEE
export TMT_PLUGIN_REPORT_POLARION_ASSIGNEE=ASSIGNEE
build
Which build was this run executed on,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_BUILD.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_BUILD``
In plan metadata:
.. code-block:: yaml
build: BUILD
On command-line:
.. code-block:: shell
--build BUILD
export TMT_PLUGIN_REPORT_POLARION_BUILD=BUILD
compose-id
Compose ID of image used for this run,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_COMPOSE_ID.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_COMPOSE_ID``
In plan metadata:
.. code-block:: yaml
compose-id: COMPOSEID
On command-line:
.. code-block:: shell
--compose-id COMPOSEID
export TMT_PLUGIN_REPORT_POLARION_COMPOSE_ID=COMPOSEID
description
Use specific test run description,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_DESCRIPTION.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_DESCRIPTION``
In plan metadata:
.. code-block:: yaml
description: DESCRIPTION
On command-line:
.. code-block:: shell
--description DESCRIPTION
export TMT_PLUGIN_REPORT_POLARION_DESCRIPTION=DESCRIPTION
file
Path to the file to store xUnit in.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_FILE``
In plan metadata:
.. code-block:: yaml
file: FILE
On command-line:
.. code-block:: shell
--file FILE
export TMT_PLUGIN_REPORT_POLARION_FILE=FILE
fips
FIPS mode enabled or disabled for this run.
Default: ``false``
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_FIPS``
In plan metadata:
.. code-block:: yaml
fips: true|false
On command-line:
.. code-block:: shell
--fips
export TMT_PLUGIN_REPORT_POLARION_FIPS=1|0
include-output-log
Include full standard output in resulting xml file.
Default: ``true``
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_INCLUDE_OUTPUT_LOG``
In plan metadata:
.. code-block:: yaml
include-output-log: true|false
On command-line:
.. code-block:: shell
--include-output-log
export TMT_PLUGIN_REPORT_POLARION_INCLUDE_OUTPUT_LOG=1|0
logs
Location of the logs for this test run,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_LOGS.
Ultimately also uses environment variable TMT_REPORT_ARTIFACTS_URL.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_LOGS``
In plan metadata:
.. code-block:: yaml
logs: LOGLOCATION
On command-line:
.. code-block:: shell
--logs LOGLOCATION
export TMT_PLUGIN_REPORT_POLARION_LOGS=LOGLOCATION
planned-in
Select a specific release to mark this test run with,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_PLANNED_IN.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_PLANNED_IN``
In plan metadata:
.. code-block:: yaml
planned-in: PLANNEDIN
On command-line:
.. code-block:: shell
--planned-in PLANNEDIN
export TMT_PLUGIN_REPORT_POLARION_PLANNED_IN=PLANNEDIN
platform
Which platform was this run executed on,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_PLATFORM.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_PLATFORM``
In plan metadata:
.. code-block:: yaml
platform: PLATFORM
On command-line:
.. code-block:: shell
--platform PLATFORM
export TMT_PLUGIN_REPORT_POLARION_PLATFORM=PLATFORM
pool-team
Which subsystem is this test run relevant for,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_POOL_TEAM.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_POOL_TEAM``
In plan metadata:
.. code-block:: yaml
pool-team: POOLTEAM
On command-line:
.. code-block:: shell
--pool-team POOLTEAM
export TMT_PLUGIN_REPORT_POLARION_POOL_TEAM=POOLTEAM
prettify
Enable the XML pretty print for generated XUnit file.
Default: ``true``
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_PRETTIFY``
In plan metadata:
.. code-block:: yaml
prettify: true|false
On command-line:
.. code-block:: shell
--prettify
export TMT_PLUGIN_REPORT_POLARION_PRETTIFY=1|0
project-id
Use specific Polarion project ID,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_PROJECT_ID. If no project ID
is found, the project ID is taken from pylero configuration default project setting as
a last resort.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_PROJECT_ID``
In plan metadata:
.. code-block:: yaml
project-id: ID
On command-line:
.. code-block:: shell
--project-id ID
export TMT_PLUGIN_REPORT_POLARION_PROJECT_ID=ID
sample-image
Which sample image was this run executed on,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_SAMPLE_IMAGE.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_SAMPLE_IMAGE``
In plan metadata:
.. code-block:: yaml
sample-image: SAMPLEIMAGE
On command-line:
.. code-block:: shell
--sample-image SAMPLEIMAGE
export TMT_PLUGIN_REPORT_POLARION_SAMPLE_IMAGE=SAMPLEIMAGE
template
Use specific test run template,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_TEMPLATE.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_TEMPLATE``
In plan metadata:
.. code-block:: yaml
template: TEMPLATE
On command-line:
.. code-block:: shell
--template TEMPLATE
export TMT_PLUGIN_REPORT_POLARION_TEMPLATE=TEMPLATE
test-cycle
Which test cycle this test run belongs to,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_TEST_CYCLE.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_TEST_CYCLE``
In plan metadata:
.. code-block:: yaml
test-cycle: TESTCYCLE
On command-line:
.. code-block:: shell
--test-cycle TESTCYCLE
export TMT_PLUGIN_REPORT_POLARION_TEST_CYCLE=TESTCYCLE
title
Use specific test run title,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_TITLE.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_TITLE``
In plan metadata:
.. code-block:: yaml
title: TITLE
On command-line:
.. code-block:: shell
--title TITLE
export TMT_PLUGIN_REPORT_POLARION_TITLE=TITLE
upload
Whether to upload results to Polarion,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_UPLOAD.
Default: ``true``
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_UPLOAD``
In plan metadata:
.. code-block:: yaml
upload: true|false
On command-line:
.. code-block:: shell
--upload
export TMT_PLUGIN_REPORT_POLARION_UPLOAD=1|0
use-facts
Use hostname and arch from guest facts,
also uses environment variable TMT_PLUGIN_REPORT_POLARION_USE_FACTS.
Default: ``false``
Environment variable: ``TMT_PLUGIN_REPORT_POLARION_USE_FACTS``
In plan metadata:
.. code-block:: yaml
use-facts: true|false
On command-line:
.. code-block:: shell
--use-facts
export TMT_PLUGIN_REPORT_POLARION_USE_FACTS=1|0
----
.. _/plugins/report/reportportal:
reportportal
------------
.. warning::
Please, be aware that the documentation below is a work in progress. We are
working on fixing it, adding missing bits and generally making it better.
Also, it was originally used for command line help only, therefore the
formatting is often suboptimal.
Report test results and their subresults to a ReportPortal instance via API.
For communication with Report Portal API is necessary to provide
following options:
* token for authentication
* url of the ReportPortal instance
* project name
In addition to command line options it's possible to use environment
variables:
.. code-block:: bash
export TMT_PLUGIN_REPORT_REPORTPORTAL_${MY_OPTION}=${MY_VALUE}
# Boolean options are activated with value of 1:
TMT_PLUGIN_REPORT_REPORTPORTAL_SUITE_PER_PLAN=1
Assuming the URL and token are provided by the environment variables,
the plan config can look like this:
.. code-block:: yaml
# Use ReportPortal as the default report for given plan
report:
how: reportportal
project: baseosqe
# Report context attributes for given plan
context:
...
environment:
...
.. code-block:: yaml
# Report description, contact, id and environment variables for given test
summary: ...
contact: ...
id: ...
environment:
...
Where the context and environment sections must be filled with
corresponding data in order to report context as attributes
(arch, component, distro, trigger, compose, etc.) and
environment variables as parameters in the Item Details.
Other reported fmf data are summary, id, web link and contact per
test.
Two types of data structures are supported for reporting to ReportPortal:
* ``launch-per-plan`` mapping (default) that results in launch-test structure.
* ``suite-per-plan`` mapping that results in launch-suite-test structure.
Supported report use cases:
* Report a new run in launch-suite-test or launch-test structure
* Report an additional rerun with ``launch-rerun`` option and same launch name (-> Retry items)
or by reusing the run and reporting with ``again`` option (-> append logs)
* To see plan progress, discover and report an empty (IDLE) run
and reuse the run for execution and updating the report with ``again`` option
* Report contents of a new run to an existing launch via the URL ID in three ways:
tests to launch, suites to launch and tests to suite.
Example:
.. code-block:: shell
# Enable ReportPortal report from the command line depending on the use case:
## Simple upload with all project, url endpoint and user token passed in command line
tmt run --all report --how reportportal --project=baseosqe --url="https://reportportal.xxx.com" --token="abc...789"
## Simple upload with url and token exported in environment variable
tmt run --all report --how reportportal --project=baseosqe
## Upload with project name in fmf data, filtering out parameters (environment variables)
## that tend to be unique and break the history aggregation
tmt run --all report --how reportportal --exclude-variables="^(TMT|PACKIT|TESTING_FARM).*"
## Upload all plans as suites into one ReportPortal launch
tmt run --all report --how reportportal --suite-per-plan --launch=Errata --launch-description="..."
## Rerun the launch with suite structure for the test results to be uploaded
## into the latest launch with the same name as a new 'Retry' tab
## (mapping based on unique paths)
tmt run --all report --how reportportal --suite-per-plan --launch=Errata --launch-rerun
## Rerun the tmt run and append the new result logs under the previous one
## uploaded in ReportPortal (precise mapping)
tmt run --id run-012 --all report --how reportportal --again
## Additional upload of new suites into given launch with suite structure
tmt run --all report --how reportportal --suite-per-plan --upload-to-launch=4321
## Additional upload of new tests into given launch with non-suite structure
tmt run --all report --how reportportal --launch-per-plan --upload-to-launch=1234
## Additional upload of new tests into given suite
tmt run --all report --how reportportal --upload-to-suite=123456
## Upload Idle tests, then execute it and add result logs into prepared empty tests
tmt run discover report --how reportportal --defect-type=Idle
tmt run --last --all report --how reportportal --again
Configuration
^^^^^^^^^^^^^
See also :ref:`Common Keys` accepted by the plugin.
api-version
Override the default reportportal API version (v1).
Default: ``v1``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_API_VERSION``
In plan metadata:
.. code-block:: yaml
api-version: VERSION
On command-line:
.. code-block:: shell
--api-version VERSION
export TMT_PLUGIN_REPORT_REPORTPORTAL_API_VERSION=VERSION
artifacts-url
Link to test artifacts provided for report plugins.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_ARTIFACTS_URL``
In plan metadata:
.. code-block:: yaml
artifacts-url: ARTIFACTS_URL
On command-line:
.. code-block:: shell
--artifacts-url ARTIFACTS_URL
export TMT_PLUGIN_REPORT_REPORTPORTAL_ARTIFACTS_URL=ARTIFACTS_URL
auto-analysis
Enable immediate auto-analysis of failed tests in ReportPortal. When enabled,
ReportPortal will automatically analyze failed test items and attempt
to identify defect types and patterns based on historical data. This requires
logs to be sent with each failure, and applies only to test items marked
with the ``To Investigate`` defect type.
Default: ``false``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_AUTO_ANALYSIS``
In plan metadata:
.. code-block:: yaml
auto-analysis: true|false
On command-line:
.. code-block:: shell
--auto-analysis
export TMT_PLUGIN_REPORT_REPORTPORTAL_AUTO_ANALYSIS=1|0
defect-type
Pass the defect type to be used for failed test, which is defined in the project
(e.g. 'Idle'). 'To Investigate' is used by default.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_DEFECT_TYPE``
In plan metadata:
.. code-block:: yaml
defect-type: DEFECT_NAME
On command-line:
.. code-block:: shell
--defect-type DEFECT_NAME
export TMT_PLUGIN_REPORT_REPORTPORTAL_DEFECT_TYPE=DEFECT_NAME
exclude-variables
Regular expression for excluding environment variables
from reporting to ReportPortal ('^TMT_.*' used by default).
Parameters in ReportPortal get filtered out by the pattern
to prevent overloading and to preserve the history aggregation
for ReportPortal item if tmt id is not provided.
Default: ``^TMT_.*``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_EXCLUDE_VARIABLES``
In plan metadata:
.. code-block:: yaml
exclude-variables: PATTERN
On command-line:
.. code-block:: shell
--exclude-variables PATTERN
export TMT_PLUGIN_REPORT_REPORTPORTAL_EXCLUDE_VARIABLES=PATTERN
launch
Set the launch name, otherwise name of the plan is used by default.
Should be defined with 'suite-per-plan' option or it will be named after the first plan.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH``
In plan metadata:
.. code-block:: yaml
launch: LAUNCH_NAME
On command-line:
.. code-block:: shell
--launch LAUNCH_NAME
export TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH=LAUNCH_NAME
launch-description
Pass the description for ReportPortal launch with 'suite-per-plan' option
or append the original (plan summary) with additional info.
Appends test description with 'upload-to-launch/suite' options.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH_DESCRIPTION``
In plan metadata:
.. code-block:: yaml
launch-description: DESCRIPTION
On command-line:
.. code-block:: shell
--launch-description DESCRIPTION
export TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH_DESCRIPTION=DESCRIPTION
launch-per-plan
Mapping launch per plan, creating one or more launches with no suite structure.
Default: ``false``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH_PER_PLAN``
In plan metadata:
.. code-block:: yaml
launch-per-plan: true|false
On command-line:
.. code-block:: shell
--launch-per-plan
export TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH_PER_PLAN=1|0
launch-rerun
Rerun the last launch based on its name and unique test paths to create Retry item
with a new version per each test. Supported in 'suite-per-plan' structure only.
Default: ``false``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH_RERUN``
In plan metadata:
.. code-block:: yaml
launch-rerun: true|false
On command-line:
.. code-block:: shell
--launch-rerun
export TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH_RERUN=1|0
launch-url
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH_URL``
In plan metadata:
.. code-block:: yaml
launch-url:
On command-line:
.. code-block:: shell
--launch-url ...
export TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH_URL=...
launch-uuid
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH_UUID``
In plan metadata:
.. code-block:: yaml
launch-uuid:
On command-line:
.. code-block:: shell
--launch-uuid ...
export TMT_PLUGIN_REPORT_REPORTPORTAL_LAUNCH_UUID=...
link-template
Jinja template that will be rendered for each test result and appended to the end
of its description. The following variables are passed to the template:
``PLAN_NAME``, ``RESULT``.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_LINK_TEMPLATE``
In plan metadata:
.. code-block:: yaml
link-template: TEMPLATE
On command-line:
.. code-block:: shell
--link-template TEMPLATE
export TMT_PLUGIN_REPORT_REPORTPORTAL_LINK_TEMPLATE=TEMPLATE
log-size-limit
Size limit in bytes for log upload to ReportPortal.
The default limit is 1 MB.
Maximum allowed limit is 32 MB.
Default: ``1 MB``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_LOG_SIZE_LIMIT``
In plan metadata:
.. code-block:: yaml
log-size-limit: SIZE
On command-line:
.. code-block:: shell
--log-size-limit SIZE
export TMT_PLUGIN_REPORT_REPORTPORTAL_LOG_SIZE_LIMIT=SIZE
project
Name of the project into which the results should be uploaded.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_PROJECT``
In plan metadata:
.. code-block:: yaml
project: PROJECT_NAME
On command-line:
.. code-block:: shell
--project PROJECT_NAME
export TMT_PLUGIN_REPORT_REPORTPORTAL_PROJECT=PROJECT_NAME
ssl-verify
Enable/disable the SSL verification for communication with ReportPortal.
Default: ``true``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_SSL_VERIFY``
In plan metadata:
.. code-block:: yaml
ssl-verify: true|false
On command-line:
.. code-block:: shell
--ssl-verify
export TMT_PLUGIN_REPORT_REPORTPORTAL_SSL_VERIFY=1|0
suite-per-plan
Mapping suite per plan, creating one launch and continuous uploading suites into it.
Recommended to use with 'launch' and 'launch-description' options.
Can be used with 'upload-to-launch' option for an additional upload of new suites.
Default: ``false``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_SUITE_PER_PLAN``
In plan metadata:
.. code-block:: yaml
suite-per-plan: true|false
On command-line:
.. code-block:: shell
--suite-per-plan
export TMT_PLUGIN_REPORT_REPORTPORTAL_SUITE_PER_PLAN=1|0
suite-uuid
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_SUITE_UUID``
In plan metadata:
.. code-block:: yaml
suite-uuid:
On command-line:
.. code-block:: shell
--suite-uuid ...
export TMT_PLUGIN_REPORT_REPORTPORTAL_SUITE_UUID=...
test-uuids
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_TEST_UUIDS``
In plan metadata:
.. code-block:: yaml
test-uuids:
On command-line:
.. code-block:: shell
--test-uuids ...
export TMT_PLUGIN_REPORT_REPORTPORTAL_TEST_UUIDS=...
token
The token to use for upload to the ReportPortal instance (from the user profile).
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_TOKEN``
In plan metadata:
.. code-block:: yaml
token: TOKEN
On command-line:
.. code-block:: shell
--token TOKEN
export TMT_PLUGIN_REPORT_REPORTPORTAL_TOKEN=TOKEN
traceback-size-limit
Size limit in bytes for traceback log upload to ReportPortal.
The default limit is 50 kB.
Maximum allowed limit is 32 MB.
Default: ``50 kB``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_TRACEBACK_SIZE_LIMIT``
In plan metadata:
.. code-block:: yaml
traceback-size-limit: SIZE
traceback-size-limit: 50 kB
traceback-size-limit: 1MB
On command-line:
.. code-block:: shell
--traceback-size-limit SIZE
export TMT_PLUGIN_REPORT_REPORTPORTAL_TRACEBACK_SIZE_LIMIT=SIZE
--traceback-size-limit '50 kB'
--traceback-size-limit 1MB
export TMT_PLUGIN_REPORT_REPORTPORTAL_TRACEBACK_SIZE_LIMIT='50 kB'
export TMT_PLUGIN_REPORT_REPORTPORTAL_TRACEBACK_SIZE_LIMIT=1MB
upload-log-pattern
List of regular expressions to look for in result log names. If any of the
patterns is found in a log file name, the log will be uploaded to ReportPortal.
Check result logs will be uploaded only if the check failed or if an error
occurred during the execution. The search mode is used for pattern matching.
See the :ref:`regular-expressions` section for details.
Default: ``avc\.txt``, ``dmesg-.*\.txt``, ``output\.txt``, ``tmt-watchdog\.txt``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_UPLOAD_LOG_PATTERN``
In plan metadata:
.. code-block:: yaml
upload-log-pattern: PATTERN
On command-line:
.. code-block:: shell
--upload-log-pattern PATTERN
export TMT_PLUGIN_REPORT_REPORTPORTAL_UPLOAD_LOG_PATTERN=PATTERN
upload-subresults
Enable/disable uploading of tmt subresults into the ReportPortal.
Default: ``false``
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_UPLOAD_SUBRESULTS``
In plan metadata:
.. code-block:: yaml
upload-subresults: true|false
On command-line:
.. code-block:: shell
--upload-subresults
export TMT_PLUGIN_REPORT_REPORTPORTAL_UPLOAD_SUBRESULTS=1|0
upload-to-launch
Pass the launch ID for an additional test/suite upload to an existing launch. ID can be
found in the launch URL. Keep the launch structure with options 'launch/suite-per-plan'.
To upload specific info into description see also 'launch-description'.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_UPLOAD_TO_LAUNCH``
In plan metadata:
.. code-block:: yaml
upload-to-launch: LAUNCH_ID
On command-line:
.. code-block:: shell
--upload-to-launch LAUNCH_ID
export TMT_PLUGIN_REPORT_REPORTPORTAL_UPLOAD_TO_LAUNCH=LAUNCH_ID
upload-to-suite
Pass the suite ID for an additional test upload to a suite
within an existing launch. ID can be found in the suite URL.
To upload specific info into description see also 'launch-description'.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_UPLOAD_TO_SUITE``
In plan metadata:
.. code-block:: yaml
upload-to-suite: SUITE_ID
On command-line:
.. code-block:: shell
--upload-to-suite SUITE_ID
export TMT_PLUGIN_REPORT_REPORTPORTAL_UPLOAD_TO_SUITE=SUITE_ID
url
The URL of the ReportPortal instance where the data should be sent to.
Default: *not set*
Environment variable: ``TMT_PLUGIN_REPORT_REPORTPORTAL_URL``
In plan metadata:
.. code-block:: yaml
url: URL
On command-line:
.. code-block:: shell
--url URL
export TMT_PLUGIN_REPORT_REPORTPORTAL_URL=URL