// Copyright (C) 2024 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only


/*!
    \page creator-how-to-create-qt-tests.html
    \previouspage creator-how-tos.html

    \ingroup creator-how-to-test

    \title Create Qt tests

    \QC integrates the \l{Qt Test} framework for unit testing Qt applications and
    libraries.

    \note Running Qt tests and displaying their results is supported in Qt 5 or later.

    To create a Qt test:

    \list 1
        \li Go to \uicontrol File > \uicontrol {New Project} >
            \uicontrol {Test Project}.
        \li Select \uicontrol {Qt Test Project} > \uicontrol Choose.
        \li In the \uicontrol {Project and Test Information} dialog, specify
            settings for the project and test.
            \image {qtcreator-new-project-qt-test.webp} {Project and Test Information dialog in the Qt Test Project wizard}
        \li In \uicontrol {Test case name}, enter a name for the test case.
        \li Select \uicontrol {Requires \QApplication} to add the include
            statement for QApplication to the \c main.cpp file of the project.
        \li Select \uicontrol {GUI Application} to create a Qt application.
        \li Select \uicontrol {Generate initialization and cleanup code}
            to add functions to your test that the testing framework
            executes to initialize and clean up the test.
        \li In \uicontrol {Build system}, select the build system to use for building
            the project: CMake, qmake, or Qbs. To build with CMake when developing with
            Qt 5 or Qt 6.4 or earlier, select \uicontrol {CMake with Qt 5 Compatibility}.
    \endlist

    \QC creates the test in the specified project directory. Edit the \c .cpp file
    to add private slots for each test function in your test.

    For more information about creating Qt tests, see \l{Creating a Test}.

    \note While scanning for tests, the parser takes into account only files that
    directly or indirectly include QTest or QtTest or its equivalent \c {QtTest/qtest.h}
    and get linked against \c QTestLib when building the project.
    Files that currently will not be built will be ignored.

    \note If \uicontrol General > \uicontrol {Process arguments} is enabled, you can
    execute tests and write one or more additional log files in parallel.
    Use the output format option \c {-o filename,format} to do so. Using standard out for
    \c filename is not allowed as \QC is using this output channel.

    \sa {Test}{How To: Test}, {Select the build system}, {Testing},
    {Qt Test Overview#Logging Options}{Logging Options}
*/

/*!
    \page creator-how-to-create-qtquick-tests.html
    \previouspage creator-how-tos.html

    \ingroup creator-how-to-test

    \title Create Qt Quick tests

    \QC integrates the \l{Qt Quick Test} unit test framework for Qt Quick
    applications, where you write test cases as JavaScript functions.

    To create a Qt Quick test:

    \list 1
        \li Go to \uicontrol File > \uicontrol {New Project} >
            \uicontrol {Test Project}.
        \li Select \uicontrol {Qt Quick Test Project} > \uicontrol Choose.
        \li In the \uicontrol {Project and Test Information} dialog, specify
            settings for the project and test.
            \image {qtcreator-new-project-qtquick-test.webp} {Project and Test Information dialog in the Qt Quick Test Project wizard}
        \li In \uicontrol {Test case name}, enter a name for the test case.
        \li Select \uicontrol {Generate setup code} to execute C++
            code before running any of the QML tests. The testing
            framework will call slots and invocable functions, as
            described in \l{Executing C++ Before QML Tests}.
        \li Select \uicontrol {Generate initialization and cleanup code}
            to add functions to your test that the testing framework
            executes to initialize and clean up the test.
        \li In \uicontrol {Qt code compatibility}, select
            \uicontrol {Qt6 and higher} or \uicontrol {Qt5 and higher} to set
            the Qt version compatibility for the test.
        \li In \uicontrol {Build system}, select the build system to use for building
            the project: CMake, qmake, or Qbs. To build with CMake when developing with
            Qt 5 or Qt 6.4 or earlier, select \uicontrol {CMake with Qt 5 Compatibility}.
    \endlist

    \QC creates the test in the specified project directory. Edit the \c .cpp file
    to add private slots for each test function in your test.

    \note While scanning for tests, the parser takes into account only files that
    directly or indirectly include QuickTest or its equivalent \c {QtQuickTest/quicktest.h}.
    If one gets found, it tries to detect statements to create a Qt Quick test application
    and if it can find any it will parse respective QML files. If the C++ part will not
    be built with the current configuration, there will be no automatic scanning of QML
    files.

    \note If \uicontrol General > \uicontrol {Process arguments} is enabled, you can
    execute tests and write one or more additional log files in parallel.
    Use the output format option \c {-o filename,format} to do so. Using standard out for
    \c filename is not supported as \QC is using this output channel.

    \sa {Test}{How To: Test}, {Select the build system}, {Testing}
*/

/*!
    \page creator-how-to-create-google-tests.html
    \previouspage creator-how-tos.html

    \ingroup creator-how-to-test

    \title Create Google tests

    To create a Google test:

    \list 1
        \li Go to \uicontrol File > \uicontrol {New Project} >
            \uicontrol {Test Project}.

        \li Select \uicontrol {Google Test Project} > \uicontrol Choose to create
            a project with boilerplate code for a Google test.

        \li In the \uicontrol {Project and Test Information} dialog, specify
            settings for the project and test:

            \image {qtcreator-new-project-google-test.webp} {Project and Test Information dialog in the Google Test Project wizard}

            \list 1

                \li In \uicontrol {Test framework}, select
                    \uicontrol {Google Test (shared libraries)} to link against
                    Google Test or \uicontrol {Google Test (headers only)} to
                    include necessary Google Test sources into the project.

                \li In \uicontrol {Test suite name}, enter a name for
                    the test suite.

                \li In \uicontrol {Test case name}, enter a name for
                    the test case.

                \li For a shared library test, you can set the path to a
                    Google C++ testing framework installation in
                    \uicontrol {Googletest install directory (optional)}.

                \li For a header-only test, you can set the path to
                    a googletest repository clone in
                    \uicontrol {Googletest source directory (optional)}.

                \li In \uicontrol {Build system}, select the build system to use
                    for building the project: CMake, qmake, or Qbs.

            \endlist

    \endlist

    \QC creates the test in the specified project directory.

    For more information about creating Google tests, see the
    \l{https://github.com/google/googletest/blob/master/docs/primer.md}
    {Google Test Primer}.

    \note While scanning for tests, the parser takes into account only files that
    directly or indirectly include \c {gtest/gtest.h}. Files that currently will
    not be built will be ignored.

    \sa {Test}{How To: Test}, {Select the build system}, {Testing}
*/

/*!
    \page creator-how-to-create-boost-tests.html
    \previouspage creator-how-tos.html

    \ingroup creator-how-to-test

    \title Create Boost tests

    To build and run Boost tests, install Boost.Test on the computer.
    Typically, the Boost installation includes it. Download Boost from
    \l{https://www.boost.org/}{Boost.org}.

    If the compiler and build system can find the Boost libraries, you
    do not need to specify the include directory when creating the test.

    To create a Boost test:

    \list 1
        \li Go to \uicontrol File > \uicontrol {New Project} >
            \uicontrol {Test Project}.
        \li Select \uicontrol {Boost Test Project} >
            \uicontrol Choose to create a project with boilerplate code for a
            Boost test.
        \li In the \uicontrol {Project and Test Information} dialog, specify
            settings for the project and test:
            \image {qtcreator-new-project-boost-test.webp} {Project and Test Information dialog in the Boost Test Project wizard}
            \list 1
                \li In \uicontrol {Test framework}, select
                    \uicontrol {Boost Test (header only)} to include needed
                    headers, or \uicontrol {Boost Test (shared libraries)}
                    to link against Boost Test.
                \li In \uicontrol {Test suite name}, enter a name for
                    the test suite.
                \li In \uicontrol {Test case name}, enter a name for
                    the test case.
                \li In \uicontrol {Boost include directory (optional)},
                    enter the path to the directory that has files needed
                    by Boost.Test, such as \c version.hpp and the \c test
                    subfolder that contains the test header files.
                \li In \uicontrol {Build system}, select the build
                    system to use for building the project: CMake, qmake, or Qbs.
            \endlist
    \endlist

    \QC creates the test in the project directory.

    For more information about creating Boost tests, see
    \l{https://www.boost.org/doc/libs/1_81_0/libs/test/doc/html/index.html}
    {Boost.Test}.

    \note While scanning for tests, the parser takes into account only files that
    directly or indirectly include one of Boost's test library headers (any include
    starting with \c {boost/test/} and having the \c {.hpp} suffix).
    Files that currently will not be built will be ignored.

    \sa {Test}{How To: Test}, {Select the build system}, {Testing}
*/

/*!
    \page creator-how-to-create-catch2-tests.html
    \previouspage creator-how-tos.html

    \ingroup creator-how-to-test

    \title Create Catch2 tests

    To build and run Catch2 tests, you can either install Catch2
    libraries and headers or use the single include header file in the
    Catch2 repository.

    If the compiler and build system can find the Catch2 headers automatically,
    you do not need to specify the include directory when creating the test.

    To create a basic Catch2 test:

    \list 1
        \li Go to \uicontrol File > \uicontrol {New Project} >
            \uicontrol {Test Project}.
        \li Select \uicontrol {Catch2 Test Project} > \uicontrol Choose to create
            a project with boilerplate code for a Catch2 test.
        \li In the \uicontrol {Project and Test Information} dialog, specify
            settings for the project and test:
            \image {qtcreator-new-project-catch2-test.webp} {Project and Test Information dialog in the Catch2 Test Project wizard}
            \list 1
                \li In \uicontrol {Test framework}, select
                    \uicontrol {Catch2 v3 (shared libraries)} to link against
                    Catch2 (recommended for version 3), or
                    \uicontrol {Catch2 v2 (header only)} to use a single header
                    include (recommended for version 2).
                \li In \uicontrol {Test case name}, specify a name
                    for the test case file.
                \li In \uicontrol {Catch2 install directory (optional)},
                    you can enter a path to the directory that has the
                    Catch2 header files.
                \li Select \uicontrol {Use own main} to use a self-defined
                    main function.
                \li Select \uicontrol {Use Qt libraries} set up the project to
                    use Qt features.
                \li In \uicontrol {Build system}, select the build
                    system to use for building the project: CMake, qmake, or Qbs.
            \endlist
    \endlist

    \QC creates the test in the project directory.

    For more information about creating Catch2 tests, see
    \l{https://github.com/catchorg/Catch2/blob/master/docs/Readme.md}
    {Catch2}.

    \note While scanning for tests, the parser takes into account only files that
    directly or indirectly include one of the common Catch2 headers
    (\c {catch2.hpp}, \c {catch_all.hpp}, and some more).
    Files that currently will not be built will be ignored.

    \sa {Test}{How To: Test}, {Select the build system}, {Testing}
*/

/*!
    \page creator-how-to-create-ctests.html
    \previouspage creator-how-tos.html

    \ingroup creator-how-to-test

    \title Create CTest Based Tests

    CTest can execute tests for CMake based projects regardless of the test
    framework. You configure tests in the project file, usually, \c CMakeLists.txt.
    Basically, you enable testing for the project and register the test
    applications or even special commands.

    \code
    enable_testing()
    add_test(NAME test_example COMMAND test_example)
    \endcode

    Add \c test_example as an executable before trying to register it as test.
    It may be any executable command including arguments.

    For more information about how to use CTest, see
    \l{https://gitlab.kitware.com/cmake/community/-/wikis/doc/ctest/Testing-With-CTest}
    {Testing with CTest}.

    \sa {Test}{How to: Test}, {Select the build system}, {Testing}
*/

/*!
    \page creator-how-to-add-tests-to-projects.html
    \previouspage creator-how-tos.html

    \ingroup creator-how-to-test

    \title Add tests to existing test projects

    To add a new file that has test cases to a project that contains a test
    project:

    \list 1
        \li Go to \uicontrol File > \uicontrol {New File}.
        \li Select \uicontrol {Test Case} > \uicontrol Choose.
        \li In \uicontrol {Test framework}, select the test framework in use.
            This has to match the test framework you are using inside the project
            you want to add this file to.
            \image {qtcreator-new-file-test-case.webp} {Test Information dialog in the Test Case wizard}
    \endlist

    The settings to specify depend on the selected test framework.

    \note This is only supported for code based test frameworks, except for
    Qt Tests. There is no official support for having multiple test cases
    inside a single Qt test application, and it is better to create
    a separate test sub project for each test case.

    \sa {Test}{How to: Test}, {Testing}
*/

/*!
    \page creator-how-to-build-and-run-tests.html
    \previouspage creator-how-tos.html

    \ingroup creator-how-to-test

    \title Build and run tests

    To build and run tests:

    \list 1

        \li Open a project that has tests.

        \li In the \uicontrol Tests view, select the tests to run.

        \li In the \uicontrol {Test Results} view, select:

            \list
                \li \imagerunsmall (\uicontrol {Run All Tests}) to run all tests.
                 \li \inlineimage {icons/qtcreator-run-selected-tests.png} {Run Selected Tests}
                    (\uicontrol {Run Selected Tests}) to run the selected tests.
                 \li \inlineimage {icons/qtcreator-run-failed-tests.png} {Run Failed Tests}
                    (\uicontrol {Run Failed Tests}) to re-run the tests which failed
                    in the last run.
                    Depending on the framework, this may select more tests if it
                    is impossible to distinguish or to fully address the test.
                 \li \inlineimage {icons/qtcreator-run-tests-in-current-file.png} {Run Tests for Current File}
                    (\uicontrol {Run Tests for Current File}) to run the tests
                    in the file currently open in the code editor.
            \endlist

    \endlist

    By default, \QC builds a project before deploying and running it.
    To run all tests without building and deploying them again, select
    \uicontrol {Run All Tests Without Deployment} in the context menu.
    To run the selected tests without deployment, select
    \uicontrol {Run Selected Tests Without Deployment}.

    You can run tests also from the context menu in the \uicontrol Tests view or
    by going to \uicontrol Tools > \uicontrol Tests.

            \note If you enable both build system based and code based tests,
            you may run tests twice when selecting \uicontrol {Run All Tests} or
            \uicontrol {Run Selected Tests}. This happens if the tests can be
            found by the code based test frameworks and are registered as test
            with the build system.

    \section1 Select tests to run

    The \uicontrol Tests view shows all the tests found for the currently active
    test frameworks in the current project. Select the test cases to run.

    \image {qtcreator-tests-view.png} {Tests view}

    If a Qt Quick test case does not have a name, it is marked
    \uicontrol Unnamed in the list. \uicontrol {Run All Tests} executes
    unnamed test cases. You cannot select or clear them.

    \QC scans the project for tests when you open the project and updates the
    test list for the currently active test frameworks when you edit tests.
    To refresh the view, select \uicontrol {Rescan Tests} in the context menu.

    To show or hide init and cleanup or data functions in the \uicontrol Tests
    view, select \inlineimage {icons/filtericon.png} {Filter Test Tree}
    (\uicontrol {Filter Test Tree}), and then select \uicontrol {Show Init and
    Cleanup Functions} or \uicontrol {Show Data Functions}. Double-click a
    function in the list to open its source code in the code editor.

    The \uicontrol Tests view lists test cases in alphabetic, case insensitive
    order. To list them in the order in which they appear in the source code,
    select \inlineimage {icons/leafsort.png} {Sort Naturally}
    (\uicontrol {Sort Naturally}).

    \section1 Run and debug tests from code editor

    You can run and debug tests in the file currently open in the code editor.
    To run all tests in the open file, go to \uicontrol Tools >
    \uicontrol Tests > \uicontrol {Run Tests for Current File}.
    \note Available only for code based test frameworks.

    To run the test currently selected in the open file, select
    \uicontrol {Run Test Under Cursor} in the context menu.
    \note Available only for code based test frameworks.

    To debug the currently selected test, select
    \uicontrol {Debug Test Under Cursor} in the context menu.
    \note Availability for build system based tests depends on the build system
    tool's abilities.

    \section1 Run CTest tests from the locator

    You can use the \c ct locator filter to run CTest tests.

    \section1 Turn on build system based tests

    By default, \QC does not handle build system based tests to avoid
    interference with code based parsers. To turn on build system based tests,
    select the respective test tool in \preferences > \uicontrol Testing
    > \uicontrol General.

    The detection of tests is usually much faster for build system based
    tests as this normally does not involve scanning or parsing.
    The information in the tests tree is usually more detailed
    when using code based tests.

    If you turn on both code based and build system based tests, you
    may duplicate tests inside the tests tree.

    \section1 Locate Qt Test data tags

    To locate Qt Test data tags:

    \list 1
        \li Select \key {Ctrl+K} (\key {Cmd+K} on \macos) to activate the locator.
        \li Enter \c qdt, followed by a space.
    \endlist

    \sa {Test}{How to: Test}, {Select the build system}, {Testing}
*/

/*!
    \page creator-preferences-testing.html
    \previouspage creator-reference.html
    \nextpage creator-vcs-git.html

    \title Testing

    \quotation
    \l {Test}{How to: Test}
    \endquotation

    \summary {Customize the handling of tests, test frameworks, and test tools.}

    To set preferences for handling tests, go to
    \preferences > \uicontrol {Testing} > \uicontrol General.

    \image {qtcreator-preferences-testing-general.webp} {General tab in Testing preferences}

    The following table summarizes the general testing preferences.

    \table
    \header
        \li Setting
        \li Value
    \row
        \li \uicontrol {Scan threads}
        \li Set the amount of worker threads used when scanning for tests.
            By default, \QC uses a quarter of the available logical CPUs.
    \row
        \li \uicontrol {Omit internal messages}
        \li Omit internal messages.
    \row
        \li \uicontrol {Omit run configuration warnings}
        \li Omit run configuration warnings for deduced configurations.
    \row
        \li \uicontrol {Limit result output}
        \li By default, test result output shows a maximum of 100,000 characters.
    \row
        \li \uicontrol {Limit result description}
        \li Set the maximum number of lines in the test result tooltip and
            description.
    \row
        \li \uicontrol {Open results when tests start}
        \li Show test results automatically when testing starts.
    \row
        \li \uicontrol {Open results when tests finish}
        \li Show test results automatically when testing ends.
    \row
        \li \uicontrol {Open results only for unsuccesfull test runs}
        \li Show test results only if they contain failed, fatal, or
            unexpectedly passed tests.
    \row
        \li \uicontrol {Automatically scroll results}
        \li The view automatically scrolls to show the latest results.
    \row
        \li \uicontrol {Group results by application}
        \li Group test results by the executable path that you use to run the
            tests. This is useful if you have multiple test executables and run
            them all at once.
    \row
        \li \uicontrol {Process arguments}
        \li Pass arguments to the test executable that you specify in the
            \l {Configure projects for running}{run configuration}. This is an
            experimental feature that might cause the execution of the test
            executable to fail. Options known to interfere with the used
            test framework are filtered out.
    \row
        \li \uicontrol {Automatically run}
        \li Automatically run all or selected tests after successfully building
            the current project.
    \row
        \li \uicontrol {Timeout}
        \li Set a maximum time in seconds to execute a test case.
     \row
        \li \uicontrol {Reset Cached Choices}
        \li Sometimes, \QC cannot deduce which executable or run configuration to
            use. If it repeatedly asks you to select the tests to run when trying
            to execute tests, you can let it cache your choices and use them where
            appropriate. \QC clears the cache when you switch to another project,
            close the current one, or select this setting.
    \row
        \li \uicontrol {Active Test Frameworks}
        \li Select tests for \QC to handle. To improve the performance of full
            scans for tests, disable test frameworks you do not use.
    \row
        \li \uicontrol {Group}
        \li Group related test cases for an active test framework.
            By default, \QC groups tests that are in the same directory.
    \endtable

    To override some preferences for the current project, go to
    \uicontrol Projects > \uicontrol {Project Settings} > \uicontrol {Testing}.

    \section1 Qt Tests

    To set preference for Qt tests, go to \preferences > \uicontrol {Testing} >
    \uicontrol {Qt Test}.

    \image {qtcreator-preferences-testing-qttest.webp} {Qt Test tab in Testing preferences}

    The code inside a benchmark test is measured, and possibly also repeated
    several times in order to get an accurate measurement. This depends on the
    measurement back-end that you select in \uicontrol {Benchmark Metrics}:

    \list
        \li Walltime
        \li CPU tick counter
        \li Event counter
        \li Valgrind Callgrind
        \li Linux Perf
    \endlist

    For more information, see \l{Creating a Benchmark}.

    The following table summarizes the other Qt test preferences.

    \table
    \header
        \li Setting
        \li Value
    \row
        \li \uicontrol {Disable crash handler while debugging}
        \li Allow the debugger to interrupt Qt tests on assertions.
    \row
        \li \uicontrol {Use XML output}
        \li XML output avoids parsing issues, but plain text is more human
            readable.
            \note Plain text misses some information, such as duration.
    \row
        \li \uicontrol {Verbose benchmarks}
        \li Receive verbose output when running benchmarks.
    \row
        \li \uicontrol {Log signals and slots}
        \li Record information about signals and slots in the test log.
    \row
        \li \uicontrol {Limit warnings}
        \li Explicitly limit the maximum number of warnings in the test log.
            Set the limit to 0 if you want no limit at all. The default limit
            is 2000.
    \row
        \li \uicontrol {Check for derived Qt Quick tests}
        \li Check for Qt Quick Tests that are derived from TestCase.
            \note This feature is rather expensive and increases the scan time
            significantly.
    \row
        \li \uicontrol {Find user-defined locations}
        \li Parse messages for the following pattern and use it as location
            information: \c {file://filepath:line}

            where \":line\" is optional.

            \note If the patterns are used in code, the location information
            for debug messages and other messages might improve, at the risk
            of some incorrect locations and lower performance.
    \endtable

    \section1 Google Tests

    To specify settings for running Google tests, go to \preferences >
    \uicontrol {Testing} > \uicontrol {Google Test}.

    \image {qtcreator-preferences-testing-googletest.webp} {Google Test tab in Testing preferences}

    The following table summarizes the Google test preferences.

    \table
    \header
        \li Setting
        \li Value
    \row
        \li \uicontrol {Run disabled tests}
        \li Execute disabled tests during a test run.
    \row
        \li \uicontrol {Throw on failure}
        \li Turn assertion failures into C++ exceptions.
    \row
        \li \uicontrol {Break on failure while debugging}
        \li Turn failures into debugger breakpoints.
    \row
        \li \uicontrol {Repeat tests}
        \li Run several iterations of the tests.
    \row
        \li \uicontrol Iterations
        \li The number of times to run the tests.
    \row
        \li \uicontrol {Shuffle tests}
        \li Make sure that the tests are independent and repeatable by running
            them in a different order each time.
    \row
        \li \uicontrol Seed
        \li The seed for initializing the randomizer. The value 0 generates
            a seed based on the current timestamp.
    \row
        \li \uicontrol {Group mode}
        \li Group Google tests by using a \uicontrol {GTest Filter}.
    \row
        \li \uicontrol {Active filter}
        \li The GTest filter to use.
    \endtable

    For more information about GTest filters, see
    \l{https://github.com/google/googletest/blob/master/docs/advanced.md#running-a-subset-of-the-tests}
    {Running a Subset of the Tests}.

    \section1 Boost Tests

    To specify settings for running Boost tests, go to \preferences >
    \uicontrol {Testing} > \uicontrol {Boost Test}.

    \image {qtcreator-preferences-testing-boosttest.webp} {Boost Test tab in Testing preferences}

    The following table summarizes the Boost test preferences.

    \table
    \header
        \li Setting
        \li Value
    \row
        \li \uicontrol {Log format}
            \li The type of events to record in the test report.
    \row
        \li \uicontrol {Report level}
        \li The verbosity level of the test result report. Select \uicontrol No
            if you do not want a report.
    \row
        \li \uicontrol Randomize
        \li Execute the tests in a random order.
    \row
        \li \uicontrol Seed
        \li The seed for initializing the randomizer. The value 0 means no
            randomization, the value 1 uses the current time, and any other
            value generates a random seed.
    \row
        \li \uicontrol {Catch system errors}
        \li Catch or ignore system errors.
    \row
        \li \uicontrol {Floating point exceptions}
        \li Detect floating point exception traps.
    \row
        \li \uicontrol {Detect memory leaks}
        \li Detect memory leaks.
    \endtable

    \section1 Catch2 Tests

    To specify settings for running Catch2 tests, go to \preferences >
    \uicontrol {Testing} > \uicontrol {Catch Test}.

    \image {qtcreator-preferences-testing-catchtest.webp} {Catch Test tab in Testing preferences}

    The following table summarizes the Catch2 test preferences.

    \table
    \header
        \li Setting
        \li Value
    \row
        \li \uicontrol {Show success}
        \li Show succeeding expressions as well. By default Catch2 prints only
            failed expressions.
    \row
        \li \uicontrol {Break on failure while debugging}
        \li Turn failures into debugger breakpoints.
    \row
        \li \uicontrol {Skip throwing assertions}
        \li Skip assertions that test for thrown exceptions.
    \row
        \li \uicontrol {Visualize whitespace}
        \li Turn whitespace into escape sequences.
    \row
        \li \uicontrol {Abort after}
        \li Abort the test after the number of failures specified inside the box.
    \row
        \li \uicontrol {Benchmark samples}
        \li The number of samples to collect while running benchmarks.
    \row
        \li \uicontrol {Benchmark resamples}
        \li The number of resamples to use for statistical bootstrapping after
            benchmarking.
    \row
        \li \uicontrol {Benchmark confidence interval}
        \li The confidence interval for statistical bootstrapping.
    \row
        \li \uicontrol {Benchmark warmup time}
        \li The warmup time for each test before benchmarking starts.
    \row
        \li \uicontrol {Disable analysis}
        \li Disable statistical analysis and bootstrapping.
    \endtable

    \section1 CTest-Based Tests

    To specify settings for running CTest-based tests, go to \preferences >
    \uicontrol {Testing} > \uicontrol {CTest}.

    \image {qtcreator-preferences-testing-ctest.webp} {CTest tab in Testing preferences}

    The following table summarizes the CTest preferences.

    \table
    \header
        \li Setting
        \li Value
    \row
        \li \uicontrol {Output on failure}
        \li Show test specific output if a test fails. Contrary to the
            CTest default this setting is on by default.
    \row
        \li \uicontrol {Schedule random}
        \li Execute the tests in random order.
    \row
        \li \uicontrol {Stop on failure}
        \li Automatically stop the test execution on the first failing test.
    \row
        \li \uicontrol {Output mode},
        \li The verbosity level of the CTest output.
            \note This setting only influences the output on the text display.
    \row
        \li \uicontrol {Repeat tests}
        \li Re-run tests, as determined by \uicontrol {Repetition mode}. Set the
            maximum number of times for repeating a test in \uicontrol {Count}.
    \row
        \li \uicontrol {Run in Parallel}
        \li Run the tests in parallel using the specified number of
            \uicontrol {Jobs}. In \uicontrol {Test load}, limit the parallel
            execution. CTest will not start a new test if it would cause the
            CPU load to pass the threshold set in \uicontrol {Threshold}.
    \endtable

    \sa {Test}{How to: Test}
*/

/*!
    \page creator-test-results.html
    \previouspage creator-how-tos.html

    \ingroup creator-how-to-test

    \title View test results

    View and filter test results in the \uicontrol{Test Results} view.

    \image {qtcreator-test-results.webp} {Visual display in the Test Results view}

    The view shows test duration for the test frameworks that support it. To
    show and hide duration information, select
    \inlineimage {icons/stopwatch.png} {Duration}.

    \section1 View Qt test output

    The \uicontrol{Test Results} view shows Qt and Qt Quick test results
    in XML format and other test results in plain text format.

    XML can be parsed more easily and reliably than plain text. However, if a Qt
    or Qt Quick test crashes, it might not produce complete XML
    code that can be parsed, which might lead to information loss. You might see
    the lost information when viewing the results as plain text.

    To view the results of Qt and Qt Quick tests as plain text:

    \list 1
        \li Go to \preferences > \uicontrol {Testing} > \uicontrol {Qt Test}.
        \li Clear \uicontrol {Use XML output}.
        \li Go to \uicontrol {Test Results}.
        \li Select
            \inlineimage {icons/text.png} {Switch Between Visual and Text Display}
            (\uicontrol {Switch Between Visual and Text Display}) to switch to
            the text display.
    \endlist

    \section1 Summary of messages

    The following table lists the messages that \uicontrol {Test Results} shows.

    \table
        \header
            \li Result
            \li Description
        \row
            \li BENCH
            \li Benchmark test.
        \row
            \li DEBUG
            \li Debug message.
        \row
            \li FAIL
            \li Test case failed. Double-click the line for more information.
        \row
            \li FATAL
            \li A fatal error occurred that stops the test case from being run,
                for example.
        \row
            \li INFO
            \li Informative message.
        \row
            \li INTERNAL
            \li Internal message.
        \row
            \li PASS
            \li Test case passed.
        \row
            \li SKIP
            \li Test case was skipped.
        \row
            \li SYSTEM
            \li An error message received from or influenced by the OS.
        \row
            \li XFAIL
            \li Test case is expected to fail, so it is marked by using the
                QEXPECT_FAIL macro. If the test case passes instead, an
                unexpected pass (XPASS) is written to the test log.
        \row
            \li XPASS
            \li Test case passed even though it was expected to fail.
        \row
            \li WARN
            \li Warning message.

    \endtable

    \section1 Filter test results

    To view only messages of a particular type:

    \list 1
        \li Go to \uicontrol {Test Results}.
        \li Select \inlineimage {icons/filtericon.png} {Filter Test Results}
            (\uicontrol {Filter Test Results}).
        \li Select the types of messages to show.
    \endlist

    To show all messages, select \uicontrol {Check All Filters}.

    To clear all message types, select \uicontrol {Uncheck All Filters}.

    \section1 Blacklist tests

    A \c BLACKLIST file for tests is mainly used internally by the Qt CI system.

    \table
        \header
            \li Result
            \li Description
        \row
            \li \c {BFAIL}
            \li Blacklisted test case failed.
        \row
            \li \c {BPASS}
            \li Blacklisted test case passed.
        \row
            \li \c {BXFAIL}
            \li Blacklisted test case failed but was marked to be expected to fail.
        \row
            \li \c {BXPASS}
            \li Blacklisted test case passed even though it was expected to fail.

    \endtable

    \sa {Test}{How to: Test}, {Testing}
*/
