Test Suite Management

Introduction

In test modules, the testng.xml file governs the test cases to be executed as a test suite. All the test packages and classes should be declared in the testng.xml and must also contain the listeners to provide the prerequisite for the test execution, such as server startup, user population etc. In the Surefire configuration of the test module pom file, you can mention which xml file should be executed. You can have any number of xml files like testng.xml with different names.

Surefire configuration

You can create a test suite by defining test classes or test packages in the testng.xml and use the <suiteXmlFile> element to specify this file. For example:

<build> 
     <plugins> 
       <plugin> 
         <artifactId>maven-surefire-plugin</artifactId> 
         <version>2.12.3</version> 
         <inherited>false</inherited> 
         <configuration> 
           <argLine>-Xms512m -Xmx1024m -XX:MaxPermSize=128m</argLine> 
           <testFailureIgnore>true</testFailureIgnore> 
           <disableXmlReport>false</disableXmlReport> 
           <parallel>false</parallel> 
           <suiteXmlFiles> 
             <suiteXmlFile>src/test/resources/testng1.xml</suiteXmlFile> 
             <suiteXmlFile>src/test/resources/testng2.xml</suiteXmlFile> 
           </suiteXmlFiles> 
         </configuration> 
       </plugin> 
     </plugins> 
   </build>

Sample TestNG XML file

Your test classes or package must be defined in the testng.xml file in order to run:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

 <suite name="as-server-setup-verifier">
    <parameter name="useDefaultListeners" value="false"/>
    <listeners>
        <listener class-name="org.wso2.carbon.automation.engine.testlisteners.TestExecutionListener"/>
        <listener class-name="org.wso2.carbon.automation.engine.testlisteners.TestManagerListener"/>
        <listener class-name="org.wso2.carbon.automation.engine.testlisteners.TestReportListener"/>
        <listener class-name="org.wso2.carbon.automation.engine.testlisteners.TestSuiteListener"/>
        <listener class-name="org.wso2.carbon.automation.engine.testlisteners.TestTransformerListener"/>
    </listeners>

     <test name="ms-integration-tests" preserve-order="true" parallel="false">
        <classes>
            <class name="org.wso2.carbon.as.integration.tests.AARServiceTestCase"/>
        </classes>
    </test>

 </suite>

As shown above, you can add any number of test blocks with the required test classes or packages. The test blocks are executed one by one.

You can defined test cases by packages or classes. If you define a package name, it will execute all the classes inside that package. If you define full qualified class names, it will only execute the classes you defined.  Also shown above are the default listeners provided by the test framework. It is recommended to keep the configuration as it is.

For more information see  testing.xml.