TestNG Listeners

TestNG listeners can be defined as interfaces that allow the extending of TestNG behavior. Implementing TestNG listener interfaces provide a way to call event handlers inside custom listener classes, and makes it possible to do certain operations in the TestNG execution cycle.

WSO2 Test Automation Framework implements the following listener interfaces and respective class names:

IExecutionListener - org.wso2.carbon.automation.engine.testlisteners.TestExecutionListener
ITestListener - org.wso2.carbon.automation.engine.testlisteners.TestManagerListener
IReporter - org.wso2.carbon.automation.engine.testlisteners.TestReportListener
ISuiteListener - org.wso2.carbon.automation.engine.testlisteners.TestSuiteListener
IAnnotationTransformer - org.wso2.carbon.automation.engine.testlisteners.TestTransformerListener

For more information see TestNG listeners.

To let TestNG know that the test framework has implemented its own listener classes, it uses the <listeners> tag in the TestNG template file. To register your listener classes, create a TestNG Suite file and then add up the listeners section to this suite file. That way, all of the tests would essentially leverage the listener capabilities.

A sample testng.xml file with registered listeners:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

 <suite name="as-server-setup-verifier">
    <parameter name="useDefaultListeners" value="false"/>
    <listeners>
        <listener class-name="org.wso2.carbon.automation.engine.testlisteners.TestExecutionListener"/>
        <listener class-name="org.wso2.carbon.automation.engine.testlisteners.TestManagerListener"/>
        <listener class-name="org.wso2.carbon.automation.engine.testlisteners.TestReportListener"/>
        <listener class-name="org.wso2.carbon.automation.engine.testlisteners.TestSuiteListener"/>
        <listener class-name="org.wso2.carbon.automation.engine.testlisteners.TestTransformerListener"/>
    </listeners>

     <test name="appserver-integration-tests" preserve-order="true" parallel="false">
        <classes>
            <class name="org.wso2.carbon.as.integration.tests.AARServiceTestCase"/>
        </classes>
    </test>
</suite>

Test Automation Framework supports the execution of internal/external utility classes in various states of the TestNG listeners. Users can define the class paths in the automation.xml file under the desired TestNG listener states. For example: The class list stored under ExtensionContext -> TestExecutionListener -> onExecutionStart elements will be executed at the beginning of the test execution. Automation Framework uses Java reflection to execute classes. So users are expected to use the interface provided by the Test Automation Framework to develop external utility classes to be executed in the test. There are 5 interfaces provided by Test Automation Framework for different TestNG listeners. Those interfaces have specific methods defined explicitly to be inline with the corresponding TestNG listeners. The interfaces are:

ExecutionListenerExtension
ReportListenerExtension
SuiteListenerExtension
TestListenerExtension
TransformListenerExtension

A sample class implementing ExecutionListenerExtension is given below. This class will start a carbon server instance before running any test suite and shut it down after running all the test suites.

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.wso2.carbon.automation.engine.annotations.ExecutionEnvironment;
import org.wso2.carbon.automation.engine.context.AutomationContext;
import org.wso2.carbon.automation.engine.context.ContextXpathConstants;
import org.wso2.carbon.automation.engine.extensions.interfaces.ExecutionListenerExtension;

 import javax.xml.xpath.XPathExpressionException;

 public class CarbonServerExtension implements ExecutionListenerExtension {
    TestServerManager serverManager;
    private static final Log log = LogFactory.getLog(CarbonServerExtension.class);
    private String executionEnvironment;

     public void initiate() {
        try {
            AutomationContext context = new AutomationContext();
            serverManager = new TestServerManager(context);
            executionEnvironment =
                    context.getConfigurationValue(ContextXpathConstants.EXECUTION_ENVIRONMENT);
        } catch (XPathExpressionException e) {
            handleException("Error while initiating test environment", e);
        }
    }

     public void onExecutionStart() {
        try {
            if (executionEnvironment.equalsIgnoreCase(ExecutionEnvironment.STANDALONE.name())) {
                serverManager.startServer();
            }
        } catch (Exception e) {
            handleException("Fail to start carbon server ", e);
        }
    }

     public void onExecutionFinish() {
        try {
            if (executionEnvironment.equalsIgnoreCase(ExecutionEnvironment.STANDALONE.name())) {
                serverManager.stopServer();
            }
        } catch (Exception e) {
            handleException("Fail to stop carbon server ", e);
        }
    }

     private static void handleException(String msg, Exception e) {
        log.error(msg, e);
        throw new RuntimeException(msg, e);
    }
}

After implementing your own custom extension, the class must be registered in the automation.xml file under the desired listener state.

<listenerExtensions>
        <platformExecutionManager>
            <extentionClasses>
                <className>
                    org.wso2.carbon.automation.extensions.servers.carbonserver.CarbonServerExtension
                </className>
             </extentionClasses>
        </platformExecutionManager>     
    </listenerExtensions>