FAQ

What are the prerequisites for test execution ?

Install the Java Cryptography Extension (JCE) policy to your JDK  before running integration tests.

Increase the maximum number of open files/file descriptors in your OS. Follow the guide at http://stackoverflow.com/questions/34588/how-do-i-change-the-number-of-open-files-limit-in-linux

Shut down any other carbon instance on your machine which could cause port bind exceptions.

How can I run Selenium tests with a different version of Firefox?

You can set the -Dwebdriver.firefox.bin=/path/to/firefox/binary property in order to set the selenium to use a different version of Firefox in your machine.

E.g. mvn clean install -Dwebdriver.firefox.bin=/home/host/software/firefox/firefox-bin

Which Firefox version should I use for test execution?

Firefox versions ranging from 20.0 to 24.0 are the recommended versions for test execution.

What runs at the backend while running ESB integration tests?

The test framework starts a simple axis2 on port 9000 before running any ESB integration test cases and hosts a few services like SimpleStockQuoteService, Axis2Service etc..

In ESB tests, you can refer those service endpoints directly.

E.g.: http://localhost:9000/services/SimpleStockQuoteService

How can I start a new Axis2Server instance while running tests?

To start a new axis2 server, you need to create a new instance of SampleAxis2Server and deploy services. Start a simple axis2 server on http port 9001:

SampleAxis2Server  axis2Server = new SampleAxis2Server("test_axis2_server_9001.xml");
axis2Server.start();
axis2Server.deployService(MTOM_SERVICE);

Deploy a service on the axis2 server:

test_axis2_server_9001.xml must be present at src/test/resources/artifacts/AXIS2/config

The service should be at src/test/resources/artifacts/AXIS2/aar.

How can I capture a message that goes from ESB to the backend service?

To capture a message from ESB to the backend service, you can start the wire monitor server on a specific port. Then use that port and host as a service endpoint to capture the message that goes out from ESB.

WireMonitorServer wireServer = new WireMonitorServer(8991);

wireServer.start();

//send the message here

String response = wireServer.getCapturedMessage();

How can I run single tests or a test package?

To run a single test class, the testng.xml located at src/test/resource should be edited as given below. Note that all other test configurations should be removed from the testng.xml.

<test name="Server-Stratup" preserve-order="true" verbose="2">
       <classes>
           <class name="org.wso2.carbon.esb.ServerStartupTestCase"/>
       </classes>
</test>

To run a single package, testng.xml files should be edited as given below:

<test name="ProxyService-Test" preserve-order="true" verbose="2">
       <packages>
           <package name="org.wso2.carbon.esb.proxyservice.test.*"/>
       </packages>
</test>

How can I check emails in automation tests?

You can use the following gmail credentials to access a dummy gmail account.

username - test.automation.dummy@gmail.com

password - test.automation

To check whether a mail is received to the inbox, you can use the gmail atom feed to retrieve the feed.

Refer to the following reference test case.

Why does my test module contain intermittent test failures?

Intermittent test failures could happen due to many reasons. Most intermittent failures happen due to an indirect reason. The issue could be within the product itself, the test case or due to a heavy load on the servers. Some observations for intermittent test failures are as follows:

1. Most intermittent failures occurred when running thousands of integration tests on a single server instance - We fixed this in ESB and GREG by splitting tests into modules and starting carbon server for each test module.
2. Not following best practices correctly:
   
   

   • Not cleaning test artifacts/resources after each test class execution.

   • Introduction of duplicate test cases with same test artifacts.

   • Not following testNG annotations properly -  E.g. Usage of dependson and priority annotations together.

   • Not closing DB connections, file streams after test execution.

3. Issues within the test case itself.
4. Test design issues and lack of knowledge on TestNG and test framework.
5. Test Framework or TestNG issues.
6. High load on the server, and it stopped responding intermittently due to the load on the server instance.
7. Heavy resource  utilisation in the test execution machine.

How can I avoid test execution on a particular environment using custom annotations?

Refer to the documentation on TestNG custom annotations.

How can I discover the admin services of WSO2 products?

By default, the WSDLs of admin services are hidden from consumers. To enable it, set the <HideAdminServiceWSDLs> element to false in the <PRODUCT_HOME>/repository/conf/carbon.xml file. Restart the server and start the WSO2 product in the OSGI console. Once the server is started, hit the enter/return key several times to get the OSGI shell in the console. In the OSGI shell, type osgi> listAdminServices. The list of admin services of your product is shown. 

To see the service contract, first, select the admin service's URL and then paste it in your browser with ?wsdl at the end. For example, https://localhost:9443/services/UserAdmin?wsdl