Features

Platform wide test execution support 

• The same test cases can be run seamlessly on a standalone product or product platform.
• The same test case can be executed as a super user or tenant user.
• Support for WSO2 product stack automation.
• Ability to execute tests on cross platforms.

Simple automation context API 

• Provides API to retrieve test automation configurations easily.

• Any configuration put into the automation.xml can be retrieved directly via Automation Context API, without any code changes.

TestNG listener based test execution

• Using testNG listeners to invoke operations defined in pluggable executors.

• Allow users to call custom operations in different stages of the test execution cycle.

REST backend support with JAX-RS services 

• Can be used when tests require RESTFul backend.

• Tomcat server inside Automation Framework extensions and the facility to deploy any JAX-RS based service through a client to the embedded Tomcat server. 

Embedded server support (FTP, Proxy server, JMS, ActiveMQ, HTTP Servers Axis2 Server) 

• Provides common set of third party servers to facilitate common integration scenarios.

• Resided in the test framework extensions module.

Annotation based test execution management support

• Test will be executed in all environments by default.

• TestNg custom annotations are introduced to exclude or include tests to run on a specific environment only.

• Test developers can choose to run tests on STANDALONE,  PLATFORM or ALL environments using custom annotations.

Third party test tooling integration support (Jmeter) 

• Test Framework provides Jmeter test script runner.

• Jmeter scripts can be easily executed by pointing to the script location. Framework will validate the result of the jmeter test execution and return a pass or fail status.

Selenium webdriver integration and ability to run UI tests to cross browser environments. 

• Supports writing test cases using selenium webdriver API.

• Uses page object based repository model to avoid frequent test changes, which might introduced by UI changes.

• Support for multiple browser types and can be configurable through the automation.xml

Set of common test utilities to facilitate better test writing experience 

• Automation Utility module is introduced to provide common set of utility methods for better test writing experience.

• Improve reusability of utility methods while writing platform tests.

Lightweight automation engine 

• Automation engine itself uses only a few dependencies. All other custom functionalities which need to be introduced at different TestNG phases can be done using pluggable execution classes.

Ability to integrate third party products and platforms.

• Third party products and platforms can be integrated to the framework easily though pluggable modules.
• Any external configuration can be introduced to automation.xml and retrieve it without any framework code changes.

Native xpath based XML parsing implementation

• Uses Java native libraries to process the automaton.xml and build configuration context.

• Enables xpath based data retrieval through automation context API.

Single configuration file to manage all configurations

• All framework related configurations are managed via the automation.xml. 

Framework extensibility capability through pluggable modules

• With this feature the Test Automation Framework users can perform custom tasks related to tests in preferred test execution phases.  

• Maintainability of external/pluggable modules outside of the Test Automation Framework core.

Enhanced user management facility

• A set of super tenant and tenant users to be used in the test runtime can be defined in the automation.xml file.

• Framework will automatically populate all tenant and users.

• Test can use both super tenant and tenant mode to execute tests.

Enhanced server management facility

• Starting and shutting down carbon servers is much easier.

• Multiple carbon server instances can be started by setting different port offsets via test framework.

Configuration file validator

• Validates automation.xml file against defined XSD.

Enhanced code coverage through EMMA

• Emma based code coverage generation facilitates the creation of coverage reports in XML and HTML formats.

• Coverage report generation based on Method, Class, LOC and Block coverage.

• Configuration of clustered setup can be easily represented in the automation configuration file.

• Configuration options to support all clustering patterns.