Data Integration

Data integration is an important part of an integration process. For example, consider the following scenario, where you have a typical integration process that is managed using the Integrator runtime of WSO2 EI. In this scenario, data stored in various, disparatedatasourcesare required in order to complete the integration use case. The data services functionality in WSO2 EI allows you to manage this integration scenario by decoupling the data from thedatasourcelayer and exposing them as data services. The service bus that manages the main integration flow will then have the capability of managing the data through the data service. Once the data service is defined, you can manipulate the data stored in the datasources by invoking the relevant operation defined in the data service. For example, you can perform the basic CRUD operations as well as other advanced operations.

See the following topics for more information:

Elements of a data service

See Elements of a Data Service for detailed information on what constitutes a data service.

Creating a data service

You can create a comprehensive data service in a few simple steps using the management console of the product. There are three ways of creating data services in WSO2 EI as explained below.

• Creating a data service from scratch

  The management console of WSO2 EI provides a convenient wizard that will guide you to create the service from scratch using a few simple steps. The following diagram depicts the steps you need to follow.
  
  Let's briefly look at each step of the Create New Data Service wizard:

  1. Define the data service: In the first step, you must specify the general details of the data service, such as the name and description of the data service. In addition, you can enable advanced options for your data service such as Namespaces, Batch Requests, Data Streaming, Transport settings and Distributed Transaction settings.

  2. Add Datasources: In the second step, you must specify the details of the datasources that should be exposed by the data service. You can add multiple datasources to the data service to achieve data federation. The parameters that you can configure for a datasource depends on the type of datasource. For example, if you use an RDBMS or Cassandra as the datasource, you can enable OData.

  3. Define Queries: In the third step, you must specify queries to process the data in a particular datasource. For example, create queries to search for data in the datasource, or insert/update/delete data. The query you define will consist of parameters that you will use to carry out the query function. For example, to insert data into the datasource, you must specify input mapping parameters and to view data from the datasource, you must specify output mapping parameters.

  4. Define Operations: In the fourth step, you can define an operation connected to a single query if you want to expose the data as a SOAP service. The operation will be used to invoke the query.

  5. Expose data as a REST resource: In the next step, you have the option of exposing the data from your data services as REST resources. To do this, you will be defining a resource connected to each query.

  See the following tutorials for step-by-step instructions on how to use the Create New Data Service wizard:

  • Creating data services from scratch.

  • Exposing data as a REST resource.

• Uploading an already created data service (.dbs file)

  If you already have a data service created, you can directly upload the .dbs file of the data service using the management console. See Uploading a Created Data Service for instructions.

• Generating a data service using a defined database structure

  If a database structure is already defined, the data service will be automatically generated for the datasource that you specify. See Generating a Data Service for instructions.

Data integration capabilities in WSO2 EI

Given below are the key features of data integration available in WSO2 EI.

• Exposing data as web services
• Exposing data as REST resources
• Processing requests in batches
• Invoking multiple operation simultaneously (request box)
• Exposing data through various transports
• Handling data federation
• Transacitonal data access and services (distirbuted transactions)
• Securing and managing data access
• Using namespaces for data services
• Monitoring data services.