Exposing a Custom Datasource as a Data Service

See the topics given below to expose a custom datasource as a data service. Also, see the samples in Data Integration Samples.

About custom datasources

Custom datasources allow you to interface data services with your own datasource implementation. There are two options for writing a custom datasource, and these two options cover most of the common business use cases as follows:

Custom tabular datasources: Used to represent data in tables, where a set of named tables contain data rows that can be queried later. A tabular datasource is typically associated with an SQL data services query. This is done by internally using our own SQL parser to execute SQL against the custom datasource. You can use the org.wso2.carbon.dataservices.core.custom.datasource.TabularDataBasedDS interface to implement tabular datasources. For a sample implementation of a tabular custom datasource, see org.wso2.carbon.dataservices.core.custom.datasource.InMemoryDataSource. Also, this is supported in Carbon datasources with the following datasource reader implementation: org.wso2.carbon.dataservices.core.custom.datasource.CustomTabularDataSourceReader.
Custom query datasources: Used when the datasource has some form of query expression support. Custom query datasources are implemented using the org.wso2.carbon.dataservices.core.custom.datasource.CustomQueryBasedDS interface. You can create any non-tabular datasource using the query-based approach. Even if the target datasource does not have a query expression format, you can create your own. For example, you can support any NoSQL type datasource this way. For a sample implementation of a query-based custom datasource, see org.wso2.carbon.dataservices.core.custom.datasource.EchoDataSource. This is supported in Carbon datasources with the following datasource reader implementation: org.wso2.carbon.dataservices.core.custom.datasource.CustomQueryDataSourceReader.

In the init methods of all custom datasources, user-supplied properties are parsed to initialize the datasource accordingly. Also, a property named __DATASOURCE_ID__, which contains a UUID to uniquely identify the current datasource, is passed.Custom datasource authors use this to identify the datasources accordingly. For example, scenarios like datasource instances communicating within a server cluster for data synchronisation.

Find the custom connectors used from the github project located at https://github.com/wso2/wso2-dss-connectors.

Adding a custom tabular datasource to the data service

Follow the steps given below to create a custom tabular datasource.

Log in to the management console using the following URL on your browser: https://localhost:9443/carbon/.
Click Create under Data Service to open the Create Data Service window.
Enter CustomTabularDataService as the data service name.
When you get to the Add New Datasource screen, enter CustomTabular as the datasource ID.
Select Custom Data Source as the datasource type.
Select Custom Tabular Data Source.
In the Custom Data Source Class field, enter the following interface: org.wso2.carbon.dataservices.core.custom.datasource.InMemoryDataSource.
Click Add New Property to enter properties that will identify the data in the datasource.
Property Value
inmemory_datasource_schema
{Users:[ID,Name]}
inmemory_datasource_records
{Users:[["1","Will Smith"],["2","Denzel Washington"]]
The new datasource will now be listed as shown below.
Click Add New Query to start defining a query. We will write a query to find and display all users in the datasource.
1. In the Query ID field, enter getAllUsers.
2. In the Datasource field, select the CustomTabular datasource that you created in the previous step.
3. In the SQL field, enter a query:
```
SELECT ID,Name FROM Users
```
4. Now let's specify output mappings for the data. We will create output mappings for the following data: ID and Name.
  1. In the Output type field, specify the format in which the query results should be presented. You can select XML, JSON or RDF. We will use XML for this tutorial.
  2. In the Grouped by element field, specify a grouping for all the output mappings. This will be the XML element that will group the query result. Enter Users in this field.
  3. In the Row Name field, specify the XML element that should group each individual result. Enter User in this field.
  4. Click Add New Output Mapping to start creating the output mapping for the ID field. Enter values as shown below:
  5. Click Add to save the output mapping.
  6. Now, add another output mapping for the Name column.
  7. You will now have the following output mappings listed for the getAllUsers query:
Click Add New Operation to open the following screen.
In the Operation Name field, enter 'getAllUsersOp'.
In the Query ID field, select getAllUsers from the list.
Save the operation.
In this tutorial, we are only defining a SOAP service to expose the data. Therefore, you can click Finish to complete the data service creation process, after defining the operation. You will now be taken to the Deployed Services screen, which shows all the data services deployed on the server.

If you click Next after defining the operation, you will be taken to the Add Resources screen, which allows you to expose the data as a REST resource. If you want to create a REST resource, see Exposing Data as a REST Resource for information.

Invoking the custom tabular data service

You can try the data service you created by using the TryIt tool that is in your product by default.

Go to the Deployed Services screen.
Click the Try this service link for the CustomTabular data service. The TryIt Tool will open with the data service.
Select the getAllUsersOp operation.

Click Send to see the result:

<Users xmlns="http://ws.wso2.org/dataservice">
   <User>
      <ID>1</ID>
      <Name>Will Smith</Name>
   </User>
   <User>
      <ID>2</ID>
      <Name>Denzel Washington</Name>
   </User>
</Users>

Adding a custom query datasource to the data service

Follow the steps given below to create a custom query datasource.

Click Create under Data Service to open the Create Data Service window.
Enter CustomQueryDataService as the data service name.
When you get to the Add New Datasource screen, enter CustomQuery as the datasource ID.
Select Custom Data Source as the datasource type.
Select Custom Query Data Source.
In the Custom Data Source Class field, enter the following interface: org.wso2.carbon.dataservices.core.custom.datasource.EchoDataSource.

Click Add New Property to enter properties that will identify the data in the datasource.

Property	Value
prop1	prop_value1
prop2	prop_value2

<property name="prop1">prop_value1</property>
<property name="prop2">prop_value2</property>

The new datasource will be listed as shown below.
Click Add New Query to start defining a query. We will write a query to find and display all users in the datasource.
1. In the Query ID field, enter getValues.
2. In the Datasource field, select the CustomQuery datasource that you created in the previous step.
3. In the Expression field, enter a query:
```
column1,column2;R1C1 :param1,R1C2 :param2;R2C1 :param1,R2C2 :param2
```
4. Now let's specify input mappings to enter data. We will create input mappings for the fields given in the expression above: Column1 and Column2
  1. Click Add New Input Mapping to open the respective screen.
  2. Define a mapping for column 1 as shown below.
  3. Click Add to save the mapping.
  4. Add another input mapping for column 2. You will now have two input mappings for the two columns as shown below.
5. Now let's specify output mappings, which will determine how the result from your query will be presented when the query is invoked. The sample datasource we are using contains the following columns: Column 1 and Column 2. We will create an output mapping for each of these columns.
  1. In the Output type field, specify the format in which the query results should be presented. You can select XML, JSON or RDF. We will use XML for this tutorial.
  2. In the Grouped by element field, specify a grouping for all the output mappings. We will enter Rows.
  3. In the Row Name field, specify the XML element that will be used to display an individual value. We will enter Row.
  4. Click Add New Output Mapping to start creating the output mapping for column. Enter values as shown below.
  5. Click Add to save the output mapping.
  6. You will now have two output mappings created as shown below.
Click Add New Operation to open the Add New Operation screen.
In the Operation Name field, enter 'getValuesOp'.
In the Query ID field, select getValues from the list.
Save the operation.
In this tutorial, we are only defining a SOAP service to expose the data. Therefore, you can click Finish to complete the data service creation process, after defining the operation. You will now be taken to the Deployed Services screen, which shows all the data services deployed on the server.

If you click Next after defining the operation, you will be taken to the Add Resources screen, which allows you to expose the data as a REST resource. If you want to create a REST resource, see Exposing Data as a REST Resource for information.

Invoking the custom query data service

You can try the data service you created by using the TryIt tool that is in your product by default.

Go to the Deployed Services screen.
Click the Try this service link for the CustomQueryDataService data service. The TryIt Tool will open with the data service.
Select the getValuesOp operation.

Enter any values for the input parameters as shown below.

<!--Exactly 1 occurrence-->
      <xs:column1 xmlns:xs="http://ws.wso2.org/dataservice">1</xs:column1>
      <!--Exactly 1 occurrence-->
      <xs:column2 xmlns:xs="http://ws.wso2.org/dataservice">2</xs:column2

Click Send to see the result:

<Rows xmlns="http://ws.wso2.org/dataservice">
   <Row>
      <C1>R1C1 :param1</C1>
      <C2>R1C2 :param2</C2>
   </Row>
   <Row>
      <C1>R2C1 :param1</C1>
      <C2>R2C2 :param2</C2>
   </Row>
</Rows>

Property	Value
inmemory_datasource_schema	{Users:[ID,Name]}
inmemory_datasource_records	{Users:[["1","Will Smith"],["2","Denzel Washington"]]