Web Resource

The data services feature supports scraping and extracting web information and presenting it as a service. To demonstrate the process of service-enabling a web resource, the following guide uses Yahoo Weather Forecasts to extract weather information.

First, create a configuration file with information regarding the web resource and the xslt file, which has information regarding the response. Given below are sample configuration and xslt files.

Sample Configuration File:

<?xml version="1.0" encoding="UTF-8"?>
<config>
 <var-def name='weatherInfo'>
  <xslt>
   <xml>
    <html-to-xml>
     <http method='get' url='http://weather.yahoo.com/'/>
    </html-to-xml>
   </xml>
   <stylesheet>
    <file path="/media/data/web/template.xsl"/>
   </stylesheet>
  </xslt>
 </var-def>
</config>

Sample XSLT File:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" omit-xml-declaration="yes" indent="yes"/>
<xsl:template match="/">
  <WeatherInfo>
   <xsl:for-each select="//div[@id='myLocContainer']/ul/li">
     <Weather>
      <Region><xsl:value-of select="a/@title"/></Region>
      <Temp><xsl:value-of select="em"/></Temp>
     </Weather>
   </xsl:for-each>
  </WeatherInfo>
</xsl:template>
</xsl:stylesheet>

Follow the instructions below to create a data service. 

1. Log on to the product's management console and select "Data Service -> Create" under the "Main" menu.

2. The "Create Data Service" page appears. Fill in the fields and click "Next". The Data Service name is mandatory. In this example, "Yahoo Weather Forecasts" is used to extract weather information.

3. The "Data Sources" page appears. Click on the "Add New Data Source" link and add your Data Source details. Select the data source type as "Web Data Source".

Example 1:

You can either save the configuration to an xml and provide the file path, or can enter as an inline configuration. In example 1, we provide the sample configuration mentioned above as a file path.

Example 2:

In example 2, we use an inline configuration.

Click "Save". You have added a Data Source to your Web Service. Click "Next" to proceed.

4. The "Queries" page appears. Initially your service does not include any queries. Click on 'Add New Query' to add a new query, input/output mappings, events etc. to your Data Service. Enter query details according to the structure of the response you want.

Scraper variable should be the same as the output name in the configuration, which actually returns the output from the configuration. Eg: It should be 'weatherInfo' which is the var-def name in the configuration.

In this example, we have used "Grouped By Element", "Row Name" and "Row namespace" as follows.

Click the "Add New Output Mapping" button as shown in the screenshot above to create how the output looks like. Once you have entered the output mapping details, click "Main Configuration" and then the "Save" button as follows:

For information on adding validations to input mappings in the query, if any, refer to section Input Validators.

Once the query details are added click "Save". You have added one query to your data service configuration. To continue, click "Next".

5. The "Operations" page appears. Click the "Add new operation" link to create an operation. Select a query to be executed when this operation is called and click Save once done. For example,

6. The Data Service configuration is done. Click "Finish". (You can continue to add resources by clicking the "Next" button if you like)

7. You will be navigated to the "Deployed services" page.

From here, you can manage your data service. For instructions, refer to section Service Management.