Web Resource

The Data Services Server supports scraping and extracting Web information and presenting it as a service.

First, create a Web resource by creating a configuration file with information regarding the Web resource and the XSLT file, which has information regarding the response. Given below are example configuration and XSLT files. The following examples use Yahoo Weather Forecasts to extract weather information.

Sample Configuration File:

<config>
 <var-def name='weatherInfo'>
  <xslt>
   <xml>
    <html-to-xml>
      <http method='get' url='http://weather.yahoo.com/'/>
    </html-to-xml>
   </xml>
   <stylesheet>
    <file path="/media/data/web/template.xsl"/>
   </stylesheet>
  </xslt>
 </var-def>
</config>

Sample XSLT File:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" omit-xml-declaration="yes" indent="yes"/>
<xsl:template match="/">
  <WeatherInfo>
   <xsl:for-each select="//div[@id='myLocContainer']/ul/li">
     <Weather>
      <Region><xsl:value-of select="a/@title"/></Region>
      <Temp><xsl:value-of select="em"/></Temp>
     </Weather>
   </xsl:for-each>
  </WeatherInfo>
</xsl:template>
</xsl:stylesheet>

Web datasources allow you to create a data service using a Web resource as the data storage type.

You can get the Add New Datasource screen shown below on the management console by executing steps 1 to 3 in creating a data service. Then, select Web Datasource as the Datasource Type. It opens Web-datasource-specific options as shown below:

The above example saves the configuration file to an XML file and provides the file path. You can also enter as an inline configuration by clicking the relevant option as shown below:

To write a sample query to this data source, see Sample 5: Querying a Web data source or read everything about query writing in Writing Data Service Queries.

Also, see a demonstration of service enabling a Web resource in Web Resource Sample.