Web Resource

The Data Services Server supports scraping and extracting Web information and presenting it as a service.

First, create a Web resource by creating a configuration file with information regarding the Web resource and the XSLT file, which has information regarding the response. Given below are example configuration and XSLT files. The following examples use Yahoo Weather Forecasts to extract weather information.

Sample Configuration File:

<config>
 <var-def name='weatherInfo'>
  <xslt>
   <xml>
    <html-to-xml>
      <http method='get' url='http://weather.yahoo.com/'/>
    </html-to-xml>
   </xml>
   <stylesheet>
    <file path="/media/data/web/template.xsl"/>
   </stylesheet>
  </xslt>
 </var-def>
</config>

Sample XSLT File:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" omit-xml-declaration="yes" indent="yes"/>
<xsl:template match="/">
  <WeatherInfo>
   <xsl:for-each select="//div[@id='myLocContainer']/ul/li">
     <Weather>
      <Region><xsl:value-of select="a/@title"/></Region>
      <Temp><xsl:value-of select="em"/></Temp>
     </Weather>
   </xsl:for-each>
  </WeatherInfo>
</xsl:template>
</xsl:stylesheet>

Next, follow the steps in Creating Datasources. When you select Web Datasource as the datasource type at the time a datasource is created, the Web-specific options appear. For example:

The above example saves the configuration file to an XML file and provides the file path. You can also enter as an inline configuration by clicking the relevant option as shown below:

To write a sample query to this datasource, see Sample 5: Querying a Web Datasource or read everything about query writing in Writing Data Service Queries.

Also, see a demonstration of service enabling a Web resource in Web Resource Sample.