Configuring a HDFS Datasource

HDFS datasource is used to set up a connection to a remote HDFS. In WSO2 DAS, you can create a HDFS datasource by specifying the datasource configuration in the <DAS_HOME>/repository/conf/datasources/analytics-datasources.xml file as follows.

<datasource>
    <name>WSO2_ANALYTICS_FS_DB_HDFS</name>
    <description>The datasource used for analytics file system</description>
    <jndiConfig>
        <name>jdbc/WSO2HDFSDB</name>
    </jndiConfig>
    <definition type="HDFS">
        <configuration>
            <property>
                <name>fs.default.name</name>
                <value>hdfs://localhost:9000</value>
            </property>
            <property>
                <name>dfs.data.dir</name>
                <value>/dfs/data</value>
            </property>
            <property>
                <name>fs.hdfs.impl</name>
                <value>org.apache.hadoop.hdfs.DistributedFileSystem</value>
            </property>
            <property>
                <name>fs.file.impl</name>
                <value>org.apache.hadoop.fs.LocalFileSystem</value>
            </property>
        </configuration>
    </definition>
</datasource>

For information on the above properties, and other properties that you can specify in this datasource configuration, go to Apache Hadoop Documentation.

Permission enforcement for remote HDFS clients

While the HDFS cluster is being set up for use by WSO2 DAS, please ensure that the UNIX user account in all clients are present in the HDFS nodes and that all users have R/W permission to HDFS root directory.

Alternatively, you can suspend the enforcement of distributed filesystem permission enforcement through adding the following configuration to <HADOOP_HOME>/etc/hadoop/hdfs-site.xml in all HDFS nodes:

<property>
	<name>dfs.permissions.enabled</name>
	<value>false</value>
</property>