Working with the File Connector

[ 1 Overview ] [ 2 Operation details ] [ 3 Sample configuration ]

Overview

The following operations allow you to work with the file connector. Click an operation name to see details on how to use it.

For a sample proxy service that illustrates how to work with the file connector, see Sample configuration.

Operation	Description

Operation	Description
append	Appends content to an existing file.
archive	Archives a file or folder.
copy	Copies a file or folder.
create	Creates a file or folder.
delete	Deletes a file or folder.
isFileExist	Checks the existence of a file.
listFileZip	Lists all files inside zip file.
move	Moves a file or folder.
read	Reads content from a file.
search	Finds a file based on a file pattern and directory pattern.
unzip	Decompresses a zip file.
ftpOverProxy	Connects to a FTP server through a proxy.
send	Sends a file to a specified location.
getSize	Returns the size of the file.
getLastModifiedTime	Returns last modified time of a file.
splitFile	Splits files into multiple chunks.
mergeFiles	Merges files into a single file.
readSpecifiedLines	Reads specified lines from a file.
readALine	Reads specified line from a file.

Operation details

This section provides further details on the operations related to file connector.

Appending content to an existing file

The append operation appends content to an existing file in a specified location.

append

<fileconnector.append>
    <destination>{$ctx:destination}</destination>
    <inputContent>{$ctx:inputContent}</inputContent>
	<position>{$ctx:position}</position>
	<encoding>{$ctx:encoding}</encoding>
	<setTimeout>{$ctx:setTimeout}</setTimeout>
    <setPassiveMode>{$ctx:setPassiveMode}</setPassiveMode>
    <setSoTimeout>{$ctx:setSoTimeout}</setSoTimeout>
    <setUserDirIsRoot>{$ctx:setUserDirIsRoot}</setUserDirIsRoot>
    <setStrictHostKeyChecking>{$ctx:setStrictHostKeyChecking}</setStrictHostKeyChecking>
</fileconnector.append>

Properties

destination: The location of the file for which content needs to be appended.
inputContent: The content to be appended.
position: Position to append the content. If you provide a valid position, content will be appended to that position. Otherwise, content will be appended at the end of the file.
encoding [optional]: The encoding that is supported. Possible values are US-ASCII ,UTF-8 and UTF-16 .
setTimeout [optional]: The timeout value on the JSC(Java Secure Channel) session in milliseconds. e.g., 100000.
setPassiveMode [optional]: Set to true if you want to enable passive mode.
setSoTimeout [optional]: The socket timeout value for the FTP client. e.g., 100000.
setUserDirIsRoot [optional]: Set to true if you want to use root as the user directory.
setStrictHostKeyChecking [optional]: Sets the host key checking to use .e.g., no.

Sample request

Following is a sample REST/JSON request that can be handled by the append operation.

Sample Request for append

{
     "destination":"/home/vive/Desktop/file/append.txt",
     "inputContent":"Add Append Text."
}

Archiving a file or folder

The archive operation archives files or folder. This operation supports the ZIP archive type.

Sample request

Following is a sample REST/JSON request that can be handled by the archives operation.

Sample Request for archives

{
     "source":"/home/vive/Desktop/file",
     "destination":"/home/user/test/file.zip",
     "includeSubDirectories":"true"
}

Copying a file

The copy operation copies files from one location to another. This operation can be used when you want to copy any kind of files and large files too, you can copy particular files with given file pattern.

copy

<fileconnector.copy>
    <source>{$ctx:source}</source>
    <destination>{$ctx:destination}</destination>
	<filePattern>{$ctx:filePattern}</filePattern>
    <setTimeout>{$ctx:setTimeout}</setTimeout>
    <setPassiveMode>{$ctx:setPassiveMode}</setPassiveMode>
    <setSoTimeout>{$ctx:setSoTimeout}</setSoTimeout>
    <setUserDirIsRoot>{$ctx:setUserDirIsRoot}</setUserDirIsRoot>
    <setStrictHostKeyChecking>{$ctx:setStrictHostKeyChecking}</setStrictHostKeyChecking>
	<includeParentDirectory>{$ctx:includeParentDirectory}</includeParentDirectory>
    <includeSubDirectories>{$ctx:includeSubDirectories}</includeSubDirectories>
</fileconnector.copy>

Properties

source: The location of the file. This can be a file on the local physical file system or a file on an FTP server.
For local files, the URI format is [file://] absolute-path, where absolute-path is a valid absolute file name for the local platform. UNC names are supported under Windows (e.g., file:///home/user/test/test.txt or file:///C:/Windows ).
For files on an FTP server, the URI format is ftp://[ username[: password]@] hostname[: port][ relative-path] (e.g., ftp://myusername:mypassword@somehost/pub/downloads/test.txt).
destination: The location where the files should be copied to.
filePattern [optional] : The pattern of the files to be copied. (e.g., [a-zA-Z][a-zA-Z]*.(txt|xml|jar))
setTimeout [optional]: The timeout value on the JSC(Java Secure Channel) session in milliseconds. e.g., 100000.
setPassiveMode [optional]: Set to true if you want to enable passive mode.
setSoTimeout [optional]: The socket timeout value for the FTP client. e.g., 100000.
setUserDirIsRoot [optional]: Set to true if you want to use root as the user directory.
setStrictHostKeyChecking [optional]: Sets the host key checking to use .e.g., no.
includeParentDirectory [optional] : Set to true if you want to include the parent directory.
includeSubDirectories [optional] : Set to true if you want to include the sub directories.

Sample request

Following is a sample REST/JSON request that can be handled by the copy operation.

Sample Request for copy

{
     "source":"/home/vive/Desktop/file",
     "destination":"/home/user/test/fileCopy",
     "filePattern":".*\.xml",
     "includeParentDirectory":"false",
     "includeSubDirectories":"false"
}

Creating a file or folder

The create operation creates a file or folder in a specified location. When creating a file, you can either create the file with content or without content.

create

<fileconnector.create>
    <filePath>{$ctx:filePath}</filePath>
    <inputContent>{$ctx:inputContent}</inputContent>
	<encoding>{$ctx:encoding}</encoding>
    <setTimeout>{$ctx:setTimeout}</setTimeout>
    <setPassiveMode>{$ctx:setPassiveMode}</setPassiveMode>
    <setSoTimeout>{$ctx:setSoTimeout}</setSoTimeout>
    <setUserDirIsRoot>{$ctx:setUserDirIsRoot}</setUserDirIsRoot>
    <setStrictHostKeyChecking>{$ctx:setStrictHostKeyChecking}</setStrictHostKeyChecking>
</fileconnector.create>

Properties

filePath: The location of the file. This can be a file on the local physical file system or a file on an FTP server.
For local files, the URI format is [file://] absolute-path, where absolute-path is a valid absolute file name for the local platform. UNC names are supported under Windows (e.g., file:///home/user/test/test.txt or file:///C:/Windows ).
For files on an FTP server, the URI format is ftp://[ username[: password]@] hostname[: port][ relative-path] (e.g., ftp://myusername:mypassword@somehost/pub/downloads/test.txt). For creating a file, the file path should have extension (e.g. , file:///home/user/test/test.txt)
inputContent [optional] : The content of the file.
encoding [optional] : The encoding that is supported. Possible values are US-ASCII ,UTF-8 and UTF-16 .
setTimeout [optional]: The timeout value on the JSC(Java Secure Channel) session in milliseconds. e.g., 100000.
setPassiveMode [optional]: Set to true if you want to enable passive mode.
setSoTimeout [optional]: The socket timeout value for the FTP client. e.g., 100000.
setUserDirIsRoot [optional]: Set to true if you want to use root as the user directory.
setStrictHostKeyChecking [optional]: Sets the host key checking to use .e.g., no.

Sample request

Following is a sample REST/JSON request that can be handled by the create operation.

Sample Request for create

{
     "filePath":"sftp://UserName:Password@Host/home/connectors/create.txt",
     "inputContent":"InputContent Text",
     "encoding":"UTF8"
}

Deleting a file or folder

The delete operation deletes a file or folder from the file system.

delete

<fileconnector.delete>
    <source>{$ctx:source}</source>
    <filePattern>{$ctx:filePattern}</filePattern>
	<setTimeout>{$ctx:setTimeout}</setTimeout>
    <setPassiveMode>{$ctx:setPassiveMode}</setPassiveMode>
    <setSoTimeout>{$ctx:setSoTimeout}</setSoTimeout>
    <setUserDirIsRoot>{$ctx:setUserDirIsRoot}</setUserDirIsRoot>
    <setStrictHostKeyChecking>{$ctx:setStrictHostKeyChecking}</setStrictHostKeyChecking>
    <includeSubDirectories>{$ctx:includeSubDirectories}</includeSubDirectories>
</fileconnector.delete>

Properties

source: The location of the file. This can be a file on the local physical file system or a file on an FTP server.
For local files, the URI format is [file://] absolute-path, where absolute-path is a valid absolute file name for the local platform. UNC names are supported under Windows (e.g., file:///home/user/test/test.zip or file:///C:/Windows ).
For files on an FTP server, the URI format is ftp://[ username[: password]@] hostname[: port][ relative-path] (e.g., ftp://myusername:mypassword@somehost/pub/downloads/test.txt).
filePattern: The pattern of the files to be deleted.(e.g., [a-zA-Z][a-zA-Z]*.(txt|xml|jar)).
setTimeout [optional]: Sets the timeout value on Jsch(Java Secure Channel) session. e.g., 100000.
setPassiveMode [optional]: Sets the passive mode to enter into passive mode. e.g., true.
setSoTimeout [optional]: Sets the socket timeout for the FTP client. e.g., 100000.
setUserDirIsRoot [optional]: Sets the whether to use the user directory as root. e.g., flase.
setStrictHostKeyChecking [optional]: Sets the host key checking to use .e.g., no.
includeSubDirectories [optional] : Set to true if you want to include the sub directories.

Sample request

Following is a sample REST/JSON request that can be handled by the delete operation.

Sample Request for delete

{
     "source":"/home/vive/Desktop/file",
     "filePattern":".*\.txt",
     "includeSubDirectories":"true"
}

Checking the existence of a file

The isFilleExist operation checks the existence of a file in a spacified location. This operation returns true if the file exists and returns false if the file does not exist in the specified location.

isFilleExist

<fileconnector.isFileExist>
    <source>{$ctx:source}</source>
    <setTimeout>{$ctx:setTimeout}</setTimeout>
    <setPassiveMode>{$ctx:setPassiveMode}</setPassiveMode>
    <setSoTimeout>{$ctx:setSoTimeout}</setSoTimeout>
    <setUserDirIsRoot>{$ctx:setUserDirIsRoot}</setUserDirIsRoot>
    <setStrictHostKeyChecking>{$ctx:setStrictHostKeyChecking}</setStrictHostKeyChecking>
</fileconnector.isFileExist>

Properties

source: The location of the file. This can be a file on the local physical file system or a file on an FTP server.
For local files, the URI format is [file://] absolute-path, where absolute-path is a valid absolute file name for the local platform. UNC names are supported under Windows (e.g., file:///home/user/test/test.zip or file:///C:/Windows ).
For files on an FTP server, the URI format is ftp://[ username[: password]@] hostname[: port][ relative-path] (e.g., ftp://myusername:mypassword@somehost/pub/downloads/test.txt).
setTimeout [optional]: Sets the timeout value on Jsch(Java Secure Channel) session. e.g., 100000.
setPassiveMode [optional]: Sets the passive mode to enter into passive mode. e.g., true.
setSoTimeout [optional]: Sets the socket timeout for the FTP client. e.g., 100000.
setUserDirIsRoot [optional]: Sets the whether to use the user directory as root. e.g., flase.
setStrictHostKeyChecking [optional]: Sets the host key checking to use .e.g., no.

Sample request

Following is a sample REST/JSON request that can be handled by the isFileExist operation.

Sample Request for isFileExist

{
     "source":"/home/vive/Desktop/file/test.txt"
}

Listing all files inside a compressed file

The listFileZip operation lists all the file paths inside a compressed file. This operation supports the ZIP archive type.

listFileZip

<fileconnector.listFileZip>
    <source>{$ctx:source}</source>
    <setTimeout>{$ctx:setTimeout}</setTimeout>
    <setPassiveMode>{$ctx:setPassiveMode}</setPassiveMode>
    <setSoTimeout>{$ctx:setSoTimeout}</setSoTimeout>
    <setUserDirIsRoot>{$ctx:setUserDirIsRoot}</setUserDirIsRoot>
    <setStrictHostKeyChecking>{$ctx:setStrictHostKeyChecking}</setStrictHostKeyChecking>
</fileconnector.listFileZip>

Properties

source: The location of the file. This can be a file on the local physical file system or a file on an FTP server.
For local files, the URI format is [file://] absolute-path, where absolute-path is a valid absolute file name for the local platform. UNC names are supported under Windows (e.g., file:///home/user/test/test.zip or file:///C:/Windows ).
For files on an FTP server, the URI format is ftp://[ username[: password]@] hostname[: port][ relative-path] (e.g., ftp://myusername:mypassword@somehost/pub/downloads/test.zip).
setTimeout [optional]: Sets the timeout value on Jsch(Java Secure Channel) session. e.g., 100000.
setPassiveMode [optional]: Sets the passive mode to enter into passive mode. e.g., true.
setSoTimeout [optional]: Sets the socket timeout for the FTP client. e.g., 100000.
setUserDirIsRoot [optional]: Sets the whether to use the user directory as root. e.g., flase.
setStrictHostKeyChecking [optional]: Sets the host key checking to use .e.g., no.

Sample request

Following is a sample REST/JSON request that can be handled by the listFileZip operation.

Sample Request for listFileZip

{
     "source":"/home/vive/Desktop/file/test.zip"
}

Moving a file

The move operation moves a file or folder from one location to another.

Limitation

The move operation can only move a file/folder within the same server. For example, you can move a file/folder from one local location to another local location, or from one remote location to another remote location on the same server. You cannot use the move operation to move a file/folder between different servers. If you want to move a file/folder from a local location to a remote location or vice versa, use the copy operation followed by delete operation instead of using the move operation.

move

<fileconnector.move>
    <source>{$ctx:source}</source>
    <destination>{$ctx:destination}</destination>
    <setTimeout>{$ctx:setTimeout}</setTimeout>
    <setPassiveMode>{$ctx:setPassiveMode}</setPassiveMode>
    <setSoTimeout>{$ctx:setSoTimeout}</setSoTimeout>
    <setUserDirIsRoot>{$ctx:setUserDirIsRoot}</setUserDirIsRoot>
    <setStrictHostKeyChecking>{$ctx:setStrictHostKeyChecking}</setStrictHostKeyChecking>
    <filePattern>{$ctx:filePattern}</filePattern>
	<includeParentDirectory>{$ctx:includeParentDirectory}</includeParentDirectory>
    <includeSubDirectories>{$ctx:includeSubDirectories}</includeSubDirectories>
</fileconnector.move>

Properties

source: The location of the file. This can be a file on the local physical file system, or a file on an FTP server.
For local files, the URI format is [file://] absolute-path where absolute-path is a valid absolute file name for the local platform. UNC names are supported under Windows (e.g., file:///home/user/test or file:///C:/Windows ).
For files on an FTP server, the URI format is ftp://[ username[: password]@] hostname[: port][ relative-path] (e.g., ftp://myusername:mypassword@somehost/pub/downloads/test.txt).
destination: The location where the file has to be moved to.
setTimeout [optional]: The timeout value on the JSC(Java Secure Channel) session in milliseconds. e.g., 100000.
setPassiveMode [optional]: Set to true if you want to enable passive mode.
setSoTimeout [optional]: The socket timeout value for the FTP client. e.g., 100000.
setUserDirIsRoot [optional]: Set to true if you want to use root as the user directory.
setStrictHostKeyChecking [optional]: Sets the host key checking to use .e.g., no.
filePattern [optional] : The pattern of the files to be copied. (e.g [a-zA-Z][a-zA-Z]*.(txt|xml|jar))
includeParentDirectory [optional] : Set to true if you want to include the parent directory.
includeSubDirectories [optional] : Set to true if you want to include the sub directories.

Sample request

Following is a sample REST/JSON request that can be handled by the move operation.

Sample Request for move

{
     "source":"/home/vive/Desktop/file",
     "destination":"/home/vive/Desktop/move",
     "filePattern":".*\.txt",
     "includeParentDirectory":"true",
     "includeSubDirectories":"true"
}

Reading content from a file

The read operation reads content from an existing file in a specified location.

read

<fileconnector.read>
    <source>{$ctx:source}</source>
    <filePattern>{$ctx:filePattern}</filePattern>
    <contentType>{$ctx:contentType}</contentType>
    <setTimeout>{$ctx:setTimeout}</setTimeout>
    <setPassiveMode>{$ctx:setPassiveMode}</setPassiveMode>
    <setSoTimeout>{$ctx:setSoTimeout}</setSoTimeout>
    <setUserDirIsRoot>{$ctx:setUserDirIsRoot}</setUserDirIsRoot>
    <setStrictHostKeyChecking>{$ctx:setStrictHostKeyChecking}</setStrictHostKeyChecking>
</fileconnector.read>

Properties

source: The location of the file. This can be a file on the local physical file system or a file on an FTP server.
For local files, the URI format is [file://] absolute-path, where absolute-path is a valid absolute file name for the local platform. UNC names are supported under Windows (e.g., file:///home/user/test/test.xml or file:///C:/Windows/me.txt ). For files on an FTP server, the URI format is ftp://[ username[: password]@] hostname[: port][ relative-path] (e.g., ftp://myusername:mypassword@somehost/pub/downloads/test.txt).
filePattern: The pattern of the file to be read.
contentType: Content type of the files processsed by the connector.
streaming [optional]: The streaming mode, This can be either true or false.
setTimeout [optional]: Sets the timeout value on Jsch(Java Secure Channel) session. e.g., 100000.
setPassiveMode [optional]: Sets the passive mode to enter into passive mode. e.g., true.
setSoTimeout [optional]: Sets the socket timeout for the FTP client. e.g., 100000.
setUserDirIsRoot [optional]: Sets the whether to use the user directory as root. e.g., flase.
setStrictHostKeyChecking [optional]: Sets the host key checking to use .e.g., no.

Note

To enable streaming for large files, you have to add the following message builder and formatter in the <ESB_HOME>/repository/conf/axis2/axis2.xml file:

Add <messageFormatter contentType="application/file"
class="org.wso2.carbon.relay.ExpandingMessageFormatter"/> under message formatters.
Add <messageBuilder contentType="application/file"
class="org.apache.axis2.format.BinaryBuilder"/> under message builders.

Sample request

Following is a sample REST/JSON request that can be handled by the read operation.

Sample Request for read

{
     "source":"/home/vive/Desktop/file",
     "contentType":"application/xml",
     "filePattern":".*\.xml",
     "streaming":"false"
}

Working with the File Connector

Overview

Operation details

Appending content to an existing file

append

Properties

Sample request

Sample Request for append

Archiving a file or folder

archive

Sample request

Sample Request for archives

Copying a file

copy

Properties

Sample request

Sample Request for copy

create

Properties

Sample Request for create

Deleting a file or folder

delete

Properties

Sample request

Sample Request for delete

Checking the existence of a file

isFilleExist

Properties

Sample request

Sample Request for isFileExist

Listing all files inside a compressed file

listFileZip

Sample request

Sample Request for listFileZip

Moving a file

move

Sample request

Sample Request for move

Reading content from a file

read

Sample request

Sample Request for read

Searching for a file