Versions Compared

Old Version 35

changes.mady.by.user Former user

Saved on

compared with

New Version Current

changes.mady.by.user Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The Virtual File System (VFS) transport is used by WSO2 ESB to process files in the specified source directory. After processing the files, it moves them to a specified location or deletes them. Note that files cannot remain in the source directory after processing or they will be processed again, so if you need to maintain these files or keep track of which files have been processed, specify the option to move them instead of deleting them after processing. If you want to move files into a database, use the VFS transport and the DBReport mediator (for an example, see Sample 271: File Processing).

Excerpt
hiddentrue

NOTE TO WRITERS: the previous sentences about files not remaining in the source directory is from http://docs.wso2.org/display/FAQ/Transports. If you need to make changes to this content, be sure to make it in the source FAQ as well!

Table of Contents
maxLevel3
minLevel3
locationtop
typeflat
separatorpipe

Enabling the transport

...

Tip

Tip: When you transfer a file to a remote FTP location via VFS, WSO2 EI tries to detect the FTP location by navigating from the root folder first. If the EI does not have at least list permission to the root (/), the file transfer fails.

Excerpt
hiddentrue

NOTE TO WRITERS: the previous sentences about files not remaining in the source directory is from http://docs.wso2.org/display/FAQ/Transports. If you need to make changes to this content, be sure to make it in the source FAQ as well!

...

Table of Contents
maxLevel3
minLevel3
locationtop
typeflat
separatorpipe

...

Enabling the transport

This transport is based on the Apache Commons VFS implementation, which is provided in the <ESB_HOME>/repository/components/plugins/commons-vfs<version>.jar file. The transport is a module of the Apache Synapse project, and the synapse-vfs-transport.jar file contains the necessary classes, including those that implement the listener and sender APIs:

...

  1. In <ESB_HOME>/repository/conf/axis2/axis2.xml, in the messageBuilders section, add the binary message builder as follows:

    Code Block
    languagehtml/xml
    <messageBuilder contentType="application/binary" class="org.apache.axis2.format.BinaryBuilder"/>

    and in the messageFormatters section, add the binary message formatter as follows:

    Code Block
    languagehtml/xml
    <messageFormatter contentType="application/binary" class="org.apache.axis2.format.BinaryFormatter"/>

  2. In the proxy service where you use the VFS transport, add the following parameter to enable streaming (see VFS service-level parameters below for more information):

    Code Block
    languagehtml/xml
    <parameter name="transport.vfs.Streaming">true</parameter>

  3. In the same proxy service, before the Send mediator, add the following property:

    Info
    titleNote

    You also need to add the following property if you want to use the VFS transport to transfer files from VFS to VFS.

    Code Block
    languagehtml/xml
    <property name="ClientApiNonBlocking" value="true" scope="axis2" action="remove"/>

    For more information, see Example 3 of the Send Mediator.

...


FailedRecordNext
RetryDurationThe time in milliseconds to wait before retrying the move task

Parameter Name

Description

Required

Possible Values

Default Value

transport.vfs.
FileURI

The URI where the files you want to process are located. You can specify connection-level parameters on the URL (see VFS URL parameters below).

When you need to access the absolute path of the URL, you can define the URL with sftpPathFromRoot:

sftp<parameter name="transport.vfs.FileURI">sftp://[ username[: password]@] hostname[: port][ absolute-path]?sftpPathFromRoot=truetrue</parameter>

Yes

A valid file URI in the following form:
file://<path>

 

transport.vfs.
ContentType

Content type of the files processed by the transport. To specify the encoding, follow the content type with a semi-colon and the character set. For example:

<parameter name="transport.vfs.ContentType“>text/plain;charset=UTF-32</parameter>
When writing a file, you can set a different encoding with the CHARACTER_SET_ENCODING property:
<property name="CHARACTER_SET_ENCODING" value="UTF-8" scope="axis2" type="STRING"/>

Yes

A valid content type for the files (e.g., text/xml). You can specify the encoding after the content type, such as text/plain;charset=UTF-32 

 

transport.vfs.
FileNamePattern

If the VFS listener should process only a subset of the files available at the specified file URI location, use this parameter to select those files by name using a regular expression.

No

A regular expression to select files by name (e.g., *\.xml)

 

transport.
PollInterval

The polling interval for the transport receiver to poll the file URI location. The value is expressed in seconds unless you add "ms" for milliseconds, e.g., "2" or "2000ms" to specify 2 seconds.

No

A positive integer.

 

transport.vfs.
ActionAfterProcess

Whether to move or delete the files after the transport has processed them.

No

MOVE or DELETE

DELETE

transport.vfs.
ActionAfterFailure

Whether to move or delete the files if a failure occurs.

No

MOVE or DELETE

DELETE

transport.vfs.
MoveAfterProcess

Where to move the files after processing if ActionAfterProcess is MOVE.

Yes, if
ActionAfterProcess
is MOVE

A valid file URI

 

transport.vfs.
MoveAfterFailure

Where to move the files after processing if ActionAfterFailure is MOVE.

Yes, if
ActionAfterFailure
is MOVE

A valid file URI

 

transport.vfs.
ReplyFileURI

The location where reply files should be written by the transport.

No

A valid file URI

 

transport.vfs.
ReplyFileName

The name for reply files written by the transport.

No

A valid file name

response.xml

transport.vfs.
MoveTimestampFormat

The pattern/format of the timestamps added to file names as prefixes when moving files.

No

A valid timestamp pattern
(e.g., yyyy-MM-dd'T'HH:mm:ss.SSSZ )

 

transport.vfs.
Streaming

Whether files should be transferred in streaming mode, which is useful when transferring large files

No

true or false

false

transport.vfs.
ReconnectTimeout

Reconnect timeout value in seconds to be used in case of an error when transferring files

No

A positive integer

30 sec

transport.vfs.
MaxRetryCount

Maximum number of retry attempts to carry out in case of errors.

No

A positive integer

3

transport.vfs.Append

When writing the response to a file, whether the response should be appended to the response file instead of overwriting the file. This value should be defined as a query parameter in the out/reply
file URI. For example:
"vfs:
file URI. For example:
"vfs:file:///home/user/test/out?
transport.vfs.Append=true"

or:

<parameter name="
transport.vfs.ReplyFileURI">
file:///home/user/test/out? /out?
transport.vfs.Append=true
</parameter>

No

true or false

false (the response file will be completely overwritten).

transport.vfs.Append=true"

or:

<parameter name=" .
MoveAfterFailedMove

Where to move the failed file.

No

A valid file URI

 

transport.vfs. ReplyFileURI">
file:///home/user/test/out?
transport.vfs.Append=true
</parameter>

No

true or false

false (the response file will be completely overwritten).
FailedRecordsFileName

The name of the file that maintains
the list of failed files.

No

A valid file name

vfs-move-failed-records.
properties

transport.vfs.
MoveAfterFailedMove FailedRecordsFile
Destination

Where to move store the failed records file.

No

A valid file folder URI

 repository/conf/

transport.vfs. .
MoveFailedRecord
FailedRecordsFileNameThe TimestampFormat

Entries in the failed records file include the name of the file that maintains
failed and the list of failed filestimestamp of its failure. This property configures the time stamp format.

No

A valid file namevfs-move-failed-records.
properties timestamp pattern
(e.g., yyyy-MM-dd'T'HH:mm:ss.SSSZ )

dd-MM-yyyy HH:mm:ss

transport.vfs.
FailedRecordsFile FailedRecordNext
DestinationWhere to store the failed records file RetryDuration

The time in milliseconds to wait before retrying the move task.

No

A folder URIrepository/conf/positive integer

3000 milliseconds

transport.vfs.
MoveFailedRecord
TimestampFormat

Entries in the failed records file include the name of the file that failed and the timestamp of its failure. This property configures the time stamp format.

No

A valid timestamp pattern
(e.g., yyyy-MM-dd'T'HH:mm:ss.SSSZ )

dd-MM-yyyy HH:mm:ssLocking

By default, file locking is enabled in the VFS transport. This parameter lets you configure the locking behavior on a per service basis. You can also disable locking globally by specifying the parameter at the receiver level and selectively enable locking only for a set of services.

No

enable or disable

enable

transport.vfs.
FileProcessCount		This setting allows you to throttle the VFS listener by processing files in batches. Specify the number of files you want to process in each batch.NoA positive integer

3000 milliseconds

transport.vfs.Locking

By default, file locking is enabled in the VFS transport. This parameter lets you configure the locking behavior on a per service basis. You can also disable locking globally by specifying the parameter at the receiver level and selectively enable locking only for a set of services.

No

enable or disable

enable

transport.vfs.
FileProcessCount		This setting allows you to throttle the VFS listener by processing files in batches. Specify the number of files you want to process in each batch.NoA positive integer, such as 10, such as 10N/A
transport.vfs.
FileProcessInterval		The interval in milliseconds between two file processes.NoA positive integer, such as 1000N/A
transport.vfs.SFTPIdentitiesLocation of the private keyNoA valid file pathN/A
transport.vfs.SFTPIdentityPassPhrasePassphrase of the private keyNoA valid passphraseN/A
transport.vfs.
FileProcessInterval		The interval in milliseconds between two file processes.NoA positive integer, such as 1000N/ASFTPUserDirIsRootIf the SFTP user directory should be treated as rootNotrue or falsetrue

Anchor
URLparams
URLparams

VFS URL parameters

...

Parameter Name

Description

Possible Values

Default Value

vfs.passive

Enable FTP passive mode. This is required when the FTP client and server are not in the same network.

true | falsefalse
transport. vfs . AppendIf file with same name exists, this parameter tells whether to create a new file and write content or append content to existing file

true | false

false
vfs.protection

Set data channel protection level using FTP PROT command

  • C - Clear
  • S - Safe(SSL protocol only)
  • E - Confidential(SSL protocol only)
  • P - Private
C
vfs.ssl.keystorePrivate key store to use for mutual SSL. Your keystore must be signed by a certificate authority. For more information, see http://docs.oracle.com/cd/E19509-01/820-3503/ggfen/index.html.String - Path of keystore 
vfs.ssl.kspasswordPrivate key store passwordString 
vfs.ssl.keypasswordPrivate key passwordString 
vfs.ssl.truststoreTrust store to use for FTPSString - Path of keystore 
vfs.ssl.tspasswordTrust store passwordString 
transport.vfs.CreateFolderIf the directory does not exists create and write the filetrue | falsefalse
transport. vfs.SendFileSynchronouslyWhether to send files synchronously to the file host. When this parameter is set to true, files will be sent one after another to the file host. This synchronous write can be configured on a per host basis.true | falsefalse

...