Input Kafka Event Adapter

The Apache Kafka input adapter reads the tail of a given file and inputs that to the WSO2 product engine. This feature is donated by Andres Gomez Ferrer. For more information, go to Apache Kafka documentation.

Prerequisites

Set up the below prerequisites to start configuring an input Apache Kafka event adapter.

1. Download Apache Kafka server. 

2. Copy the following client JAR files from <KAFKA_HOME>/lib/ directory  to <CEP_HOME>/repository/components/lib/ directory.
   
   • kafka_2.10-0.8.1.jar
   • zkclient-0.3.jar
   • scala-library-2.10.1.jar
   • zookeeper-3.3.4.jar
3. Copy all the JAR files, which are located in <KAFKA_HOME>/lib/ directory to <CEP_HOME>/samples/lib/ directory.

Configuring input Kafka event adapter

You can configure input Kafka event adapter using the management console or using a configuration file.

Configure using the management console

Follow the instructions below to configure input Kafka event adapter using the management console. This deploys the event adapter in the <PRODUCT_HOME>/

repository/deployment/server/inputeventadaptors/ directory.

1. Log in to product's management console, and click Configure.
2. Click Input Event Adaptors in the Event Processor Configs menu, and then click Add Input Event Adaptor.

3. Enter details as shown in the below example. 
   

4.  Click Add Event Adaptor.

Configure using a configuration file

Follow the instructions below to configure input Kafka event adapter using a configuration file.

1. Create an XML file with the following Kafka input event adapter configurations. Input event adapter implementation must start with <inputEventAdaptor> as the root element.

   <inputEventAdaptor name="kafkaInputAdaptor" statistics="disable"
     trace="disable" type="kafka" xmlns="http://wso2.org/carbon/eventadaptormanager">
     <property name="optional.configuration">zk.connectiontimeout.ms:6000,zk.sessiontimeout.ms:7000</property>
     <property name="group.id">test-group</property>
     <property name="zookeeper.id">127.0.0.1:2181</property>
     <property name="threads">5</property>
   </inputEventAdaptor>

2. Add the XML file to the  <PRODUCT_HOME>/repository/deployment/server/inputeventadaptors/ directory. Since hot deployment is enabled, you can simply add/remove files to deploy/undeploy to/from the server.  
    

Configuring input Kafka event adapter message properties

The configured input event adapters are used in Event Builders. With input event adapter message properties, event builders are able to extract data from the incoming events to build the event streams. Following are the specific message properties of input Kafka event adapter.

The message properties of the input Kafka event adapter are described below.