Input Kafka Event Adapter

The Apache Kafka input adapter reads the tail of a given file and inputs that to the WSO2 product engine. This feature is donated by Andres Gomez Ferrer. For more information, go to Apache Kafka documentation.

Prerequisites

Set up the below prerequisites to start configuring an input Apache Kafka event adapter.

1. Download Apache Kafka server. 

2. Copy the following client JAR files from <KAFKA_HOME>/lib/ directory  to <CEP_HOME>/repository/components/lib/ directory.
   
   • kafka_2.10-0.8.1.jar
   • zkclient-0.3.jar
   • scala-library-2.10.1.jar
   • zookeeper-3.3.4.jar
3. Copy all the JAR files, which are located in <KAFKA_HOME>/lib/ directory to <CEP_HOME>/samples/lib/ directory.

Configuring input Kafka event adapter

You can configure input Kafka event adapter using the management console or using a configuration file.

Configure using the management console

Follow the instructions below to configure input Kafka event adapter using the management console. This deploys the event adapter in the <PRODUCT_HOME>/

repository/deployment/server/inputeventadaptors/ directory.

1. Log in to product's management console, and click Configure.
2. Click Input Event Adaptors in the Event Processor Configs menu, and then click Add Input Event Adaptor.

3. Enter details as shown in the below example. 
   

4.  Click Add Event Adaptor.

Configure using a configuration file

Follow the instructions below to configure input Kafka event adapter using a configuration file.

1. Create an XML file with the following Kafka input event adapter configurations. Input event adapter implementation must start with <inputEventAdaptor> as the root element.

   <inputEventAdaptor name="kafkaInputAdaptor" statistics="disable"
     trace="disable" type="kafka" xmlns="http://wso2.org/carbon/eventadaptormanager">
     <property name="optional.configuration">zk.connectiontimeout.ms:6000,zk.sessiontimeout.ms:7000</property>
     <property name="group.id">test-group</property>
     <property name="zookeeper.id">127.0.0.1:2181</property>
     <property name="threads">5</property>
   </inputEventAdaptor>

2. Add the XML file to the  <PRODUCT_HOME>/repository/deployment/server/inputeventadaptors/ directory. Since hot deployment is enabled, you can simply add/remove files to deploy/undeploy to/from the server.  
    

Configuring input Kafka event adapter message properties

The configured input event adapters are used in Event Builders. With input event adapter message properties, event builders are able to extract data from the incoming events to build the event streams. Message properties specific to input event adapters of the kafka type can be configured as follows.

1. Log into the CEP Management Console. For detailed instructions, see Running the Product.
2. In the Main tab, click Event Streams to open the Available Event Streams page.

3. In the Main tab, click Event Streams to open the Available Event Streams page.

4. Click the Inflows link of the event stream to which the relevant input event adapter is connected. As a result, Event In-Flows (<Input_Event_Adapter_Name>) opens.

5. Click on the relevant event builder name to open the Event Builder Details page.
   

   The message properties of a kafka input event adapter are described below.

   Property	Description
   Topic Kafka	Name of the Kafka topic to which, input messages are published.
   Input Mapping Type	The incoming event type to be mapped.