Troubleshooting Event Receivers

The possible exceptions relating to event receivers that may occur are as follows:

Queue full exception

Sample Error log	event for endpoint group [ ( Receiver URL : tcp://das-1.-prod.local:7611, Authentication URL : ssl://das-1.amway-prod.local:7711),( Receiver URL : tcp://das-2.amway-prod.local:7611, Authentication URL : ssl://das-2.-prod.local:7711) ], 139882 events dropped so far. {org.wso2.carbon.databridge.agent.DataPublisher} TID: [-1] [] [2017-05-23 00:05:53,708] ERROR {org.wso2.carbon.databridge.agent.endpoint.DataEndpoint} - Unable to send events to the endpoint. {org.wso2.carbon.databridge.agent.endpoint.DataEndpoint} org.wso2.carbon.databridge.agent.exception.DataEndpointException: Cannot send Events
Occurrence	This occurs on the client side when using the WSO2Event event receiver.
Possible reasons	The TCP connection between the databridge client and the DAS server may not be established. The performance of WSO2 DAS may not be tuned to handle the event load sent by the databridge client. If the database is an RDBMS database such as MySQL, this exception can occur due to limited capacity.
Troubleshooting options	To check the TCP connection, enable event tracing and event logs, and make sure that one or more events are published to WSO2 DAS. To enable event tracing, see Event Tracer. To enable event logging, see Logger Event Publisher.
Recommended action	To check the database persistence and database receiver rate, the following properties can be set when the DAS server is started: `-DprofileReceiver=true`:This enables you to check the throughput per receiver. This creates the `receiver-perf.txt` file in the `DAS_HOME`. The throughput is calculated and published in this file for every 100000 events. `-DreceiverStatsCutoff=2000`: This specifies the number of events for which the `receiver-perf.txt` file generated via the `-DprofileReceiver` property is updated. e.g., If 2000 is specified, new statistics are inserted into the `receiver-perf.txt` file for every batch of 2000 events received via the databridge agent. The default value is 100000 events. This property is only available for this WSO2 Data Analytics Server version as a WUM update effective from 19th November 2017. For more information about updating WSO2 Data Analytics Server, see Updating WSO2 Products. `-DprofilePersistence=true`: This allows you to check the throughput at the persistence (i.e., Data Access Layer) level. This creates the `persistence-perf.txt` file in the `DAS_HOME`. `-DpersistenceStatsCutoff=1000`: This property specifies the number of events for which the `persistence-perf.txt` file generated via the `-DprofilePersistence` property is updated. If this property is not set, 100000 is considered the default number. NoSQL databases such as HBase are recommended for high throughput environments to avoid this exception.

Dropping incorrectly formatted events

Exception	wso2carbon.log:114:TID: [-1234] [] [2017-03-31 00:00:01,155] ERROR {org.wso2.carbon.databridge.core.internal.queue.QueueWorker} - Dropping wrongly formatted event sent for -1234 {org.wso2.carbon.databridge.core.internal.queue.QueueWorker} wso2carbon.log-115-org.wso2.carbon.databridge.core.exception.EventConversionException: Error when converting 1 of event bundle with events 6 wso2carbon.log-116- at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:181)
Occurrence	This may occur when you publish events to WSO2 DAS before or after starting the DAS server.
Possible reasons	If this exception occurs for events that were sent before the DAS server start-up, it can be due to a delay in the deployment of one or more artifacts deployed via CAR files. If this exception occurs for events that are sent after the DAS server start-up, it is because the relevant event stream is not defined or deployed in WSO2 DAS. For more information about event streams, see Understanding Event Streams and Event Tables.
Troubleshooting options	Go to the `<DAS_HOME>/repository/deployment/server/eventstreams` directory and check whether the required event stream definition exists.
Recommended action	Ensure that the required event streams are defined/deployed.

Enabling debugging logs allow you to further monitor event receivers and prevent exceptions. For more information, see Debugging Event Receivers.