...
| Sample Error log | | Code Block |
|---|
event for endpoint group [ ( Receiver URL : tcp://das-1.-prod.local:7611, Authentication URL : ssl://das-1.amway-prod.local:7711),( Receiver URL : tcp://das-2.amway-prod.local:7611, Authentication URL : ssl://das-2.-prod.local:7711) ], 139882 events dropped so far. {org.wso2.carbon.databridge.agent.DataPublisher}
TID: [-1] [] [2017-05-23 00:05:53,708] ERROR
{org.wso2.carbon.databridge.agent.endpoint.DataEndpoint} - Unable to send events to the endpoint. {org.wso2.carbon.databridge.agent.endpoint.DataEndpoint}
org.wso2.carbon.databridge.agent.exception.DataEndpointException: Cannot send Events |
|
|---|
| Occurrence | This occurs on the client side when using the WSO2Event event receiver. |
|---|
| Possible reasons | - The TCP connection between the databridge client and the DAS server may not be established.
- The performance of WSO2 DAS server may not be tuned unresponsive because it is unable to handle the event load sent by the databridge client.If due to resource constraints, or because it is not tuned properly. The insufficiently allocated resources may include CPU, memory, disk space, network bandwiidth, DB capacity etc.
- The database may be unable to receive the data fast enough, resulting in it being a bottleneck. This can happen if the database is an RDBMS database such as MySQL, this exception can occur due to limited capacity.
|
|---|
| Troubleshooting options | To check the TCP connection, enable event tracing and event logs, and make sure that one or more events are published to received by WSO2 DAS. To log all the events received via WSO2Event event receivers deployed in DAS, add the following logger to the <DAS_HOME>/repository/conf/log4j.properties file.
org.wso2.carbon.event.input.adapter.wso2event.internal.ds.WSO2EventAdapterServiceDS=DEBUG For more information, see the following topics: |
|---|
| Recommended action | - To check the database persistence and database receiver rate, the following properties can be set when the DAS server is started (e.g.,
./bin/wso2server.sh -DprofileReceiver=true):-
-DprofileReceiver=true :This enables you to check the throughput per receiver. This creates the receiver-perf.txt file in the DAS_HOME. The throughput is calculated and published in this file for every 100000 events. -
-DreceiverStatsCutoff=2000 : This specifies the number of events for which the receiver-perf.txt file generated via the -DprofileReceiver property is updated. e.g., If 2000 is specified, new statistics are inserted into the r eceiver-perf.txt file for every batch of 2000 events received via the databridge agent. The default value is 100000 events. -
-DprofilePersistence=true : This allows you to check the throughput at the persistence (i.e., Data Access Layer) level. This creates the persistence-perf.txt file in the DAS_HOME . -
-DpersistenceStatsCutoff=1000 : This property specifies the number of events for which the persistence-perf.txt file generated via the -DprofilePersistence property is updated. If this property is not set, 100000 is considered the default number.
- NoSQL databases such as HBase are recommended for high throughput environments to avoid this exception.
|
|---|
...
| Exception | | Code Block |
|---|
wso2carbon.log:114:TID: [-1234] [] [2017-03-31 00:00:01,155] ERROR {org.wso2.carbon.databridge.core.internal.queue.QueueWorker} - Dropping wrongly formatted event sent for -1234 {org.wso2.carbon.databridge.core.internal.queue.QueueWorker}
wso2carbon.log-115-org.wso2.carbon.databridge.core.exception.EventConversionException: Error when converting 1 of event bundle with events 6
wso2carbon.log-116- at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:181) |
|
|---|
| Occurrence | This may occur when you publish events to WSO2 DAS before or after starting the server startup of DAS serveris completed. |
|---|
| Possible reasons | - If this exception occurs for events that were sent before the DAS server start-up, it can be due to a delay in the deployment of one or more artifacts deployed via CAR files.
- If this exception occurs for events that are sent after the DAS server start-up, it is because the relevant event stream is not defined or deployed in WSO2 DAS. For more information about event streams, see Understanding Event Streams and Event Tables.
|
|---|
| Troubleshooting options | Ensure that - Check whether the required event stream is included in the list of event streams displayed in the Available Event Streams page. To access this page, access the DAS Management Console, go to the Main tab and click Streams in the left navigator. For more information, see Understanding Event Streams and Event Tables .
- Go to the
<DAS_HOME>/repository/deployment/server/eventstreams directory and check whether the required event stream definition exists.
| Recommended action | - Check the startup log to see whether the required event
|
|---|
Troubleshooting WSO2Event Receivers using databridge debug logs
org.wso2.carbon.databridge.core.internal.queue.QueueWorker provides debug logs which can be enabled in DAS management console.
To enable databridge debug logs, follow the steps given below.
- Go to
Home → Configure → Logging - Filter logger by 'org.wso2.carbon.databridge.core.internal.queue'
- Set log level to '
DEBUG'
Under above configuration, following information will be logged.
...
| Code Block |
|---|
<size of event queue> + " messages in queue before " + <name of the current thread> + " worker has polled queue ". |
...
| Code Block |
|---|
"Dispatching event to " + <number of subscribers> + " subscriber(s) ". |
...
- stream is successfully deployed as shown in the following example.
 Image Added
|
| Recommended action | - If the stream is not deployed, redeploy the stream or the car file that contains it. For more information, see Packaging Artifacts s a C-App Archive.
- Define the required event stream. For more information, see Understanding Event Streams and Event Tables.
To prevent the DAS server from receiving events before event streams are deployed, add the following property to the <dataReceiver name="Thrift"> configuration in the <DAS_SERVER>/repository/conf/data-bridge/data-bridge-config.xml file.
<config name="waitingTimeInMilliSeconds">{waitingTime}</config> | Info |
|---|
You need to restart the DAS server after adding this configuration for it to be effective. |
|
|---|
| Tip |
|---|
Enabling debugging logs allow you to further monitor event receivers and prevent exceptions. For more information, see Debugging Event Receivers. |