Versions Compared

Old Version 16

changes.mady.by.user Former user

Saved on

compared with

New Version 17

changes.mady.by.user Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

You can troubleshoot and trace possible errors that can occur with WSO2 Message Broker in a given environment by using the methods given below.

...

 

TableDescription

MB_SLOT

Each slot, the assigned node ID and the current status are maintained here.

MB_SLOT_MESSAGE_ID

Whenever a node communicates a possible slot range to the coordinator, the node will decide on the appropriate message ID range to be included in the slot and update this table with the last endMessageID submitted by the node for a given queue/topic.

MB_NODE_TO_LAST_PUBLISHED_ID

This table contains the last published message ID for each node in order to calculate the global safe zone (minimum messageID from all nodes) that is required for deleting slots upon completion.

MB_QUEUE_TO_LAST_ASSIGNED_ID

Whenever a slot is given by the coordinator to an MB node for processing, its endMessageID is updated in this table against the destination name.

Within this context, you can infer the following validations in the database at any given time:

  1. There should not be any slots in the MB_SLOT table if the MB_METADATA table is empty. This is an eventual guarantee. Even if there are slots queued for deletion, this rule must still be satisfied after some time.

  2. There should be no records in the MB_METADATA table if the MB_CONTENT table is empty (one-to-one relationship).

  3. Given the minimum message ID in the MB_NODE_TO_LAST_PUBLISHED_ID table, all slots within the MB_SLOT table with the “assigned” status (state = 2) and the endMessageID less than the minimum published ID should be deleted (or at-least be cleared after some time).

Retrieving logs from the JMS client

You can simply monitor the logs from the JMS clients connecting to WSO2 MB by enabling the following startup property on the clients:

Code Block
-Damqj.protocol.logging.level=true

Monitoring JAVA metrics

The metrics dashboard of WSO2 MB provides general JVM metrics as well as MB-specific metrics to help you identify how the broker is running in a loaded/relaxed environment. This functionality will give you information such as the unexpected increases of delivery channels, latencies of database reads/writes etc., which will help you identify possible errors in the system. See the documentation on metrics for instructions on how to configure and use the metrics dashboard.

Identifying common warnings/logs

The following table details some of the most common warning messages/logs that can be encountered when working with WSO2 MB. You will also find here the possible causes and solutions for such warnings/logs.

Warning

Cause

Solutions / Approach

[WARN] Invalid message state transition from <state1>
suggested: <state2>  Message ID: 93293291982

This means that the message lifecycle has deviated from the expected execution path. Example: getting SENT -> SCHEDULED_TO_SEND.

First realize , distinguish the expected execution order through class using the “org.wso2.andes.kernel.MessageStatus”[11] class and then try to pinpoint identify the exact phase point at which the issue occurs. (High occurred. There is a high probability of race conditions.)

[WARN] Invalid State transition from <stateA> suggested : <stateB> Slot ID : MyQueue|22309482...

This means that the slot lifecycle (used for delivery) has deviated from the expected execution path. Example: CREATED -> DELETED.

First realize , distinguish the expected execution order through class using the “org.wso2.andes.kernel.SlotState” [12]class and then try to pinpoint identify the exact phase point at which the issue occurs. (High occurred. There is a high probability of race conditions.)

[WARN] Error when trying to read property <property>. Switching to default value : <defaultValue>

This can happen if the MB configuration file (broker.xml) is not up to date, or if it has been edited.

Make Be sure that the correct version of the broker.xml file is within the pack.

[INFO] Local subscription ADDED [TestQueue]ID=635@NODE/10.100.5.115:4000/T=1456518837302/D=true/X=false/O=null/E=
amq.direct/ET=org.wso2.andes.server.exchange.DirectExchange$1@2db707df/EUD=0/S=true {org.wso2.andes.subscription.SubscriptionStore}

This log is printed whenever a new queue/topic subscription is added to the cluster.

Lifecycle The lifecycle of a subscription is ADDED -> DELETED. In case of durable topic subscriptions, a subscription may be disconnected before being deleted (. If a subscription is disconnected, its the messages will still persist until the subscriber is deleted).

*Please note Note that this log is not printed if a durable subscription is disconnected and added for the second time after the cluster startup (Since it’s only a state update in terms of durable topics)starts.

Info Level log.

Channel created (ID: 21765) {org.wso2.andes.kernel.AndesChannel}

This log is printed every time a publish/subscribe channel is established with between the client and the broker node. An Andes channel is mapped (one-to-one) to an AMQChannel or an MQTTPublisherChannel.

Info level log.