...
| Sample Error Log | | Code Block |
|---|
TID: [-1] [] [2018-02-13 04:49:40,345] ERROR {org.wso2.carbon.analytics.dataservice.core.indexing.IndexNodeCoordinator} - Error in processing staging index data: Error in processing index staging entry: [-1234:NEW_CUSTOMERS_BY_QUARTER_PRODUCT_AND_POD] does not exist {org.wso2.carbon.analytics.dataservice.core.indexing.IndexNodeCoordinator}
org.wso2.carbon.analytics.datasource.commons.exception.AnalyticsException: Error in processing index staging entry: [-1234:NEW_CUSTOMERS_BY_QUARTER_PRODUCT_AND_POD] does not exist
at org.wso2.carbon.analytics.dataservice.core.indexing.IndexNodeCoordinator.processStagingEntry(IndexNodeCoordinator.java:961)
at org.wso2.carbon.analytics.dataservice.core.indexing.IndexNodeCoordinator.access$600(IndexNodeCoordinator.java:69)
at org.wso2.carbon.analytics.dataservice.core.indexing.IndexNodeCoordinator$StagingDataIndexWorker.run(IndexNodeCoordinator.java:997)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) |
|
|---|
| Occurence | When the INSERT OVERWRITE operation is carried out. |
|---|
| Possible Reasons | The corruption of meta information and indexing that could have resulted from one of the following: - Deployments or the removal of deployments in the nodes of the DAS setup.
- Configuration changes in the DAS nodes of the setup/.
- Creating/deleting tables in the event store or the process store.
|
|---|
| Recommended Action | To resolve this issue, reindex your data following the steps below: - Shut down all the DAS nodes in your deployment.
Truncate the following tables. | Info |
|---|
Table names are encoded in characters that cannot be read by humans for all database types other than HBase and Cassandra. Therefore, if the database type is not HBase or Cassandra, you need to use the encoded versions of these table names depending on the database type when performing this step. |
ANX_X1000__INDEX_STAGING_DATA_*IDX_X1000__INDEX_STAGING_DATA_*
Drop all the tables with names that start with ANX and IDX such as the examples given below. | Info |
|---|
Table names are encoded in characters that cannot be read by humans for all database types other than HBase and Cassandra. Therefore, this step is relevant only if the database type used is HBase or Cassandra because the table names given below can only exist for ese these two database types. |
ANX_X1000__INDEX_STAGING_DATA_*IDX_X1000__INDEX_STAGING_DATA_*
- For each node, remove all the index data stored in the
<DAS_HOME>/repository/data directory. - In the
<DAS_HOME>/repository/conf/analytics/local-shard-allocation-config.conf file, change the mode for all the shards from NORMAL to INIT. This must be done for each node. - Restart the DAS nodes.
|
|---|
Out Of Memory
...
Image Removed
...
Issues relating to the disk space
| Occurence | Disk space related issues occur when a high volume of data is stored in the databases. |
|---|
| Possible Reasons | The indexing operation of the bulk of data handled maybe using almost all the available memory based on the taxanomy cache size specified in the <DAS_HOME>/repository/conf/analytics/analytics-config.xml file. e.g., If the available memory is 3.6 GB and 2.9 GB of that is used by the indexing operation as shown in the sample given above.
 Image Removed Tables with the ANX__8 prefix contain a huge amount of data and it takes the majority of the DB size. |
|---|
| Recommended Action | To overcome resolve this out of memory issue, increase the taxonomy cache size reindex your data by following the procedure given steps below.: - Install the latest WUM update.
- Add the following configuration under
analytics-dataservice-configuration in Shut down all the DAS nodes in your deployment. - Delete all the contents of the
<DAS_HOME>/repository/conf/analytics/analyticsHOME>repository/data directory except the local-shard-allocation-config.xml file (if it does not already exist).| Info |
|---|
When ths configuration is not added in the analytics-config.xml file, the default values shown below applyconf file. - In the
WSO2_ANALYTICS_EVENT_STORE_DB database, delete all the tables with the ANX__8 prefix. Open the <DAS_HOME>repository/data/local-shard-allocation-config.conf file, and change the status of all the shards from NORMAL top INIT, so that the content looks as follows. Restart the server| Code Block |
|---|
| <taxonomy-writer-cache>
<cacheType>DEFAULT</cacheType>
<!--<LRUType>STRING</LRUType>-->
<!--<cacheSize>4096</cacheSize>-->
</taxonomy-writer-cache> |
To save the large taxanomy indexes in the file system of DAS instead of in the memory, change the value of the cacheType parameter to LRU as shown below.
<cacheType>LRU</cacheType> | Info |
|---|
When the value specified for the cacheType parameter is DEFAULT, taxanomy indexes are saved in the DAS memory. |
Uncomment the LRUType parameter.
| Info |
|---|
The LRU type can be STRING or HASHED. The STRING value ensures that indexing is correctly done without collision. The HASHED value does not ensure the same, but it uses less RAM. |
- Uncomment the
cacheSize parameter. Then increase the value for this parameter as required. | 0,INIT
1,INIT
2,INIT
3,INIT
4,INIT
5,INIT |
| Tip |
|---|
If you find any other content in the local-shard-allocation-config.conf file, remove it. |
- Restart the DAS nodes. Once the re-indexing is completed, the status of the shards in the
local-shard-allocation-config.conf file changes from INIT to NORMAL.
|
|---|
For more information about indexing, see Indexing Overview.