ESB Analytics stores data and performs various analysis operations on them according to defned defined analytic queries. Thereby, as the volume of the data stored grows over time, the analysis and summarization jobs will also consume more time. Hence, purging data allows you to increase the performance of ESB Analytics by reducing the time taken to execute the analytics scripts as well as optimising optimizing the disk usage.
| Warning |
|---|
Decide on a retention period based on the number of days you need to retain data for analysis and purge data that are older than that in the below tables. |
ORG_WSO2_ESB_ANALYTICS_STREAM_EVENTORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECONDORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECONDALLORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTEORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTEALLORG_WSO2_ESB_ANALYTICS_STREAM_STATPERHOURORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMONTHORG_WSO2_ESB_ANALYTICS_STREAM_STATPERDAYORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERSECONDORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMINUTEORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERHOURORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERDAYORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMONTH
There are two three ways to purge data in ESB Analytics:
| Table of Contents | ||||
|---|---|---|---|---|
|
Using the Carbon App
For EI versions 6.1.0-6.3.0, purging analytics data can be managed via the org.wso2.analytics.ei.realtime Carbon application. This application is deployed by default if you get the WUM update timestamp: 1565079455315.
For more information about WUM-updating WSO2 products, see Getting Continuous Updates.
Using the per-table method
...
- Open the
<EI_HOME>/wso2/analytics/conf/analytics/analytics-config.xmlfile. Change the content under the
<analytics-data-purging>property as shown below:Note Change the value of the
<data-retention-days>property as per your requirement.Code Block language xml <analytics-data-purging> <purging-enable>true</purging-enable> <cron-expression>0 0 12 * * ?</cron-expression> <purge-include-tables> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_EVENT</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECOND</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERSECONDALL</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTE</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMINUTEALL</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERHOUR</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERMONTH</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_STATPERDAY</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERSECOND</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMINUTE</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERHOUR</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERDAY</table> <table>ORG_WSO2_ESB_ANALYTICS_STREAM_MEDIATORSTATPERMONTH</table> </purge-include-tables> <data-retention-days>2</data-retention-days> </analytics-data-purging>The properties of the above configuration file are shown below.
Property Description <purging-enable>
Change the value to trueif you want to enable data purging.<cron-expression>
The cron expression to define how you want to schedule the data purging operation. For example, the following cron expression will configure the archive job to run at 12:00 PM (noon) every day : 0 0 12 * * ?For more information on cron expressions, go to Oracle Documentation.<purge-include-table-patterns>
Specify the tables of which you want to purge data. By default, it is configured to perform data purging on all tables as follows: <table>.*</table>However, you can specify the required tables by defining a regular expression or a table name within the<table>property. Define one tag per each regular expression if you want to specify multiple tables.<data-retention-days>
Define the value as to keep data of only the last 'n' no of days back in the selected table. For example, the default value 365 will purge all data stored before 1 year.
Info You can purge all records by setting a minus value (e.g. -1) for
<data-retention-days>.- Save your changes.