Analytics Data Backup / Restore Tool

The analytics data backup / restore tool can be used to backup the already existing record store and file system of a DAS server to a specific directory in our machine. The backed up data can be restored later from the same directory to the current DAS node.

The following script is used to backup and restore data.

On Windows: <PRODUCT_HOME>\bin\analytics-backup.bat --run
On Linux/Solaris/Mac OS: <PRODUCT_HOME>/bin/analytics-backup.sh

The following table describes the arguments that are used with the backup script mentioned above.

Argument	Purpose	Examples
`-backupRecordStore`	To backup the record store where the persisted events are saved. This	`./analytics-backup.sh -backupRecordStore -dir /home/user/backup` The above command backs up all the tables in the DAS record store to the `/home/user/backup` directory.
`-backupFileSystem`	To backup the file system where data relating to the indexing of events.	`./analytics-backup.sh -backupFileSystem -dir /home/user/backup` The above command backs up indexing information relating to all the tables in DAS to the `/home/user/backup` directory.
`-dir <directory>`	Directory used as the target when backing up the record store/file system, or as the source when restoring already backed up record store/file system. The directly should be specified in every command issued to backup or restore data.	`-dir /home/user/backup` The above argument in all the other examples specifies the `/home/user/backup` directory as the location where backed up data should be stored or as the source from which data should be restored.
`-restoreRecordStore`	To restore a record store. Before you use this command, the record store you want to restore should be already backed up.	`./analytics-backup.sh -restoreRecordStore -dir /home/user/backup` The above command restores record store data currently saved in the `/home/user/backup` directory to the current DAS node.
`-restoreFileSystem`	To restore a file system. Before you use this command, the file system you want to restore should be already backed up.	`./analytics-backup.sh -restoreFileSystem -dir /home/user/backup` The above command restores indexing related data currently saved in the `/home/user/backup` directory to the current DAS node.
-deleteTables	Deletes the specified table list. You need to specify the table list to be deleted using the -tables parameter.	`./analytics-backup.sh -deleteTable -tables "tempTable" -tenantId -1234`
-purge	Allows data purging for a given time range.	`./analytics-backup.sh -purge -table "tempTable" -tenantId -1234 -timeFrom 15-10-11-12:40:00 -timeTo 15-10-15-17:00:00`
`-enableIndexing`	Indexing is disabled in the data restoration step by default. This is because if the target DAS server is already running, that server will also index the same data causing an index corruption. If you are certain that the target DAS server is not running, you can index the data at the time it is restored by using this argument. If this argument is not used while the target DAS server is not running, the data restored from the time the target server is started up is indexed in the usual way.	`./analytics-backup.sh -restoreRecordStore -restoreFileSystem -enableIndexing -dir /home/user/backup` The above command restores both record store data and indexing related data currently saved in the `/home/user/backup` directory to the current DAS node, and indexes the data at the time of restoration.
`-reindexEvents`	This switch will make the tool re-index the data already there in a table. This can be useful in a scenario where an the indexing is corrupted for some reason, or if some of the older data is not indexed because it was not mentioned in the schema etc..	`./analytics-backup.sh -restoreRecordStore -restoreFileSystem -reindexEvents -dir /home/user/backup` The above command restores both record store data and indexing related data currently saved in the `/home/user/backup` directory to the current DAS node, and reindexes the event at the time of restoration.
`-tables <table list>`	This argument is used to specify the list of event tables that should be backed up or restored. If you do not use this argument to back up data from one or more specific tables, all the available event tables will be backed up.	`/analytics-backup.sh -backupRecordStore -tables Table1,Table2,Table3 -dir /home/user/backup` The above command backs up the data of `Table1`, `Table2`, `Table3` event tables to the `/home/user/backup` directory. `./analytics-backup.sh -restoreRecordStore -tables Table1,Table2,Table3 -dir /home/user/backup` The above command restores record store data of `Table1`, `Table2`, `Table3` event tables that are currently saved in the `/home/user/backup` directory to the current DAS node.
`-tenantId <tenant id (default is super tenant)>`	This argument is used to select the tenant ID(s) of which the events should be backed up.	`./analytics-backup.sh -backupRecordStore -tenantId abc -dir /home/user/backup` The above command backs up all the event tables of the `abc` tenant from the DAS record store.
`-disableStaging`	This argument is used to disable staging. Staging involves including the indication whether the data is indexed or not when backing up/restoring the DAS record store. This task incurs a greater system overhead. Therefore, it is allowed to disable staging when the number of records being backed up/restored is high. Backing up and restoring the file system via `-backupFileSystem` and `-restoreFileSystem` allows you to preserve the indexing which would enable searches to be carried out on restored data.	`./analytics-backup.sh -backupRecordStore -disableStaging` The above command backs up all the event tables in the DAS record store. Indications of whether data is indexed or not are removed before backing up the records.
`-timeFrom <yy-mm-dd-hh:mm:ss>`	This argument specifies the starting time (inclusive) when defining the time period that should be considered when backing up/restoring events. This argument should be used together with the `-timeto` argument.	`./analytics-backup.sh -backupRecordStore -timeFrom 15-10-11-12:40:00 -timeTo 15-10-15-17:00:00 -dir /home/user/backup` The above command backs up all the data in the DAS record store created between 12.45PM on 11th October 2015 to 5.00PM on 15th October 2015. The data is backed up in the `/home/user/backup` directory.
`-timeTo <yy-mm-dd-hh:mm:ss>`	This argument specifies the ending time (inclusive) when defining the time period that should be considered when backing up/restoring events. This argument should be used together with the `-timefrom` argument.	`./analytics-backup.sh -backupRecordStore -timefrom 15-10-11-12-40-00 -timeto 15-10-15-17-00-00 -dir /home/user/backup` The above command backs up all the data in the DAS record store created between 12.45PM on 11th October 2015 to 5.00PM on 15th October 2015. The data is backed up in the `/home/user/backup` directory.

All the arguments described in the above table are displayed in your console when you run the <PRODUCT_HOME>\bin\analytics-backup.bat --run or the <PRODUCT_HOME>/bin/analytics-backup.sh without adding any argument.

Sample commands

Backing up data

./analytics-backup.sh -restoreRecordStore -tables Table1,Table2,Table3 -backupFileSystem -timeFrom 15-08-11-08:00:00 -timeTo 15-10-15-22:30:00 -dir /home/user/backup

The above command backs up the records of the Table1, Table2, Table3 event tables as well as the related indexing information in the /home/user/backup directory. Only the data entered between 8.00 AM on 11th August 2015 and 10.30 PM on 15th October 2015 are selected to be backed up.

Restoring data

./analytics-backup.sh -restoreRecordStore -tables Table1,Table2,Table3 -restoreRecordStore -timeFrom 15-08-11-09:00:00 -timeTo 15-08-11-10:00:00 -dir /home/user/backup

The above command restores records of the Table1, Table2, Table3 event tables as well as the related indexing information from the /home/user/backup directory. Only data entered between 9.00 - 10.00 AM on 11th August 2015 are selected to be restored.

Searching restored data

In order to enable searching for persisted data, WSO2 indexes events using Apache Lucene (for more information, see Configuring Indexes). The persisted data is saved in the record store and the information relating to the indices of that data is stored in the file system. Therefore, if you want to carry out searches once you restore backed up data, you need to restore both the record store and the file system. If you restore only the record store it will not be possible to carry out searches for the restored data using the Data Explorer. For more information on searching using the Data Explorer, see Searching Data by Categories.