Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This option allows you to create servers that are dedicated for specific operations such as event receiving, analytics, indexing, etc.

All index data are stored in the file system, partitioned into unit is known as shards. For detailed information about shard configuration and allocation, see Storing Index Data.

Configuring common parameters

The following parameters are common for both the Analytics Record Store and the Analytics File System

Data purging parameters

ParameterDescriptionDefault Value
<purging-enable>This parameter specifies whether the functionality to purge data from event tables is enabled or not.false
<cron-expression>A regex expression to select the tables from which data should be purged.0 0 0 * * ?
<purge-include-tables>A list of event tables from which data should be purged can be defined as subelements of this element. 
<data-retention-days>The number of days for which the data should be retained in the event tables that were selected to have their data purged. All the data in these tables are cleared once the number of days that equal the value specified for this parameter have elapsed.365

 

Other Parameters

The number of index shards that are allowed to exist in the local file system at a given time. The number specified should be higher than the number of indexing nodes in the DAS cluster. The ideal number can be calculated as follows.

number of indexing nodes * [CPU cores used for indexing per node]

ParameterDescriptionDefault Value
<analytics-lucene-analyzer>

The implementation of the Analytics Lucene Analyzer is defined as a subelement of this parameter.

e.g.,  <implementation>org.apache.lucene.analysis.standard.StandardAnalyzer</implementation>

 
<indexReplicationFactor>The index data replication factor to be used in clustered mode. This tells how many replicas that should be kept when indexing operations are done. 0 means, no replication. It should be set to 1 or higher to have high availability.1
<shardCount>
Note

This parameter can only be set once for the lifetime of the cluster, and cannot be changed later on.

6<shardIndexRecordBatchSize>The amount of index data (in bytes) to be processed at a time by a shard index worker. Minimum value is 1000.20971520

<shardIndexWorkerInterval>

The interval in milliseconds during which a shard index processing worker thread will sleep during index processing operations. This parameter together with the shardIndexRecordBatchSize parameter can be used to increase the final index batched data amount processed by the indexer at a given time. Usually, a higher batch data amount results in a higher throughput of the indexing operations. However, it also causes an increased latency of record insertion to indexing. Minimum value of this is 10, and a maximum value is 60000 (1 minute).

1500