WSO2 DAS has a distributed indexing engine which that is built on top of Apache Lucene. Data is indexed and saved in the Data Access Layer of DAS in a store referred to as the Analytics File System. This section explains how indexing used is carried out in WSO2 DAS.
A main requirement of DAS is to support arbitrary key value pairs. Most of the RDBMS have a fixed schema and does do not support storing arbitrary key values. In DAS, records are stored as blobs. In that This way, the data fields in a record can be converted to a blob and store them along stored together with the record id ID and the timestamp of the record. Storing data as blobs has one drawback. That is database level indexing cannot be used for blobs to search by fields. To overcome this issue, before records are sent through the indexer before the records are persisted in DAS , records are sent through the Indexer, so so that the indexer can keep a searchable copy of record data along with the record idID. This indexer does not store the field values, it . It only stores the record id. It only keep ID and keeps an index of data fields. When a search query is sent to DAS, the indexer will look for identifies the matching records, get gets their record idsIDs, send and sends them to the persistence layer/recordstore. In recordstore level, the record blobs are deserialized and return returned as the record objects from the AnalyticsDataService implementation.
Indexing Architecture
...