Apache Spark is used as the core analytics engine in DAS 3.0.0. For information on writing Spark queries to analyze the collected data, see Data analysis using SQL. see Spark Query Language.
| Anchor | ||||
|---|---|---|---|---|
|
...
Analytics scripts
Spark Analytics scripts are used when you have to execute a set of Spark queries in a sequence. Also, you can schedule a Spark a Analytics script, to trigger it to execute the query automatically in a given period of time. (E.g. fire at 12 (noon) every day, or fire at every minute starting at 2 p.m. and ending at 2:59 p.m. every day etc.). You need to configure this scheduled time using a cron expression. For more information about cron expressions, see Cron Trigger Tutorial.
...
Follow the steps below to add a new Spark script.
- Log in to the WSO2 DAS management consoleManagement Console.
- Click In the Main tab, and then click Add in the Scripts menu as shown below.
Enter the following details related click Scripts to open the Available Analytics Scripts page. Click Add New Analytics Script to open the Add New Analytics Script page. Then enter the following details related to your script as shown in the example below.
Script Name MyFirstSparkScriptMyFirstAnalyticsScript Spark SQL Queries define table Log (server_name string, ip STRING, tenant INTEGER, sequence LONG, summary STRING);
SELECT ip FROM Log;
SELECT server_name, count(*) FROM Log GROUP BY server_name;
SELECT COUNT(*) FROM Log WHERE summary LIKE '%Joe%';
SELECT substr(summary, 1, 5) FROM Log;
SELECT LAST(ip) FROM Log;
Cron Expression 0 * * * * ?
Info This cron expression defines the schedule time of the script to execute it in every minute. From the time you save the script, the script will be executed at the beginning of every minute. (E.g:.10:21:00, 10:22:00, 10:23:00,..)
- Click Execute, to execute the provided queries. This will display the results as follows.
- Click Add, to add the configured script.
...
Follow the steps below to add a new Spark edit an existing Analytics script.
- Log in to the WSO2 DAS management consoleManagement Console.
- Click In the Main tab, and then click Scripts in the Spark menuclick Scripts to open the Available Analytics Scripts page.
Click List, and click the Edit link of the corresponding script as shown below.
Edit for the script you want to edit.
Change the content of the script as required. You can update the scheduling information as well.
Note When you do not enter any value for the scheduling time, then your script is not scheduled to execute. However, if you want to ensure that your script is valid, click Execute. This will execute the queries that you give in the queries window.
For example, you can edit the script created above to unschedule the scheduled time as shown below.
- Click Update to save the changes as shown above.
...
Follow the steps below to delete a Spark an Analytics script.
- Log in to the WSO2 DAS management consoleManagement Console.
- Click In the Main tab, and then click Scripts in the Spark menuclick Scripts to open the Available Analytics Scripts page.
- Click List, and click the Delete link of the corresponding script as shown below.
Delete for the script you want to delete.
Click Yes in the dialog box which appears to confirm deletion.
Note If you delete the script you cannot undo that operation, and it will be completely removed from the system. Also, deleting the script will delete the scheduled task associated with it.
...
Follow the steps below to execute the script content.
- Log in to the WSO2 DAS management consoleManagement Console.
- Click In the Main tab, and then click Scripts in the Spark menuclick Scripts to open the Available Analytics Scripts page.
- Click List, and click the Execute link of the corresponding script as shown below.
Execute for the script you want to execute.
- Now, the script execution job is immediately dispatched to Spark engine. It will display the results once the job is completed as shown below.
